Skip to content

Commit 0f8c676

Browse files
committed
rootfs cache: refresh configng every build; fingerprint tier + hash into cache_type
Two bugs that conspired to make 'new configng commit but still served the old rootfs from cache' the default behaviour. 1. The fetch_from_repo for armbian-configng was gated inside the '[[ -z $DESKTOP_ENVIRONMENT ]]' branch of interactive_desktop_main_configuration, so it only fired when the user picked a DE from the interactive dialog. Every non-interactive build (CI, items-from-inventory, scripted local with DESKTOP_ENVIRONMENT= pre-set) skipped the fetch — which left whatever stale clone happened to live in cache/sources/armbian-configng/ as the source of truth for the CONFIGNG_DESKTOPS_HASH input. Hoist the fetch + parser-path sanity check up above the gate so every BUILD_DESKTOP=yes invocation pulls branch:main. fetch_from_repo is idempotent on an already-cloned tree (just a git pull) so the warm-cache cost is one up-to-date check. 2. The resolved rootfs tarball filename is rootfs-<arch>-<release>-<cache_type>_<yyyymm>-<cache_id>.tar.zst. cache_id = packages_hash came from AGGREGATED_ROOTFS_HASH + hook hash + bash hash — none of which see armbian-configng. cache_type encoded DESKTOP_ENVIRONMENT ('xfce-desktop', 'gnome-desktop', …) but not DESKTOP_TIER and not any fingerprint of the configng tree. Consequences: - A tier=minimal rootfs and a tier=full rootfs for the same DE collided on one filename. Whichever built first got cached; every subsequent tier silently reused its tarball. - A configng commit that changed which packages a DE installs (yaml edit, tier_override bump, new browser mapping) produced a different CONFIGNG_DESKTOPS_HASH in artifact_input_variables, but the local tarball was still named the same, so the lookup kept hitting the pre-change artifact. Fold both into cache_type. Net rename: rootfs-arm64-trixie-xfce-desktop_…tar.zst → rootfs-arm64-trixie-xfce-desktop-mid-a1b2c3d4_…tar.zst 8-char suffix is the configng commit prefix. Only injected for BUILD_DESKTOP=yes; only when the value is a real SHA (not the 'undetermined' / 'unknown' sentinels the computation falls back to on clone failures).
1 parent 465162d commit 0f8c676

2 files changed

Lines changed: 64 additions & 13 deletions

File tree

lib/functions/configuration/config-desktop.sh

Lines changed: 29 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -30,22 +30,38 @@ function interactive_desktop_main_configuration() {
3030

3131
display_alert "desktop-config" "DESKTOP_ENVIRONMENT entry: ${DESKTOP_ENVIRONMENT}" "debug"
3232

33+
# Refresh the armbian-configng clone on EVERY desktop build,
34+
# regardless of whether DESKTOP_ENVIRONMENT was pre-set. The
35+
# clone feeds two downstream consumers:
36+
#
37+
# 1. The interactive DE-selection dialog below — only fires
38+
# when DESKTOP_ENVIRONMENT is empty.
39+
# 2. artifact_rootfs_config_dump, which reads the clone's
40+
# `git log -1 -- tools/modules/desktops/` as the
41+
# CONFIGNG_DESKTOPS_HASH input that (via create-cache.sh's
42+
# cache_type) fingerprints the rootfs tarball filename.
43+
#
44+
# Keeping the fetch inside the `-z DESKTOP_ENVIRONMENT` branch
45+
# meant every non-interactive build (CI, items-from-inventory,
46+
# scripted local) skipped it — so the hash was computed against
47+
# whatever stale clone happened to be on disk and the build
48+
# happily cache-hit a pre-configng-change rootfs. Hoisting the
49+
# fetch up here makes the clone authoritative for every
50+
# BUILD_DESKTOP=yes invocation.
51+
fetch_from_repo "https://github.com/armbian/configng" "armbian-configng" "branch:main"
52+
53+
local configng_dir="${SRC}/cache/sources/armbian-configng"
54+
local yaml_dir="${configng_dir}/tools/modules/desktops/yaml"
55+
local parser="${configng_dir}/tools/modules/desktops/scripts/parse_desktop_yaml.py"
56+
57+
if [[ ! -f "${parser}" ]]; then
58+
exit_with_error "Desktop parser not found at ${parser}" \
59+
"armbian-config clone may be incomplete"
60+
fi
61+
3362
# --- DE selection ---
3463
if [[ -z $DESKTOP_ENVIRONMENT ]]; then
3564

36-
# Fetch armbian-config (configng) to get the YAML desktop
37-
# definitions and the standalone Python parser.
38-
fetch_from_repo "https://github.com/armbian/configng" "armbian-configng" "branch:main"
39-
40-
local configng_dir="${SRC}/cache/sources/armbian-configng"
41-
local yaml_dir="${configng_dir}/tools/modules/desktops/yaml"
42-
local parser="${configng_dir}/tools/modules/desktops/scripts/parse_desktop_yaml.py"
43-
44-
if [[ ! -f "${parser}" ]]; then
45-
exit_with_error "Desktop parser not found at ${parser}" \
46-
"armbian-config clone may be incomplete"
47-
fi
48-
4965
# EXPERT mode controls which editorial `status:` values are
5066
# surfaced in the dialog:
5167
# default: `status: supported` only

lib/functions/rootfs/create-cache.sh

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,41 @@ function calculate_rootfs_cache_id() {
3939
[[ -n ${DESKTOP_ENVIRONMENT} ]] && cache_type="${DESKTOP_ENVIRONMENT}-desktop"
4040
[[ ${BUILD_MINIMAL} == yes ]] && cache_type="minimal"
4141

42+
# Fold DESKTOP_TIER into the cache name for desktop builds.
43+
# tier=minimal, tier=mid and tier=full install different
44+
# package sets under the same DE, so they can't share a
45+
# cache tarball — otherwise the first tier to build wins and
46+
# every subsequent tier of that DE silently reuses its rootfs.
47+
if [[ ${BUILD_DESKTOP} == yes && -n ${DESKTOP_TIER} ]]; then
48+
cache_type="${cache_type}-${DESKTOP_TIER}"
49+
fi
50+
51+
# Fold a short fingerprint of the resolved armbian-configng
52+
# desktop tree into the cache name. The desktop package set is
53+
# chosen by configng YAMLs at rootfs-create time
54+
# (`armbian-config --api module_desktops install mode=build`),
55+
# not by the aggregation layer — so packages_hash and
56+
# AGGREGATED_ROOTFS_HASH are blind to configng entirely. Without
57+
# this, a configng commit that changes which packages a DE
58+
# installs leaves the existing rootfs tarball untouched in the
59+
# cache and every subsequent build cache-hits the pre-change
60+
# version.
61+
#
62+
# CONFIGNG_DESKTOPS_HASH is set by artifact_rootfs_config_dump
63+
# above from `git log -1 -- tools/modules/desktops/` in the
64+
# cache/sources/armbian-configng clone. 8-char prefix keeps
65+
# the filename readable while still giving us ~2^32 of
66+
# collision resistance — same truncation style the rest of
67+
# this file uses for the hash_hooks/bash_hash components.
68+
if [[ ${BUILD_DESKTOP} == yes ]]; then
69+
declare _configng_fp="${artifact_input_variables[CONFIGNG_DESKTOPS_HASH]:-}"
70+
if [[ -n "${_configng_fp}" \
71+
&& "${_configng_fp}" != "undetermined" \
72+
&& "${_configng_fp}" != "unknown" ]]; then
73+
cache_type="${cache_type}-${_configng_fp:0:8}"
74+
fi
75+
fi
76+
4277
# allow extensions to modify cache_type, since they may have used add_packages_to_rootfs() or remove_packages()
4378
cache_type="${cache_type}${EXTRA_ROOTFS_NAME:-""}"
4479

0 commit comments

Comments
 (0)