Compare commits

...
Sign in to create a new pull request.
Author SHA1 Message Date
2171f8ea3b
test(verify): follow redirects and bypass the edge cache
Two ways the harness lied about what it was measuring.

The router answers an index request with a redirect, and fetch() did not
pass -L, so a --live byte comparison compared against a 302 body rather
than the index a client ends up with.

A purge is asynchronous, so a run started right after a reindex read
whatever the edge still held. That produced a phantom '161 regressions'
on amd64/alpine323 whose PACKAGES.rds was already correct - the .gz was
simply a stale copy, and re-measuring after the purge settled showed
zero. A verification tool that races the thing it verifies is worse than
no tool.
2026-08-31 10:26:09 +00:00
ed131e19fa
test(verify): report uneven coverage instead of failing on it
Same mistake as the source-fallback share. The packages a non-primary
minor lacks are ABI-risky ones built under the slot's primary minor,
which a client on another minor cannot safely load: their absence is
correct, not a defect. Failing on it blocked resolute and alpine324 on
both arches while they regressed nobody.

Parity gates the claim we make about coverage; MAX_REGRESSIONS gates
whether routing is safe to enable. Only the second belongs in the exit
code.
2026-08-31 09:59:55 +00:00

View file

@ -43,10 +43,13 @@ MINORS=${MINORS:-"4.4 4.5 4.6"}
EXCLUDED_MINORS=${EXCLUDED_MINORS:-"4.3"}
# How many Path: targets to HEAD-check per slot/minor. 0 disables.
SAMPLE=${SAMPLE:-5}
# Largest package-count shortfall a non-primary minor may have against the best
# minor on the same slot before coverage counts as uneven. A slot built under
# one R minor carries fewer per-minor binaries for the others; until that gap
# closes, "full coverage for ABI-sensitive packages" is not a claim we can make.
# Package-count shortfall against the best minor on the same slot, above which
# coverage is reported as uneven. Reported, not failed on: the packages a
# non-primary minor lacks are ABI-risky ones built under the primary minor,
# which a client on another minor cannot safely load anyway, so their absence
# is correct. This gates the *claim* ("full coverage for ABI-sensitive
# packages"), not whether routing is safe to enable - MAX_REGRESSIONS does
# that.
PARITY_TOLERANCE=${PARITY_TOLERANCE:-25}
# Source fallbacks are reported, not failed on. Since bincraft learned to keep
# a matching-minor generic binary out of a fallback's shadow, a remaining
@ -101,10 +104,12 @@ fetch() {
return 0
fi
local status
# -L: the router answers an index request with a redirect, so the bytes a
# client ends up with are only visible by following it.
if [ -n "$ua" ]; then
status=$(curl -sS -A "$ua" -o "$dest" -w '%{http_code}' --max-time 120 "$url" 2>/dev/null)
status=$(curl -sSL -A "$ua" -o "$dest" -w '%{http_code}' --max-time 120 "$url" 2>/dev/null)
else
status=$(curl -sS -o "$dest" -w '%{http_code}' --max-time 120 "$url" 2>/dev/null)
status=$(curl -sSL -o "$dest" -w '%{http_code}' --max-time 120 "$url" 2>/dev/null)
fi
echo "$status" > "$dest.status"
echo "$status"
@ -299,7 +304,8 @@ for arch in $ARCHES; do
[ "$gap" -gt "$PARITY_TOLERANCE" ] && uneven="$uneven R$minor:-$gap"
done
if [ -n "$uneven" ]; then
bad "$slot coverage uneven across minors (vs best $best):$uneven"
printf ' note: %s coverage uneven across minors (vs best %s):%s\n' \
"$slot" "$best" "$uneven"
else
ok "$slot coverage parity across minors (best $best, all within $PARITY_TOLERANCE)"
fi