test(verify): report uneven coverage instead of failing on it #180

Merged
pat-s merged 1 commit from test/parity-as-note into main 2026-08-31 10:00:50 +00:00

View file

@ -43,10 +43,13 @@ MINORS=${MINORS:-"4.4 4.5 4.6"}
EXCLUDED_MINORS=${EXCLUDED_MINORS:-"4.3"}
# How many Path: targets to HEAD-check per slot/minor. 0 disables.
SAMPLE=${SAMPLE:-5}
# Largest package-count shortfall a non-primary minor may have against the best
# minor on the same slot before coverage counts as uneven. A slot built under
# one R minor carries fewer per-minor binaries for the others; until that gap
# closes, "full coverage for ABI-sensitive packages" is not a claim we can make.
# Package-count shortfall against the best minor on the same slot, above which
# coverage is reported as uneven. Reported, not failed on: the packages a
# non-primary minor lacks are ABI-risky ones built under the primary minor,
# which a client on another minor cannot safely load anyway, so their absence
# is correct. This gates the *claim* ("full coverage for ABI-sensitive
# packages"), not whether routing is safe to enable - MAX_REGRESSIONS does
# that.
PARITY_TOLERANCE=${PARITY_TOLERANCE:-25}
# Source fallbacks are reported, not failed on. Since bincraft learned to keep
# a matching-minor generic binary out of a fallback's shadow, a remaining
@ -299,7 +302,8 @@ for arch in $ARCHES; do
[ "$gap" -gt "$PARITY_TOLERANCE" ] && uneven="$uneven R$minor:-$gap"
done
if [ -n "$uneven" ]; then
bad "$slot coverage uneven across minors (vs best $best):$uneven"
printf ' note: %s coverage uneven across minors (vs best %s):%s\n' \
"$slot" "$best" "$uneven"
else
ok "$slot coverage parity across minors (best $best, all within $PARITY_TOLERANCE)"
fi