test(verify): gate on regressions against the generic slot, not fallback rate #179

Merged

pat-s merged 1 commit from test/regression-gate into main

2026-08-31 09:55:45 +00:00

pat-s commented

2026-08-31 09:51:03 +00:00

Owner

Motivation

The readiness check added in #175 failed a slot when more than 10% of its per-minor entries were source fallbacks. That stopped being a meaningful signal once rpkgs/bincraft#113 and #114 landed.

Since bincraft keeps a matching-minor generic binary out of a fallback's shadow, a surviving fallback means the generic slot's binary was built under a different minor — unsafe for that client anyway. Serving source there is correct, just slow. Failing on that share blocks slots that are genuinely ready: amd64/noble sits at 53% for 4.5 and 4.6 while regressing nobody.

Change

Gate on the thing that actually decides enablement: packages a client of minor M would receive as source through per-minor routing while the generic slot holds a binary built under M itself. That is strictly worse than not routing at all, and must be zero.

Fallback share is still printed, as context rather than a verdict.

Verification

Measured across every reindexed slot and minor after the reindex=all run: zero regressions everywhere.

slot	4.4	4.5	4.6
amd64/noble	0	0	0
amd64/jammy	0	0	0
amd64/rhel9	0	0	0
amd64/rhel10	0	0	0
amd64/resolute	0	0	0
arm64/noble	0	0	0

shellcheck clean; script exercised against the live indexes.

## Motivation The readiness check added in #175 failed a slot when more than 10% of its per-minor entries were source fallbacks. That stopped being a meaningful signal once rpkgs/bincraft#113 and #114 landed. Since bincraft keeps a matching-minor generic binary out of a fallback's shadow, a surviving fallback means the generic slot's binary was built under a **different** minor — unsafe for that client anyway. Serving source there is correct, just slow. Failing on that share blocks slots that are genuinely ready: `amd64/noble` sits at 53% for 4.5 and 4.6 while regressing nobody. ## Change Gate on the thing that actually decides enablement: packages a client of minor M would receive as **source** through per-minor routing while the generic slot holds a binary built under **M itself**. That is strictly worse than not routing at all, and must be zero. Fallback share is still printed, as context rather than a verdict. ## Verification Measured across every reindexed slot and minor after the `reindex=all` run: zero regressions everywhere. | slot | 4.4 | 4.5 | 4.6 | |---|---|---|---| | amd64/noble | 0 | 0 | 0 | | amd64/jammy | 0 | 0 | 0 | | amd64/rhel9 | 0 | 0 | 0 | | amd64/rhel10 | 0 | 0 | 0 | | amd64/resolute | 0 | 0 | 0 | | arm64/noble | 0 | 0 | 0 | `shellcheck` clean; script exercised against the live indexes.

pat-s added 1 commit

2026-08-31 09:51:03 +00:00

test(verify): gate on regressions against the generic slot, not fallback rate 258bb9c892

A source-fallback percentage stopped being a readiness signal once
bincraft learned to keep a matching-minor generic binary out of a
fallback's shadow. What survives now is the case where the generic binary
was built under a different minor: unsafe for this client anyway, so
serving source is correct, just slow. Failing on that share would block
slots that are in fact ready - amd64/noble sits at 53% for 4.5 and 4.6
while regressing nobody.

Gate on the thing that decides it instead: packages this client would
receive as source through per-minor routing while the generic slot holds
a binary built under its own minor. That is strictly worse than not
routing, and must be zero. Fallback share is still printed, as context
rather than a verdict.

Measured zero across every reindexed slot and minor.

pat-s merged commit a3004695a7 into main

2026-08-31 09:55:45 +00:00

pat-s deleted branch test/regression-gate

2026-08-31 09:55:45 +00:00

pat-s referenced this pull request from a commit

2026-08-31 09:55:46 +00:00

test(verify): gate on regressions against the generic slot, not fallback rate (#179)