fix(audit): count a source fallback as a missing binary #157

Merged

pat-s merged 1 commit from fix/audit-counts-source-fallback into main

2026-08-09 10:37:42 +00:00

pat-s commented

2026-08-09 10:31:59 +00:00

Owner

Problem

The audit's definition of "missing" was missing_dt <- cran_dt[!s3_dt], where s3_dt came from listing the bucket. But an object being present does not mean a binary was built: when a build fails, bincraft publishes the CRAN source tarball in its place so the package stays installable, and a listing cannot tell the two apart.

So every failed build looked done. On amd64/alpine324 that is 13 547 of 24 134 comparable objects (56%) — all CRAN sources from the June 2026 bootstrap, none ever reported here, so weekly-rebuild-missing never retried any of them. amd64/noble sits at 4.6% for comparison.

What this changes

Reads the slot's own PACKAGES index instead of listing the bucket, and counts only records carrying a Built stamp. bincraft writes that stamp only for what it actually built (rpkgs/bincraft#105), so the index answers the question a listing cannot.

Side effects of reading the index rather than the bucket:

the audit reports how many records the index holds and how many are served as source, so the gap is visible in the log
s3fs is no longer used, so the audit needs no B2 credentials
an object present but absent from the index is now counted as missing, which is correct: clients only see the index

Sequencing

This is inert until the matching bincraft change ships and a slot is re-indexed. A slot last indexed by an older bincraft carries Built on every record, including the fallbacks, so the audit reports exactly what it does today. Verified against the live indices:

amd64/alpine324: records=24152 built=24152 source-served=0
amd64/noble:     records=24681 built=24681 source-served=0

After a re-index those source-served counts become the real ones, and the packages behind them start appearing in issue #63 and getting rebuilt.

## Problem The audit's definition of "missing" was `missing_dt <- cran_dt[!s3_dt]`, where `s3_dt` came from listing the bucket. But an object being present does not mean a binary was built: when a build fails, bincraft publishes the CRAN source tarball in its place so the package stays installable, and a listing cannot tell the two apart. So every failed build looked done. On `amd64/alpine324` that is 13 547 of 24 134 comparable objects (56%) — all CRAN sources from the June 2026 bootstrap, none ever reported here, so `weekly-rebuild-missing` never retried any of them. `amd64/noble` sits at 4.6% for comparison. ## What this changes Reads the slot's own `PACKAGES` index instead of listing the bucket, and counts only records carrying a `Built` stamp. bincraft writes that stamp only for what it actually built (rpkgs/bincraft#105), so the index answers the question a listing cannot. Side effects of reading the index rather than the bucket: - the audit reports how many records the index holds and how many are served as source, so the gap is visible in the log - `s3fs` is no longer used, so the audit needs no B2 credentials - an object present but absent from the index is now counted as missing, which is correct: clients only see the index ## Sequencing This is inert until the matching bincraft change ships and a slot is re-indexed. A slot last indexed by an older bincraft carries `Built` on every record, including the fallbacks, so the audit reports exactly what it does today. Verified against the live indices: ``` amd64/alpine324: records=24152 built=24152 source-served=0 amd64/noble: records=24681 built=24681 source-served=0 ``` After a re-index those `source-served` counts become the real ones, and the packages behind them start appearing in issue #63 and getting rebuilt.

pat-s added 1 commit

2026-08-09 10:32:00 +00:00

fix(audit): count a source fallback as a missing binary 5e58eb5c91

An object being present in the slot does not mean a binary was built: when a
build fails, bincraft publishes the CRAN source tarball in its place, and the
bucket listing this used could not tell the two apart. That is how
amd64/alpine324 came to hold 13547 CRAN sources that this audit never reported
and weekly-rebuild-missing therefore never retried.

- read the slot's own PACKAGES index instead of listing the bucket, and count
  only records carrying a Built stamp, which bincraft writes only for what it
  actually built
- report how many records the index holds and how many are served as source
- drop the now-unused s3fs dependency, so the audit needs no B2 credentials

A slot last indexed by a bincraft that predates the matching fix stamps Built
on every record, so this reports exactly what it used to until re-indexed.

pat-s merged commit 0703969088 into main

2026-08-09 10:37:42 +00:00

pat-s referenced this pull request from a commit

2026-08-09 10:37:42 +00:00

fix(audit): count a source fallback as a missing binary (#157)

pat-s deleted branch fix/audit-counts-source-fallback

2026-08-09 10:37:42 +00:00