build-cran-binaries/local
Repository files (latest commit first)
Filename Latest commit message Latest commit date
pat-s 0703969088
Some checks failed
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/manual/weekly-audit-missing/1 Pipeline was successful
ci/crow/manual/weekly-audit-missing/2 Pipeline was successful
ci/crow/manual/weekly-audit-missing/3 Pipeline was successful
ci/crow/manual/weekly-audit-missing/4 Pipeline was successful
ci/crow/manual/weekly-audit-missing/7 Pipeline was successful
ci/crow/manual/weekly-audit-missing/8 Pipeline was successful
ci/crow/manual/weekly-audit-missing/9 Pipeline was successful
ci/crow/manual/weekly-audit-missing/11 Pipeline was successful
ci/crow/manual/weekly-audit-missing/12 Pipeline was successful
ci/crow/manual/weekly-audit-missing/13 Pipeline was successful
ci/crow/manual/weekly-audit-missing/16 Pipeline was successful
ci/crow/manual/weekly-audit-missing/17 Pipeline was successful
ci/crow/manual/weekly-audit-missing/15 Pipeline was successful
ci/crow/manual/weekly-audit-missing/10 Pipeline was successful
ci/crow/manual/weekly-audit-missing/18 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/manual/weekly-audit-missing/14 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/manual/build-all-versions/2 Pipeline is pending
ci/crow/manual/build-all-versions/3 Pipeline is pending
ci/crow/manual/build-all-versions/4 Pipeline is pending
ci/crow/manual/process-updates/12 Pipeline is pending
ci/crow/manual/process-updates/14 Pipeline is pending
ci/crow/manual/process-updates/16 Pipeline is pending
ci/crow/manual/process-updates/18 Pipeline is pending
ci/crow/manual/repair-built-stamp/2 Pipeline is pending
ci/crow/manual/weekly-audit-missing/6 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/1 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/2 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/3 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/4 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/5 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/6 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/7 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/8 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/9 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/10 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/11 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/12 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/14 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/15 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/16 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/18 Pipeline is pending
ci/crow/manual/process-updates/10 Pipeline was canceled
ci/crow/manual/build-all-versions/1 Pipeline was canceled
ci/crow/manual/process-updates/17 Pipeline was canceled
ci/crow/manual/weekly-audit-missing/5 Pipeline was canceled
ci/crow/manual/process-updates/15 Pipeline was canceled
ci/crow/manual/process-updates/7 Pipeline failed
ci/crow/manual/process-updates/9 Pipeline failed
ci/crow/manual/process-updates/6 Pipeline failed
ci/crow/manual/process-updates/13 Pipeline failed
ci/crow/manual/process-updates/8 Pipeline failed
ci/crow/manual/process-updates/1 Pipeline was canceled
ci/crow/manual/weekly-rebuild-missing/13 Pipeline failed
ci/crow/manual/build-all-versions-install-deps/1 Pipeline was canceled
ci/crow/manual/process-updates/11 Pipeline failed
ci/crow/manual/process-updates/3 Pipeline was canceled
ci/crow/manual/process-updates/5 Pipeline was canceled
ci/crow/manual/process-updates/4 Pipeline was canceled
ci/crow/manual/process-updates/2 Pipeline was canceled
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/17 Pipeline was successful
fix(audit): count a source fallback as a missing binary (#157)
## Problem

The audit's definition of "missing" was `missing_dt <- cran_dt[!s3_dt]`, where `s3_dt` came from listing the bucket. But an object being present does not mean a binary was built: when a build fails, bincraft publishes the CRAN source tarball in its place so the package stays installable, and a listing cannot tell the two apart.

So every failed build looked done. On `amd64/alpine324` that is 13 547 of 24 134 comparable objects (56%) — all CRAN sources from the June 2026 bootstrap, none ever reported here, so `weekly-rebuild-missing` never retried any of them. `amd64/noble` sits at 4.6% for comparison.

## What this changes

Reads the slot's own `PACKAGES` index instead of listing the bucket, and counts only records carrying a `Built` stamp. bincraft writes that stamp only for what it actually built (rpkgs/bincraft#105), so the index answers the question a listing cannot.

Side effects of reading the index rather than the bucket:

- the audit reports how many records the index holds and how many are served as source, so the gap is visible in the log
- `s3fs` is no longer used, so the audit needs no B2 credentials
- an object present but absent from the index is now counted as missing, which is correct: clients only see the index

## Sequencing

This is inert until the matching bincraft change ships and a slot is re-indexed. A slot last indexed by an older bincraft carries `Built` on every record, including the fallbacks, so the audit reports exactly what it does today. Verified against the live indices:

```
amd64/alpine324: records=24152 built=24152 source-served=0
amd64/noble:     records=24681 built=24681 source-served=0
```

After a re-index those `source-served` counts become the real ones, and the packages behind them start appearing in issue #63 and getting rebuilt.

Reviewed-on: #157
2026-08-09 10:37:41 +00:00
..
patches fix(patches): force RcppParallel to use its bundled oneTBB (#146) 2026-07-31 09:48:41 +00:00
tests refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
archive-missed-pkgs.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
build-all.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
build-one.R feat: patch registry + wiring for per-package patching (#103) 2026-06-30 08:50:19 +00:00
check-cdn-cache.sh add alpine321 to scripts 2024-12-29 10:18:55 +01:00
check-NA.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
debug-packages-writing.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
dedupe-audit-issue.R fix(audit): replace arch subsection in place instead of appending duplicates (#138) 2026-07-21 08:51:23 +00:00
detect-duplicates.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
excluded-packages.json feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
failing-builds-classify.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
failing-builds-report.R feat(local): detect dependency-cascade failures generally, not just RcppParallel (#128) 2026-07-16 08:23:54 +00:00
fetch-rebuild-packages-from-issue.R fix(audit): replace arch subsection in place instead of appending duplicates (#138) 2026-07-21 08:51:23 +00:00
find-R-api-packages.sh chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
install-bincraft.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
label-removed-cran-packages.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
last-processed-by-platform.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
migrate-s3-hetzner-to-backblaze.sh chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
missing-cran-packages-db.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
missing-cran-packages.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
missing-packages-in-index.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
packages-to-build.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
packages-without-any-binary.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
proposal-tracking-lib.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
proposal-tracking.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
propose-patches.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
query-pkgs-without-old-versions.R chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
r-minor-helpers.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
repair-built-stamp.R fix(index): add a slot repair for a broken Built stamp and retire alpine 3.21 (#150) 2026-08-07 09:15:04 +00:00
test-package-loading.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
trial-build-patch.R fix(local): make trial-build-patch.R detect non-throwing build failures (#134) 2026-07-18 08:37:04 +00:00
trial-build-registry.R fix(local): make the trial-build gate detect non-throwing build failures (#130) 2026-07-16 08:24:03 +00:00
uvr-install.sh chore(deps): update dependency nbafrank/uvr to v0.4.5 2026-08-04 02:32:01 +00:00
validate-patches.R feat(local): auto-propose registry patches and track the feedback loop (#117) 2026-07-14 15:03:24 +00:00
weekly-missing-binaries-audit.R fix(audit): count a source fallback as a missing binary (#157) 2026-08-09 10:37:41 +00:00