build-cran-binaries/local
Repository files (latest commit first)
Filename Latest commit message Latest commit date
pat-s 844ad9749b
Some checks failed
ci/crow/cron/process-updates/16 Pipeline is pending
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline failed
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/manual/backfill-flat-safety/1 Pipeline was successful
ci/crow/manual/backfill-flat-safety/3 Pipeline was successful
ci/crow/manual/backfill-flat-safety/5 Pipeline was successful
ci/crow/manual/backfill-flat-safety/7 Pipeline was successful
ci/crow/manual/backfill-flat-safety/9 Pipeline was successful
ci/crow/manual/backfill-flat-safety/11 Pipeline was successful
ci/crow/manual/backfill-flat-safety/13 Pipeline was successful
ci/crow/manual/backfill-flat-safety/15 Pipeline was successful
ci/crow/manual/backfill-flat-safety/17 Pipeline was successful
ci/crow/manual/backfill-flat-safety/18 Pipeline was successful
ci/crow/manual/backfill-flat-safety/4 Pipeline was successful
ci/crow/manual/backfill-flat-safety/10 Pipeline was successful
ci/crow/manual/backfill-flat-safety/14 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/manual/backfill-flat-safety/8 Pipeline was successful
ci/crow/manual/backfill-flat-safety/6 Pipeline was successful
ci/crow/manual/backfill-flat-safety/16 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/manual/backfill-flat-safety/12 Pipeline was successful
ci/crow/manual/backfill-flat-safety/2 Pipeline was successful
ci/crow/manual/reindex/7 Pipeline failed
ci/crow/manual/reindex/11 Pipeline failed
ci/crow/manual/reindex/15 Pipeline failed
ci/crow/manual/reindex/9 Pipeline failed
ci/crow/manual/reindex/3 Pipeline failed
ci/crow/manual/reindex/13 Pipeline was successful
ci/crow/manual/reindex/17 Pipeline was successful
ci/crow/manual/reindex/8 Pipeline was successful
ci/crow/manual/reindex/4 Pipeline was successful
ci/crow/manual/reindex/1 Pipeline was successful
ci/crow/manual/reindex/2 Pipeline was successful
ci/crow/manual/reindex/5 Pipeline was successful
ci/crow/manual/reindex/6 Pipeline was successful
ci/crow/manual/reindex/10 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/manual/reindex/12 Pipeline was successful
ci/crow/manual/reindex/14 Pipeline was successful
ci/crow/manual/reindex/16 Pipeline was successful
ci/crow/manual/reindex/18 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
fix(abi): enumerate the flat slot from S3, not through the edge (#203)
## Why

The first backfill run reported:

```
amd64/resolute: 27105 flat binaries, 8187 compiled and worth inspecting
Done: 300 inspected, 7 safe for every minor, 1 unsafe, 292 skipped
```

292 of 300 skipped, and `27105` is larger than the flat index (~24k). Both have the same cause: the script read `PACKAGES.gz` over HTTPS from inside R, which sends an R User-Agent, so **the edge routed it to a per-minor index**. It then enumerated union records carrying `Path: <minor>`, whose objects do not exist at the flat path, and every download 404'd.

An ironic failure given what this whole change is about, and one the trial run existed to catch.

## What changed

- List the flat slot with `s3fs::s3_dir_ls()`, keeping only objects with no `<minor>/` prefix and nothing under `Archive/`. The bucket cannot be routed and is authoritative about what is actually in the flat slot.
- Fetch each object from S3 too. The edge does not rewrite tarball requests, but it does cache them, and the verdict is about the object.
- `NeedsCompilation` still comes from CRAN's own index, which is not behind our edge.

## Verification

The next trial run is the check: `flat objects` should now match the flat index rather than exceed it, and `skipped` should be near zero.

Reviewed-on: #203
2026-09-05 09:41:57 +00:00
..
patches fix(patches): support RcppParallel 6.2.1 2026-08-30 14:26:37 +00:00
tests feat(rebuild): shard the weekly rebuild and make each shard resumable (#163) 2026-08-12 08:30:29 +00:00
archive-missed-pkgs.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
backfill-abi-classification.R feat(abi): backfill classifier verdicts for pre-per-minor packages (#199) 2026-09-04 10:34:49 +00:00
backfill-flat-safety.R fix(abi): enumerate the flat slot from S3, not through the edge (#203) 2026-09-05 09:41:57 +00:00
build-all.R fix(build): strip the s3 scheme so per-minor objects reach their pass (#192) 2026-09-02 07:11:10 +00:00
build-one.R feat: patch registry + wiring for per-package patching (#103) 2026-06-30 08:50:19 +00:00
check-cdn-cache.sh add alpine321 to scripts 2024-12-29 10:18:55 +01:00
check-NA.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
debug-packages-writing.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
dedupe-audit-issue.R fix(audit): replace arch subsection in place instead of appending duplicates (#138) 2026-07-21 08:51:23 +00:00
detect-duplicates.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
excluded-packages.json feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
failing-builds-classify.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
failing-builds-report.R feat(local): detect dependency-cascade failures generally, not just RcppParallel (#128) 2026-07-16 08:23:54 +00:00
fetch-rebuild-packages-from-issue.R fix(audit): replace arch subsection in place instead of appending duplicates (#138) 2026-07-21 08:51:23 +00:00
find-R-api-packages.sh chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
install-bincraft.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
label-removed-cran-packages.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
last-processed-by-platform.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
migrate-s3-hetzner-to-backblaze.sh chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
missing-cran-packages-db.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
missing-cran-packages.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
missing-packages-in-index.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
packages-to-build.R fix(build): scope the errored-package filter to the snapshot's R minor (#194) 2026-09-03 13:37:04 +00:00
packages-without-any-binary.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
proposal-tracking-lib.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
proposal-tracking.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
propose-patches.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
prune-stale-flat-binaries.R fix(abi): prune flat-slot binaries of ABI-risky packages (#200) 2026-09-04 10:54:09 +00:00
query-pkgs-without-old-versions.R chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
r-minor-helpers.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
rebuild-missing-helpers.R feat(rebuild): shard the weekly rebuild and make each shard resumable (#163) 2026-08-12 08:30:29 +00:00
rebuild-missing.R feat(rebuild): shard the weekly rebuild and make each shard resumable (#163) 2026-08-12 08:30:29 +00:00
repair-built-stamp.R fix(index): add a slot repair for a broken Built stamp and retire alpine 3.21 (#150) 2026-08-07 09:15:04 +00:00
test-package-loading.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
trial-build-patch.R fix(local): make trial-build-patch.R detect non-throwing build failures (#134) 2026-07-18 08:37:04 +00:00
trial-build-registry.R fix(local): make the trial-build gate detect non-throwing build failures (#130) 2026-07-16 08:24:03 +00:00
uvr-install.sh fix(rebuild): harden split workflow setup (#164) 2026-08-13 13:38:28 +00:00
validate-patches.R feat(local): auto-propose registry patches and track the feedback loop (#117) 2026-07-14 15:03:24 +00:00
weekly-missing-binaries-audit.R fix(audit): count a source fallback as a missing binary (#157) 2026-08-09 10:37:41 +00:00