build-cran-binaries/local
Repository files (latest commit first)
Filename Latest commit message Latest commit date
pat-s af4cc6f414
Some checks failed
ci/crow/manual/verify-repo-routing/15 Pipeline was successful
ci/crow/manual/verify-repo-routing/3 Pipeline failed
ci/crow/manual/prune-stale-flat-binaries/15 Pipeline was successful
ci/crow/manual/reindex/7 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline failed
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/manual/verify-repo-routing/16 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/manual/repair-unsafe-flat-objects/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
feat(ci): verify what clients actually receive from every published slot (#209)
## Why

Everything built so far reasons about the *binaries*: `abi_flat_safety` records what an object's symbols say it can load under, and the index writer drops records on that verdict.
None of it proves what a **client** actually receives, which is the product of the CDN rewrite, the per-minor indexes and the `Path:` field acting together.

The reprex that found the `base64enc` failure ran on a workstation, could only reach amd64, and did not survive the session.
This makes it a pipeline.

## What it does

For every slot, under every R minor the image carries, it installs a fixed package set through each URL form the website documents and checks that each one loads in a fresh subprocess.

| form | repository | user agent |
| --- | --- | --- |
| `arch` | `https://cran.rpkgs.com/<arch>/<codename>/latest` | R's own |
| `generic` | `https://cran.rpkgs.com/` | the string the website's snippet builds |
| `flat` | the arch URL | `curl/8.5.0`, so the edge cannot read an R minor |

`arch` and `generic` are contractual, so a failure there fails the run.
`flat` measures what a client hiding its R minor is exposed to: it is reported and never fatal, because pruning the flat slot is a separate decision.

Two details that matter:

- **CRAN is deliberately absent from `repos`.** A package missing from our index must fail loudly rather than be quietly compiled from source, which would hide exactly the gap this exists to find.
- **One process per form.** `available.packages()` caches its index in `tempdir()` keyed by repository URL, and `arch` and `flat` share a URL. In a single process the second form would read the first form's index and prove nothing.

## Verification

Run against the live repository in `build-env-ubuntu:noble` under R 4.6.0, mid-repair:

```
amd64/noble R 4.6 via arch:    11 of 11 packages usable
amd64/noble R 4.6 via generic: 11 of 11 packages usable
amd64/noble R 4.6 via flat:     8 of 11 packages usable
```

The three `flat` failures are `Rcpp`, `data.table` and `yaml`, each `Built: R 4.4.3` and each failing `dyn.load()` under 4.6.
That is the unrouted-client exposure, measured rather than estimated.

The rows also show the routed forms serving a mix of per-minor and flat objects (`Rcpp` at `R 4.6.0`, `stringi` at `R 4.4.3`), which is the intended behaviour: only objects that cannot load cross-minor get a per-minor copy.

Reviewed-on: #209
2026-09-09 12:52:16 +00:00
..
patches fix(patches): support RcppParallel 6.2.1 2026-08-30 14:26:37 +00:00
tests feat(rebuild): shard the weekly rebuild and make each shard resumable (#163) 2026-08-12 08:30:29 +00:00
archive-missed-pkgs.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
backfill-abi-classification.R feat(abi): backfill classifier verdicts for pre-per-minor packages (#199) 2026-09-04 10:34:49 +00:00
backfill-flat-safety.R fix(abi): enumerate the flat slot from S3, not through the edge (#203) 2026-09-05 09:41:57 +00:00
build-all.R fix(build): strip the s3 scheme so per-minor objects reach their pass (#192) 2026-09-02 07:11:10 +00:00
build-one.R feat: patch registry + wiring for per-package patching (#103) 2026-06-30 08:50:19 +00:00
check-cdn-cache.sh add alpine321 to scripts 2024-12-29 10:18:55 +01:00
check-NA.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
debug-packages-writing.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
dedupe-audit-issue.R fix(audit): replace arch subsection in place instead of appending duplicates (#138) 2026-07-21 08:51:23 +00:00
detect-duplicates.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
excluded-packages.json feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
failing-builds-classify.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
failing-builds-report.R feat(local): detect dependency-cascade failures generally, not just RcppParallel (#128) 2026-07-16 08:23:54 +00:00
fetch-rebuild-packages-from-issue.R fix(audit): replace arch subsection in place instead of appending duplicates (#138) 2026-07-21 08:51:23 +00:00
find-R-api-packages.sh chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
install-bincraft.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
label-removed-cran-packages.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
last-processed-by-platform.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
list-unsafe-flat-objects.R fix(abi): do not error on the nothing-to-repair path (#208) 2026-09-09 10:55:15 +00:00
migrate-s3-hetzner-to-backblaze.sh chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
missing-cran-packages-db.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
missing-cran-packages.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
missing-packages-in-index.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
packages-to-build.R fix(build): scope the errored-package filter to the snapshot's R minor (#194) 2026-09-03 13:37:04 +00:00
packages-without-any-binary.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
proposal-tracking-lib.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
proposal-tracking.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
propose-patches.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
prune-stale-flat-binaries.R fix(abi): prune flat-slot binaries of ABI-risky packages (#200) 2026-09-04 10:54:09 +00:00
query-pkgs-without-old-versions.R chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
r-minor-helpers.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
rebuild-missing-helpers.R feat(rebuild): shard the weekly rebuild and make each shard resumable (#163) 2026-08-12 08:30:29 +00:00
rebuild-missing.R feat(rebuild): shard the weekly rebuild and make each shard resumable (#163) 2026-08-12 08:30:29 +00:00
repair-built-stamp.R fix(index): add a slot repair for a broken Built stamp and retire alpine 3.21 (#150) 2026-08-07 09:15:04 +00:00
test-package-loading.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
trial-build-patch.R fix(local): make trial-build-patch.R detect non-throwing build failures (#134) 2026-07-18 08:37:04 +00:00
trial-build-registry.R fix(local): make the trial-build gate detect non-throwing build failures (#130) 2026-07-16 08:24:03 +00:00
uvr-install.sh fix(rebuild): harden split workflow setup (#164) 2026-08-13 13:38:28 +00:00
validate-patches.R feat(local): auto-propose registry patches and track the feedback loop (#117) 2026-07-14 15:03:24 +00:00
verify-repo-routing.R feat(ci): verify what clients actually receive from every published slot (#209) 2026-09-09 12:52:16 +00:00
weekly-missing-binaries-audit.R fix(audit): count a source fallback as a missing binary (#157) 2026-08-09 10:37:41 +00:00