build-cran-binaries/local
Repository files (latest commit first)
Filename Latest commit message Latest commit date
pat-s 0da29ab3f6
Some checks failed
ci/crow/manual/build-all-versions-install-deps/2 Pipeline is pending
ci/crow/manual/build-all-versions/5 Pipeline is pending
ci/crow/manual/build-all-versions/6 Pipeline is pending
ci/crow/manual/build-all-versions/7 Pipeline is pending
ci/crow/manual/build-all-versions/8 Pipeline is pending
ci/crow/manual/process-updates/2 Pipeline is pending
ci/crow/manual/process-updates/4 Pipeline is pending
ci/crow/manual/process-updates/6 Pipeline is pending
ci/crow/manual/process-updates/8 Pipeline is pending
ci/crow/manual/process-updates/10 Pipeline is pending
ci/crow/manual/process-updates/12 Pipeline is pending
ci/crow/manual/process-updates/14 Pipeline is pending
ci/crow/manual/process-updates/16 Pipeline is pending
ci/crow/manual/process-updates/18 Pipeline is pending
ci/crow/manual/repair-built-stamp/2 Pipeline is pending
ci/crow/manual/trial-build-registry/2 Pipeline is pending
ci/crow/manual/trial-build-registry/4 Pipeline is pending
ci/crow/manual/trial-build-registry/6 Pipeline is pending
ci/crow/manual/trial-build-registry/8 Pipeline is pending
ci/crow/manual/trial-build-registry/10 Pipeline is pending
ci/crow/manual/trial-build-registry/12 Pipeline is pending
ci/crow/manual/trial-build-registry/14 Pipeline is pending
ci/crow/manual/trial-build-registry/16 Pipeline is pending
ci/crow/manual/trial-build-registry/18 Pipeline is pending
ci/crow/manual/weekly-audit-missing/2 Pipeline is pending
ci/crow/manual/weekly-audit-missing/4 Pipeline is pending
ci/crow/manual/weekly-audit-missing/6 Pipeline is pending
ci/crow/manual/weekly-audit-missing/8 Pipeline is pending
ci/crow/manual/weekly-audit-missing/10 Pipeline is pending
ci/crow/manual/weekly-audit-missing/12 Pipeline is pending
ci/crow/manual/weekly-audit-missing/14 Pipeline is pending
ci/crow/manual/weekly-audit-missing/16 Pipeline is pending
ci/crow/manual/weekly-audit-missing/18 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/2 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/4 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/5 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/6 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/7 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/8 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/9 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/10 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/11 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/12 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/13 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/14 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/15 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/16 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/17 Pipeline is pending
ci/crow/manual/weekly-rebuild-missing/18 Pipeline is pending
ci/crow/manual/process-updates/9 Pipeline was successful
ci/crow/manual/trial-build-registry/1 Pipeline was successful
ci/crow/manual/trial-build-registry/3 Pipeline was successful
ci/crow/manual/trial-build-registry/5 Pipeline was successful
ci/crow/manual/process-updates/7 Pipeline was successful
ci/crow/manual/trial-build-registry/7 Pipeline was successful
ci/crow/manual/trial-build-registry/9 Pipeline was successful
ci/crow/manual/trial-build-registry/11 Pipeline was successful
ci/crow/manual/trial-build-registry/13 Pipeline was successful
ci/crow/manual/auto-apply-patches Pipeline was successful
ci/crow/manual/trial-build-registry/15 Pipeline was successful
ci/crow/manual/trial-build-registry/17 Pipeline was successful
ci/crow/manual/weekly-audit-missing/1 Pipeline was successful
ci/crow/manual/weekly-audit-missing/5 Pipeline was successful
ci/crow/manual/weekly-audit-missing/3 Pipeline was successful
ci/crow/manual/weekly-audit-missing/7 Pipeline was successful
ci/crow/manual/weekly-audit-missing/9 Pipeline was successful
ci/crow/manual/weekly-audit-missing/11 Pipeline was successful
ci/crow/manual/weekly-audit-missing/13 Pipeline was successful
ci/crow/manual/weekly-audit-missing/17 Pipeline was successful
ci/crow/manual/weekly-audit-missing/15 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/1 Pipeline was canceled
ci/crow/manual/weekly-rebuild-missing/3 Pipeline was canceled
ci/crow/manual/process-updates/3 Pipeline was canceled
ci/crow/manual/process-updates/15 Pipeline was canceled
ci/crow/manual/process-updates/11 Pipeline was canceled
ci/crow/manual/weekly-patch-proposals Pipeline was canceled
ci/crow/manual/process-updates/13 Pipeline was canceled
ci/crow/manual/process-updates/1 Pipeline was canceled
ci/crow/manual/process-updates/5 Pipeline was canceled
ci/crow/manual/process-updates/17 Pipeline was canceled
ci/crow/cron/process-updates/15 Pipeline failed
fix(index): add a slot repair for a broken Built stamp and retire alpine 3.21 (#150)
## Problem

`arm64/alpine321` and `arm64/alpine322` advertise a broken stamp:

```
arm64/alpine321 :: 22930  Built: R 4.4.0; NA; 2026-07-31 13:35:52 UTC; unix
arm64/alpine322 :: 24696  Built: R 4.5.0; NA; 2026-07-31 13:51:54 UTC; unix
```

The per-minor sub-slots (`contrib/4.4`, `4.5`, `4.6`) are affected too.
All 18 other slots are correct.

uvr matches the stamp's platform triple plus R minor to pick binary over source, so nothing matches `NA` and both slots silently serve as source-only, which is exactly the regression bincraft#85 added the stamp to prevent.
`install.packages()` is unaffected, since it reads `Built:` from each tarball's own `DESCRIPTION`.
The tarballs are fine (`arm64/alpine322/.../dress.graph_0.8.3.tar.gz` carries `aarch64-unknown-linux-musl`), and so is the R that built them (`r-4.5.0_1_aarch64.apk` ships `R_PLATFORM='aarch64-unknown-linux-musl'`).
Only the index is wrong.

bincraft#96 stops a stamp like this being written again, but it cannot repair what is already there.

## Why not just re-run the index update

`upload_package_index()` reuses the slot's remote `PACKAGES.db`, and cranlike's `update_db()` only reparses files whose md5 changed, so entries already in the database keep the stamp they were written with.

Dropping `PACKAGES.db` to force a full reparse does work, and it is what bincraft#85's rollout note suggested, but for an S3 repo cranlike reads each package's metadata from the CRAN *source* mirror on GitHub.
A 25k-entry slot is then 25k requests to raw.githubusercontent.com, with a real risk of being rate-limited part-way through and leaving the slot half-written.

Only the `Built` column is wrong, so this corrects it in place instead: patch the column in `PACKAGES.db`, put the database back, and let `upload_package_index()` re-emit `PACKAGES*` from it.
`update_db()` always rewrites the index files even when nothing was reparsed, so no tarball is re-read and nothing is fetched from GitHub.

## Change

- `local/repair-built-stamp.R` — repairs the generic slot plus every per-minor sub-slot. Dry-run by default; `--apply` writes. The replacement comes from `bincraft::built_stamp()` under the R running the script, so it is exactly what a healthy run would have written, and bincraft#96's guard makes a broken build image fail rather than write a second bad stamp.
- `.crow/repair-built-stamp.yaml` — manual pipeline, routed by `target_arch` to the matching agent group and platform image, with `dry_run` defaulting to `true`.
- `.crow/archive-missed-packages.yaml` — drop the two `alpine321` matrix entries. Alpine 3.21 is EOL: the website advertises only 3.23/3.24 and `process-updates.yaml` already dropped it, so that slot is retired rather than repaired.

## Verification

`crow lint .crow/` passes on all nine pipelines.
`air format` and `jarl check` are clean; the script parses, and the `/opt/R` minor-version derivation was checked against `4.4.3 / 4.5.3 / 4.6.0 / current` → `4.4 4.5 4.6`.
The repair itself is unrun by design — it needs B2 credentials and an arm64 agent.

## Rollout

1. Cut a bincraft release so `local/install-bincraft.R` picks up #96 (it resolves the latest `vX.Y.Z` tag, and #96 is only on `main`).
2. Run this pipeline with `target_arch=arm64`, `OS=alpine`, `OS_VERSION=3.22`, `R_VERSION=4.5.3`, `dry_run=true` and check the reported counts.
3. Re-run with `dry_run=false`.
4. Confirm: `curl -sS https://cran.devxy.io/arm64/alpine322/latest/src/contrib/PACKAGES | grep '^Built:' | sort | uniq -c`

`arm64/alpine321` is deliberately left alone.

Reviewed-on: #150
2026-08-07 09:15:04 +00:00
..
patches fix(patches): force RcppParallel to use its bundled oneTBB (#146) 2026-07-31 09:48:41 +00:00
tests refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
archive-missed-pkgs.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
build-all.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
build-one.R feat: patch registry + wiring for per-package patching (#103) 2026-06-30 08:50:19 +00:00
check-cdn-cache.sh add alpine321 to scripts 2024-12-29 10:18:55 +01:00
check-NA.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
debug-packages-writing.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
dedupe-audit-issue.R fix(audit): replace arch subsection in place instead of appending duplicates (#138) 2026-07-21 08:51:23 +00:00
detect-duplicates.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
excluded-packages.json feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
failing-builds-classify.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
failing-builds-report.R feat(local): detect dependency-cascade failures generally, not just RcppParallel (#128) 2026-07-16 08:23:54 +00:00
fetch-rebuild-packages-from-issue.R fix(audit): replace arch subsection in place instead of appending duplicates (#138) 2026-07-21 08:51:23 +00:00
find-R-api-packages.sh chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
install-bincraft.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
label-removed-cran-packages.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
last-processed-by-platform.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
migrate-s3-hetzner-to-backblaze.sh chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
missing-cran-packages-db.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
missing-cran-packages.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
missing-packages-in-index.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
packages-to-build.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
packages-without-any-binary.R feat: R-minor-sensitive binary builds (full + iterative) (#84) 2026-06-14 08:04:40 +00:00
proposal-tracking-lib.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
proposal-tracking.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
propose-patches.R feat(local): aggregate blocked-on-dependency reporting by dependency (#131) 2026-07-16 21:24:03 +00:00
query-pkgs-without-old-versions.R chore(local): add S3 migration and CRAN-source helper scripts 2026-06-14 13:02:48 +02:00
r-minor-helpers.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
repair-built-stamp.R fix(index): add a slot repair for a broken Built stamp and retire alpine 3.21 (#150) 2026-08-07 09:15:04 +00:00
test-package-loading.R refactor: migrate package installation from pak to uvr (#147) 2026-07-31 12:19:26 +00:00
trial-build-patch.R fix(local): make trial-build-patch.R detect non-throwing build failures (#134) 2026-07-18 08:37:04 +00:00
trial-build-registry.R fix(local): make the trial-build gate detect non-throwing build failures (#130) 2026-07-16 08:24:03 +00:00
uvr-install.sh chore(deps): update dependency nbafrank/uvr to v0.4.5 2026-08-04 02:32:01 +00:00
validate-patches.R feat(local): auto-propose registry patches and track the feedback loop (#117) 2026-07-14 15:03:24 +00:00
weekly-missing-binaries-audit.R fix(audit): replace arch subsection in place instead of appending duplicates (#138) 2026-07-21 08:51:23 +00:00