The cache handed to build_binary_package() as s3_package_cache was the raw
bucket listing, and the build list subtracted the same listing. Neither could
tell a binary from a package whose build failed and was published as its CRAN
source, so every source fallback read as "already built" and was skipped for
good - which is how alpine324 accumulated ~13.5k of them.
- drop objects the slot's index reports as served from source, which bincraft
marks by leaving the Built stamp off
- build s3_dt from the filtered listing too, since it is subtracted from the
build list and would otherwise exclude the packages that need building
- log how many objects were dropped
Archived objects have no index record and are kept: unknown means binary, never
"rebuild it". A slot last indexed by a bincraft that predates the Built change
stamps everything, so its cache is unchanged from before.
`bincraft` dropped pak in favour of uvr (5.0.x, "Dependencies and their system requirements are now installed with `uvr` instead of pak during `build_binary_package()`"), so the pipelines, helper scripts and images in this repo move with it.
## Approach
uvr is project-scoped in a way pak is not: `uvr add` refuses to run outside a project and always installs into `.uvr/library/`, and only `uvr sync` honours `--library`. So there is no one-line `pak::pak(...)` equivalent. `local/uvr-install.sh` encapsulates the dance — bootstrap a pinned uvr, mint a throwaway project under `TMPDIR`, `uvr add --no-install`, then `uvr sync --library <target>`. Keeping the project outside the checkout also keeps `uvr init`'s `.Rprofile` from hijacking `.libPaths()` for every other R call in the pipeline.
This matches what bincraft itself does (`uvr sync --install-system-deps --library <lib>`), and bincraft requires `uvr` on `PATH`, which the bootstrap provides: every pipeline that calls `bincraft::` runs `install-bincraft.R` (and therefore the bootstrap) first.
## Changes
| File | Change |
| --- | --- |
| `local/uvr-install.sh` | **New.** The single replacement for `pak::pak(...)`. Bootstraps uvr `v0.4.4`, resolves the R interpreter from `UVR_R_BIN`/`R_VERSION`/`PATH`, pins the manifest to that R's exact version, and syncs into `UVR_TARGET_LIB`/`R_LIBS_USER`. |
| `local/install-bincraft.R` | Installs `forgejo::codefloe.com/rpkgs/bincraft@<tag>` instead of a `git::` URL; keeps the `git ls-remote` tag resolution. Exports `UVR_R_BIN`/`UVR_TARGET_LIB` from `R.home()`/`.libPaths()[1]` so the per-R-minor passes target their own R and library. |
| `.crow/auto-apply-patches.yaml`, `.crow/weekly-patch-proposals.yaml`, `.crow/weekly-audit-missing.yaml`, `.crow/weekly-rebuild-missing.yaml`, `.crow/build-all-versions-install-deps.yaml` | `pak::pak(...)` → `UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh ...`. The explicit `UVR_R_BIN` matters in `build-all-versions-install-deps.yaml`, which has no `R_VERSION` in its step environment. |
| `.crow/build-all-versions.yaml`, `.crow/process-updates.yaml`, `.crow/weekly-rebuild-missing.yaml` | `R_PKG_CACHE_DIR` → `UVR_CACHE_DIR` + `UVR_PACKAGES_DIR` on the same `/mnt/cache` volume, preserving the amd64-off/arm64-on split. Drops the `rm -rf .../pkgcache/_metadata/...` cleanup. |
| `local/r-minor-helpers.R`, `local/build-all.R`, `local/tests/test-trim-pkgcache.R` | Removes `trim_pkgcache_metadata()`, its every-25-packages call and its tests. uvr's cache does not mint a fresh ~70 MB snapshot per `PACKAGES` change. |
| `.crow/build-all-versions-install-deps.yaml` | Drops `pak::sysreqs_db_update()`; uvr resolves sysreqs from its vendored `r-system-requirements` rules via `--install-system-deps`. |
| `docker/Containerfile-shiny-app` | Bootstraps uvr and drives both dependency installs through one uvr project with `UVR_LIBRARY` pointed at the image's R library. |
| `docker/build-one.Dockerfile` | Ships `uvr-install.sh` at `/work/local/` so `install-bincraft.R` finds it. |
| `docker/reprex/alpine.sh` | Replaces `pak::local_install_deps()` with DESCRIPTION parsing + `uvr add`. |
| `local/test-package-loading.R` | Installs via the helper instead of `pak::pkg_install()`. |
| `README.md` | Documents uvr for sysreq inference, archived-version installs and cache clearing. |
| `renovate.json` | Tracks the `UVR_PIN` in `uvr-install.sh` via `github-releases`. |
## Behaviour notes
- **`weekly-audit-missing` still takes bincraft from the default branch**, not the latest release tag, matching what the `git::` pak call did. Called out in a comment rather than silently changed.
- **The uvr pin is repo-wide.** bincraft resolves `uvr` from `PATH` and pins no version of its own, so `UVR_PIN` in `uvr-install.sh` governs the whole pipeline.
- **Persistent caches now also benefit bincraft**, which reads `UVR_CACHE_DIR`/`UVR_PACKAGES_DIR` from the inherited pipeline environment.
- **`uvr sync` will not prune the shared library.** Pruning is disabled whenever `--library` is passed (`do_prune = prune && library_override.is_none()`), so `/mnt/cache/R-pkgs` keeps bincraft and its dependencies. The wipe-on-ABI-mismatch path is *not* similarly guarded, which is why the helper pins the manifest to the active R's exact version.
- **`plans/` and `specs/` are untouched** — they are dated records of decisions made in June 2026 and describe bincraft's then-pak-based internals; rewriting them would misstate history.
## Verification
`shellcheck`, all pre-commit hooks (on this commit's file range) and the `local/tests/` suite (100 assertions) pass.
Not yet exercised in CI: the build-env images do not ship `uvr`, so the per-step `curl install.sh` bootstrap is untested against a real image. Worth a manual `build-all-versions-install-deps` run before merging.
Reviewed-on: #147
## Summary
Builds R-minor-sensitive CRAN packages once per installed R minor version (into per-minor S3 slots `…/contrib/<x.y>/`) and everything else once into the generic slot, driven by bincraft 4.2.0's ABI classifier. Both the full and iterative pipelines are covered.
## What's in here
**Detection / precompute**
- `local/r-minor-helpers.R` — pure `classify_from_metadata()` (NeedsCompilation / risky `LinkingTo`) + `parse_build_args()`, with unit tests.
- `local/packages-to-build.R` — adds a per-package `r_minor_sensitive` flag: cheap CRAN-metadata rules first, source download + `bincraft::needs_per_minor_recompile()` only for the ambiguous compiled subset (fail-safe to sensitive). Classified once per package, applied to all versions.
**Full build**
- `local/build-all.R` — passes the per-row `is_r_minor_sensitive` flag; new `--sensitive-only` mode builds just the risky subset.
- `.crow/build-all-versions-{amd64,arm64}.yaml` — install-deps persists the sensitive subset; build step runs a sensitive-only pass under each non-primary `/opt/R/*` minor; new step uploads the generic index plus a per-minor index for each minor.
**Iterative build**
- All 14 `.crow/process-updates-*.yaml` — primary pass uses `r_minor_detection = 'classifier'`; a sensitive-only multi-R pass builds risky updates under each other minor; per-minor index upload added.
**Tooling / housekeeping**
- Pins bincraft `v4.1.1` → `v4.2.0` across all workflows; removes the superseded standalone `build-r-minor-sensitive-packages.yaml`.
- Adds prek/pre-commit hooks (prettier, markdownlint, editorconfig-checker, yamllint, air) and applies them repo-wide; excludes the verbatim GPL `LICENSE.md` and auxiliary shell scripts.
- Design + implementation docs under `docs/superpowers/`.
## Requires before merge
- A `v4.2.0` git tag must be pushed on the bincraft repo (codefloe.com/rpkgs/bincraft) — the workflow install steps pin `@v4.2.0`. The full-build install-deps clones `main`, so it is unaffected.
Reviewed-on: #84
s3fs uses reticulate/Python which significantly increases R process memory.
When pak tries to fork R to install package dependencies, the enlarged
process can't be duplicated within the container memory limit.
Now the install-deps step saves the S3 file listing as s3_cache.rds,
and the build step reads it with readRDS() — no s3fs loading needed.
The precompute script was constructing S3 paths as "redhat10" while
bincraft uses "rhel10" as the codename. This mismatch caused the
precompute to find no existing packages in S3, producing ~77K false
positive package versions that all get skipped at build time.
Now uses bincraft::set_codename() to ensure path alignment.
Also fixes nrow() vs length() in install-deps summary message.
packages-to-build.R now saves the full (Package, Version) data.table
so the build step can pass specific tags to build_binary_package(),
eliminating redundant per-package tag discovery and S3 checks.
Also reduces archive versions from 9 to 4 (+ 1 release = 5 total).
The S3 credentials, endpoint, and bucket name were still pointing to the old Hetzner storage after the Backblaze migration, causing silent build failures.
