915 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
pat-s	a126d74cd3	fix(build): keep source fallbacks out of the S3 package cache (#159) ... ## Problem This is the gap flagged in rpkgs/bincraft#106. `build_binary_package()` has a fast path that compares against `s3_package_cache` instead of querying S3 per package, and that cache is produced here: ```r s3_pkgs <- s3fs::s3_dir_ls(".../latest/src/contrib", recurse = TRUE) saveRDS(basename(s3_pkgs), "/mnt/cache/packages/s3_cache.rds") ``` A raw bucket listing cannot tell a binary from a package whose build failed and was published as its CRAN source — the two occupy the same key. So every source fallback reads as "already built" and is skipped for good. That is how `alpine324` accumulated ~13.5k of them. The same listing feeds `s3_dt`, which is subtracted from the build list at line 194 (`pkgs <- pkgs_no_error[!s3_dt]`). That one matters more: it excludes the very packages that need building, before `build_binary_package()` is even called. ## What this changes Drops from the listing every object the slot's own index reports as served from source. bincraft leaves the `Built` stamp off exactly those records (rpkgs/bincraft#105), so the index already carries the answer and no credentials, downloads or extra API calls are needed. Both consumers are fixed: the saved cache and `s3_dt`. The cache stays a plain filename vector, so the build container still needs no `s3fs`/reticulate — that was the point of saving it in the first place. Two deliberately conservative edges: - archived objects have no index record, so they are kept. Unknown means binary, never "rebuild it". - if the index cannot be read, the full listing is kept and a warning is printed, so a CDN blip cannot mass-schedule a rebuild. ## Verification The script parses, and the new block run against the live indices: ``` amd64/alpine324: index=24235 source-served=13542 e.g. AATtools_0.0.3.tar.gz, ABCDscores_7.0.0.tar.gz amd64/noble: index=24681 source-served=0 ``` `alpine324` is re-indexed by bincraft 5.1.1, so 13 542 objects drop out and those packages become buildable. `noble` has not been re-indexed yet, so every record still carries `Built`, nothing is dropped, and its behaviour is exactly what it is today — the safe failure mode this relies on. The new log line makes it visible per run: ``` S3 cache: N objects, M served as CRAN source, K usable binaries ``` ## Sequencing Needs rpkgs/bincraft#106 (and a release) before a rebuild actually builds: this fixes the bulk build path's list, #106 fixes the per-package pre-build skip. Reviewed-on: #159	2026-08-09 16:27:51 +00:00
pat-s	f8e31af75b	fix(ci): make every manual gate default to a value that matches nothing (#158) ... ## Problem A manual `crow pipeline create` instantiates **every** file in `.crow/`, and a declared variable default is applied even when the run never passed that variable. A gate is therefore only a gate if its default matches nothing. #155 fixed the three pipelines that had no manual gate at all. It missed that a *permissive default* leaves a pipeline just as exposed. Demonstrated the expensive way: creating a pipeline with only ``` --var weekly_audit_missing=alpine-324-amd64 ``` also started `build-all-versions` — because its gate `target_arch` defaults to `amd64`, which matches its own amd64 matrix rows — and `process-updates` across every row, because that gate defaults to `all`. The run was killed before any `Upload package indexes` step produced output and both alpine324 indices were verified unchanged, but `build-all-versions` uploads binaries and rewrites indexes, so the next one might not be caught in time. Before: | pipeline | gate | default | fired on an unrelated manual run | | --- | --- | --- | --- | | `build-all-versions` | `target_arch` | `amd64` | amd64 rows — builds and uploads | | `build-all-versions-install-deps` | `target_arch` | `amd64` | amd64 rows | | `weekly-rebuild-missing` | `weekly_rebuild_missing` | `all` | every row | | `weekly-audit-missing` | `weekly_audit_missing` | `all` | every row | | `process-updates` | `process_cran_updates` | `all` | every row | | `repair-built-stamp` | `repair_built_stamp` | `arm64` | arm64 rows | `archive-missed-packages` was the one that behaved, because its gate variable is never declared and so matches nothing. That is the property this restores everywhere. ## What this changes Each of the six gets a `none` option on its gate variable and defaults to it, so a manual run has to name its target explicitly. The reason is recorded next to the default, where someone would go to change it. `none` is used rather than dropping the default so the expression always has a defined value to compare, instead of relying on undefined-variable semantics. Cron triggers are untouched — they match on the `cron:` name, not the variable. ## Verification `crow lint .crow/` reports all ten configs valid. Auditing every pipeline that accepts a manual event: ``` archive-missed-packages.yaml: gate=task default=<none> auto-apply-patches.yaml: gate=auto_apply_patches default='false' build-all-versions-install-deps.yaml gate=target_arch default=none build-all-versions.yaml: gate=target_arch default=none process-updates.yaml: gate=process_cran_updates default=none repair-built-stamp.yaml: gate=repair_built_stamp default=none trial-build-registry.yaml: gate=trial_build_registry default='false' weekly-audit-missing.yaml: gate=weekly_audit_missing default=none weekly-patch-proposals.yaml: gate=weekly_patch_proposals default='false' weekly-rebuild-missing.yaml: gate=weekly_rebuild_missing default=none ``` Every gate now defaults to something that matches no matrix row. Reviewed-on: #158	2026-08-09 15:31:14 +00:00
pat-s	0703969088	fix(audit): count a source fallback as a missing binary (#157) ... ## Problem The audit's definition of "missing" was `missing_dt <- cran_dt[!s3_dt]`, where `s3_dt` came from listing the bucket. But an object being present does not mean a binary was built: when a build fails, bincraft publishes the CRAN source tarball in its place so the package stays installable, and a listing cannot tell the two apart. So every failed build looked done. On `amd64/alpine324` that is 13 547 of 24 134 comparable objects (56%) — all CRAN sources from the June 2026 bootstrap, none ever reported here, so `weekly-rebuild-missing` never retried any of them. `amd64/noble` sits at 4.6% for comparison. ## What this changes Reads the slot's own `PACKAGES` index instead of listing the bucket, and counts only records carrying a `Built` stamp. bincraft writes that stamp only for what it actually built (rpkgs/bincraft#105), so the index answers the question a listing cannot. Side effects of reading the index rather than the bucket: - the audit reports how many records the index holds and how many are served as source, so the gap is visible in the log - `s3fs` is no longer used, so the audit needs no B2 credentials - an object present but absent from the index is now counted as missing, which is correct: clients only see the index ## Sequencing This is inert until the matching bincraft change ships and a slot is re-indexed. A slot last indexed by an older bincraft carries `Built` on every record, including the fallbacks, so the audit reports exactly what it does today. Verified against the live indices: ``` amd64/alpine324: records=24152 built=24152 source-served=0 amd64/noble: records=24681 built=24681 source-served=0 ``` After a re-index those `source-served` counts become the real ones, and the packages behind them start appearing in issue #63 and getting rebuilt. Reviewed-on: #157	2026-08-09 10:37:41 +00:00
pat-s	0c86b2692d	fix(cdn): declare the User-Agent cache vary instead of dropping it (#156) ... ## Problem `tofu plan` after importing the middleware script shows: ``` ~ resource "bunnynet_pullzone" "cran_rpkgs_com" { ~ cache_vary_headers = [ - "User-Agent", ] ``` The zone carries `cache_vary_headers = ["User-Agent"]`, set before this configuration existed. `cdn.tf` never declared it, so the first apply of the managed middleware would remove it — as a side effect of an unrelated change, with no decision recorded anywhere. ## What this changes Declares the attribute with its current value, so the pull zone is a no-op in that plan. ## Why keep it rather than let it go On paper the router makes it redundant. The only UA-dependent responses it produces are redirects, and those carry `Cache-Control: no-store`; their targets are concrete per-slot, per-minor URLs whose content depends only on the path. Dropping the vary would also be a genuine win, since otherwise every distinct R version string (`R (4.5.3 x86_64-pc-linux-musl …)`) keys its own copy of every tarball. It stays anyway, for now: - it is the second line of defence against the one failure mode that would be quiet and confusing — an R 4.6 client served the 4.5 index - Bunny honouring `no-store` on an edge-script response has been confirmed for today's redirects (`cdn-cache: BYPASS` on `max-age=0`), but not for the new script in production - keeping it is the status quo, so it cannot regress anything Removing it is worth doing on its own, once per-minor routing is confirmed live and the redirects can be observed bypassing cache — not as a side effect of enabling that routing. ## Verification `tofu validate` passes. Re-planning after this merges should leave `bunnynet_pullzone.cran_rpkgs_com` unchanged, reducing the plan to the script `content` update and the new `UNION_SLOTS` variable. Reviewed-on: #156	2026-08-09 10:08:40 +00:00
pat-s	1e843523a0	fix(ci): gate the three ungated pipelines on their own variable (#155) ... ## Problem A manual `crow pipeline create` instantiates **every** pipeline in `.crow/`, and each one decides for itself whether to run. Three had nothing to decide with — their only manual condition was a bare `event: manual`: - `auto-apply-patches` — pushes to `auto/registry-patch-proposals` and opens/updates a PR - `weekly-patch-proposals` — posts and edits two Forgejo issues - `trial-build-registry` — starts a build per matrix row, on both arches So they fired on *any* manual trigger in this repo, whatever it was for. That is how they came to run alongside a manual `process-updates` run for `alpine-324-arm64` (#10723), which is also why that pipeline is marked failure. `repair-built-stamp.yaml` already documents the rule this breaks: > The gate variable is `repair_built_stamp`, not `target_arch` … Every pipeline here gates on a variable named after itself for exactly that reason. ## What this changes Each of the three gets a gate variable named after the pipeline, `evaluate`d on the manual event, defaulting to off: ```yaml variables: auto_apply_patches: description: 'Run the auto-patch proposer. Also gates this pipeline.' options: ['true', 'false'] default: 'false' when: - event: manual evaluate: 'auto_apply_patches == "true"' - event: cron cron: auto-apply-patches ``` Cron triggers are untouched, so the scheduled runs behave exactly as before. The run-manually comments in all three headers were also stale: they documented `--var task=<name>` with `woodpecker-cli`, and no pipeline evaluates a `task` variable. They now show the real invocation. ## Note on the sibling pipelines The already-gated pipelines use `default: all` (e.g. `weekly_rebuild_missing`). If Crow applies a declared default to a variable that an API-created pipeline never passed, those would match on an unrelated manual run too — `weekly-rebuild-missing` would be an expensive way to find out. I could not settle that from #10723 because its step logs have since expired, so I left them alone rather than guess. The three fixed here default to `'false'`, which is safe under either semantics. ## Verification `crow lint .crow/` passes. Auditing every pipeline that accepts a manual event now reports a gate on all ten: ``` build-all-versions-install-deps.yaml: gated auto-apply-patches.yaml: gated weekly-patch-proposals.yaml: gated weekly-audit-missing.yaml: gated repair-built-stamp.yaml: gated archive-missed-packages.yaml: gated weekly-rebuild-missing.yaml: gated trial-build-registry.yaml: gated build-all-versions.yaml: gated process-updates.yaml: gated ``` Reviewed-on: #155	2026-08-09 10:02:05 +00:00
automation-bot	4cb8b692ab	chore(deps): update pre-commit hook editorconfig-checker/editorconfig-checker to v3.11.1	2026-08-09 00:32:28 +00:00
automation-bot	6bc8213a2f	chore(deps): update pre-commit hook editorconfig-checker/editorconfig-checker to v3.10.0	2026-08-08 00:32:07 +00:00
pat-s	2f732457d2	feat(edge): route PACKAGES requests to the per-R-minor slot (#152) ... ## Problem `install.packages("curl")` fails in `reg.devxy.io/r/r-alpine:4.5-3.24` with "package 'curl' is not available for this version of R", on both arches. `curl` is not missing from the repo: it is in `…/latest/src/contrib/4.5/` and `…/4.6/`, the per-minor slots that base R cannot address. The image's repo URL resolves to `…/latest/src/contrib`, whose index does not list it. On `amd64/alpine324` that is 2 886 packages invisible to `install.packages()` (23 on `amd64/noble`) — what issue #63 records as "missing binaries". Two further findings while investigating: - The middleware only ever rewrote the bare `cran.rpkgs.com/src/contrib/…` form, and that form was broken for every Linux client on a stock R user agent: `ALPINE_REGEX`/`UBUNTU_REGEX`/`RHEL_REGEX` need a Posit-style UA that carries the distro, so stock R fell through to `extractOs()` and got redirected to `/amd64/linux-musl/latest/…`, a slot that does not exist. - `PACKAGES*` is served `cdn-cache: BYPASS` (bincraft uploads it `no-store`), so the middleware sees every index request and no purge is needed for routing changes to take effect. ## What this changes **`edge/rpkgs-router.ts`** — the middleware, now a reviewed file in this repo rather than dashboard state. It routes `PACKAGES`, `PACKAGES.gz` and `PACKAGES.rds` into `…/src/contrib/<x.y>/` for slots listed in `UNION_SLOTS`, and nothing else. Tarballs are deliberately left alone. R keeps the `contriburl` it *asked for*, not the one the redirect served it, so every tarball URL is resolved against the flat directory and the union index steers the per-minor ones with a `Path: <x.y>` field. Rewriting a tarball request here would send flat-slot packages into a directory that does not hold them. Also in the script: the phantom `linux-gnu`/`linux-musl` fallback is gone (an unidentifiable distro goes to CRAN, as an unparseable UA already did), and every redirect carries `Cache-Control: no-store` since its target depends on the User-Agent. The macOS branches are unchanged. **`cdn.tf`** — `bunnynet_compute_script.rpkgs_router` with `content = file("edge/rpkgs-router.ts")`, the `UNION_SLOTS` variable, and `middleware_script` pointing at the resource instead of the literal `29277`. `UNION_SLOTS` is empty, so merging and applying this changes no client's behaviour. A slot is added only once bincraft has republished its per-minor index as a union (rpkgs/bincraft#97); routing to a raw per-minor index would hide every package it does not carry. Rolling back is a variable edit, not a deploy. **`specs/`, `plans/`** — the design and the implementation plan, including the two approaches that were rejected (edge-side merge, moving the minor up the path) and why. ## Verification `just edge-test` runs 13 routing cases against the SDK's local server, so what is tested is the artifact that gets deployed; pass-through cases proxy to the real origin. All pass. End to end, with the middleware in front of a locally built union index for `amd64/alpine324` (31 507 records), inside the runtime image: ``` curl: 7.1.0 -> …/latest/src/contrib/4.5 -> curl_7.1.0.tar.gz 717 725 B jsonlite: 2.0.0 -> …/latest/src/contrib -> jsonlite_2.0.0.tar.gz 1 055 849 B ``` `tofu validate` passes. `tofu plan` has not been run: no `BUNNYNET_API_KEY` available in this environment. ## Before applying The script pre-dates this configuration, so it must be adopted, not created: ```sh tofu import bunnynet_compute_script.rpkgs_router 29277 tofu plan ``` The plan should show an in-place `content` update and no replacement of the pull zone. Without the import, tofu creates a second script and repoints the zone at it. Note that `name = "rpkgs-router"` will rename the existing script on apply. ## Not fixed here `install.packages("curl")` on `alpine324` will now *resolve*, then fail to build: that slot's tarballs are byte-identical CRAN **source** tarballs (no `Meta/`, no `Built:` in DESCRIPTION) which the index nevertheless stamps `Built: R 4.5.3; …-linux-musl`. Sampled: `amd64/alpine324` 3/12 binary, `arm64/alpine324` 13/30, `amd64/noble` 12/12, `amd64/alpine323` 17/20. That slot needs a rebuild, tracked separately. Reviewed-on: #152	2026-08-07 14:12:22 +00:00
pat-s	77ce1d212d	fix(ci): gate the Built-stamp repair on its own variable (#151) ... ## Problem `repair-built-stamp.yaml` gated on `target_arch`: ```yaml when: - event: manual evaluate: 'target_arch == "${ARCH}"' ``` That is the same variable `build-all-versions.yaml` and `build-all-versions-install-deps.yaml` gate on. A manual run passing `target_arch=arm64` to reach the repair therefore matched all three, so triggering a dry-run repair also queued a full arm64 rebuild. I hit this triggering the alpine 3.22 dry run (pipeline 10706), which I killed. `crow pipeline ps` renders empty states on this version, so I could not confirm from the CLI whether the rebuild workflows started before the kill or only sat queued behind the running cron jobs; no output was attributable to them. ## Change Gate on `repair_built_stamp` instead. Every other pipeline here already gates on a variable named after itself (`process_cran_updates`, `weekly_audit_missing`, `weekly_rebuild_missing`), and `target_arch` was the odd one out being shared by two. The header now records the collision and the exact invocation, so the next pipeline added here does not repeat it. ## Verification `crow lint` passes; `prek` clean. Grep of every trigger condition in `.crow/` confirms `repair_built_stamp` is unique and that no other pipeline gates on `OS`, `OS_VERSION`, `R_VERSION` or `dry_run` alone: ``` archive-missed-packages.yaml: task == "archive-missed-packages" build-all-versions.yaml: target_arch == "${ARCH}" build-all-versions-install-deps.yaml: target_arch == "${ARCH}" weekly-audit-missing.yaml: weekly_audit_missing == ... repair-built-stamp.yaml: repair_built_stamp == "${ARCH}" process-updates.yaml: process_cran_updates == ... weekly-rebuild-missing.yaml: weekly_rebuild_missing == ... ``` ## Note for whoever runs it `crow pipeline create` against this instance returned HTTP 504 while still creating the pipeline. Retrying on that error duplicates the run: I created four before noticing. Verify with `pipeline ls` rather than trusting the exit status. Reviewed-on: #151	2026-08-07 09:41:53 +00:00
pat-s	0da29ab3f6	fix(index): add a slot repair for a broken Built stamp and retire alpine 3.21 (#150) ... ## Problem `arm64/alpine321` and `arm64/alpine322` advertise a broken stamp: ``` arm64/alpine321 :: 22930 Built: R 4.4.0; NA; 2026-07-31 13:35:52 UTC; unix arm64/alpine322 :: 24696 Built: R 4.5.0; NA; 2026-07-31 13:51:54 UTC; unix ``` The per-minor sub-slots (`contrib/4.4`, `4.5`, `4.6`) are affected too. All 18 other slots are correct. uvr matches the stamp's platform triple plus R minor to pick binary over source, so nothing matches `NA` and both slots silently serve as source-only, which is exactly the regression bincraft#85 added the stamp to prevent. `install.packages()` is unaffected, since it reads `Built:` from each tarball's own `DESCRIPTION`. The tarballs are fine (`arm64/alpine322/.../dress.graph_0.8.3.tar.gz` carries `aarch64-unknown-linux-musl`), and so is the R that built them (`r-4.5.0_1_aarch64.apk` ships `R_PLATFORM='aarch64-unknown-linux-musl'`). Only the index is wrong. bincraft#96 stops a stamp like this being written again, but it cannot repair what is already there. ## Why not just re-run the index update `upload_package_index()` reuses the slot's remote `PACKAGES.db`, and cranlike's `update_db()` only reparses files whose md5 changed, so entries already in the database keep the stamp they were written with. Dropping `PACKAGES.db` to force a full reparse does work, and it is what bincraft#85's rollout note suggested, but for an S3 repo cranlike reads each package's metadata from the CRAN *source* mirror on GitHub. A 25k-entry slot is then 25k requests to raw.githubusercontent.com, with a real risk of being rate-limited part-way through and leaving the slot half-written. Only the `Built` column is wrong, so this corrects it in place instead: patch the column in `PACKAGES.db`, put the database back, and let `upload_package_index()` re-emit `PACKAGES*` from it. `update_db()` always rewrites the index files even when nothing was reparsed, so no tarball is re-read and nothing is fetched from GitHub. ## Change - `local/repair-built-stamp.R` — repairs the generic slot plus every per-minor sub-slot. Dry-run by default; `--apply` writes. The replacement comes from `bincraft::built_stamp()` under the R running the script, so it is exactly what a healthy run would have written, and bincraft#96's guard makes a broken build image fail rather than write a second bad stamp. - `.crow/repair-built-stamp.yaml` — manual pipeline, routed by `target_arch` to the matching agent group and platform image, with `dry_run` defaulting to `true`. - `.crow/archive-missed-packages.yaml` — drop the two `alpine321` matrix entries. Alpine 3.21 is EOL: the website advertises only 3.23/3.24 and `process-updates.yaml` already dropped it, so that slot is retired rather than repaired. ## Verification `crow lint .crow/` passes on all nine pipelines. `air format` and `jarl check` are clean; the script parses, and the `/opt/R` minor-version derivation was checked against `4.4.3 / 4.5.3 / 4.6.0 / current` → `4.4 4.5 4.6`. The repair itself is unrun by design — it needs B2 credentials and an arm64 agent. ## Rollout 1. Cut a bincraft release so `local/install-bincraft.R` picks up #96 (it resolves the latest `vX.Y.Z` tag, and #96 is only on `main`). 2. Run this pipeline with `target_arch=arm64`, `OS=alpine`, `OS_VERSION=3.22`, `R_VERSION=4.5.3`, `dry_run=true` and check the reported counts. 3. Re-run with `dry_run=false`. 4. Confirm: `curl -sS https://cran.devxy.io/arm64/alpine322/latest/src/contrib/PACKAGES | grep '^Built:' | sort | uniq -c` `arm64/alpine321` is deliberately left alone. Reviewed-on: #150	2026-08-07 09:15:04 +00:00
automation-bot	02dd5a695d	chore(deps): update terraform bunnynet to ~> 0.17	2026-08-05 00:32:29 +00:00
automation-bot	e02be7518f	chore(deps): update dependency nbafrank/uvr to v0.4.5	2026-08-04 02:32:01 +00:00
pat-s	ee15c50f53	refactor: migrate package installation from pak to uvr (#147) ... `bincraft` dropped pak in favour of uvr (5.0.x, "Dependencies and their system requirements are now installed with `uvr` instead of pak during `build_binary_package()`"), so the pipelines, helper scripts and images in this repo move with it. ## Approach uvr is project-scoped in a way pak is not: `uvr add` refuses to run outside a project and always installs into `.uvr/library/`, and only `uvr sync` honours `--library`. So there is no one-line `pak::pak(...)` equivalent. `local/uvr-install.sh` encapsulates the dance — bootstrap a pinned uvr, mint a throwaway project under `TMPDIR`, `uvr add --no-install`, then `uvr sync --library <target>`. Keeping the project outside the checkout also keeps `uvr init`'s `.Rprofile` from hijacking `.libPaths()` for every other R call in the pipeline. This matches what bincraft itself does (`uvr sync --install-system-deps --library <lib>`), and bincraft requires `uvr` on `PATH`, which the bootstrap provides: every pipeline that calls `bincraft::` runs `install-bincraft.R` (and therefore the bootstrap) first. ## Changes | File | Change | | --- | --- | | `local/uvr-install.sh` | **New.** The single replacement for `pak::pak(...)`. Bootstraps uvr `v0.4.4`, resolves the R interpreter from `UVR_R_BIN`/`R_VERSION`/`PATH`, pins the manifest to that R's exact version, and syncs into `UVR_TARGET_LIB`/`R_LIBS_USER`. | | `local/install-bincraft.R` | Installs `forgejo::codefloe.com/rpkgs/bincraft@<tag>` instead of a `git::` URL; keeps the `git ls-remote` tag resolution. Exports `UVR_R_BIN`/`UVR_TARGET_LIB` from `R.home()`/`.libPaths()[1]` so the per-R-minor passes target their own R and library. | | `.crow/auto-apply-patches.yaml`, `.crow/weekly-patch-proposals.yaml`, `.crow/weekly-audit-missing.yaml`, `.crow/weekly-rebuild-missing.yaml`, `.crow/build-all-versions-install-deps.yaml` | `pak::pak(...)` → `UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh ...`. The explicit `UVR_R_BIN` matters in `build-all-versions-install-deps.yaml`, which has no `R_VERSION` in its step environment. | | `.crow/build-all-versions.yaml`, `.crow/process-updates.yaml`, `.crow/weekly-rebuild-missing.yaml` | `R_PKG_CACHE_DIR` → `UVR_CACHE_DIR` + `UVR_PACKAGES_DIR` on the same `/mnt/cache` volume, preserving the amd64-off/arm64-on split. Drops the `rm -rf .../pkgcache/_metadata/...` cleanup. | | `local/r-minor-helpers.R`, `local/build-all.R`, `local/tests/test-trim-pkgcache.R` | Removes `trim_pkgcache_metadata()`, its every-25-packages call and its tests. uvr's cache does not mint a fresh ~70 MB snapshot per `PACKAGES` change. | | `.crow/build-all-versions-install-deps.yaml` | Drops `pak::sysreqs_db_update()`; uvr resolves sysreqs from its vendored `r-system-requirements` rules via `--install-system-deps`. | | `docker/Containerfile-shiny-app` | Bootstraps uvr and drives both dependency installs through one uvr project with `UVR_LIBRARY` pointed at the image's R library. | | `docker/build-one.Dockerfile` | Ships `uvr-install.sh` at `/work/local/` so `install-bincraft.R` finds it. | | `docker/reprex/alpine.sh` | Replaces `pak::local_install_deps()` with DESCRIPTION parsing + `uvr add`. | | `local/test-package-loading.R` | Installs via the helper instead of `pak::pkg_install()`. | | `README.md` | Documents uvr for sysreq inference, archived-version installs and cache clearing. | | `renovate.json` | Tracks the `UVR_PIN` in `uvr-install.sh` via `github-releases`. | ## Behaviour notes - **`weekly-audit-missing` still takes bincraft from the default branch**, not the latest release tag, matching what the `git::` pak call did. Called out in a comment rather than silently changed. - **The uvr pin is repo-wide.** bincraft resolves `uvr` from `PATH` and pins no version of its own, so `UVR_PIN` in `uvr-install.sh` governs the whole pipeline. - **Persistent caches now also benefit bincraft**, which reads `UVR_CACHE_DIR`/`UVR_PACKAGES_DIR` from the inherited pipeline environment. - **`uvr sync` will not prune the shared library.** Pruning is disabled whenever `--library` is passed (`do_prune = prune && library_override.is_none()`), so `/mnt/cache/R-pkgs` keeps bincraft and its dependencies. The wipe-on-ABI-mismatch path is *not* similarly guarded, which is why the helper pins the manifest to the active R's exact version. - **`plans/` and `specs/` are untouched** — they are dated records of decisions made in June 2026 and describe bincraft's then-pak-based internals; rewriting them would misstate history. ## Verification `shellcheck`, all pre-commit hooks (on this commit's file range) and the `local/tests/` suite (100 assertions) pass. Not yet exercised in CI: the build-env images do not ship `uvr`, so the per-step `curl install.sh` bootstrap is untested against a real image. Worth a manual `build-all-versions-install-deps` run before merging. Reviewed-on: #147	2026-07-31 12:19:26 +00:00
pat-s	fe2289af56	fix(patches): force RcppParallel to use its bundled oneTBB (#146) ... ## Why `RcppParallel` binaries built on ubuntu 22.04 still record `NEEDED libtbb.so.12` and cannot `dyn.load` without the distro TBB, although #145 landed the link-order patch and build-env-images dropped `TBB_INC`/`TBB_LIB` the same morning. The link-order fix does not survive a build image that exports those variables, and the published images still do: `.crow/images.yaml` in build-env-images triggers on `cron`/`manual` only, so every image in the registry is still the 2026-07-21 build. With `TBB_LIB` set, `configure.R` never reaches the bundled-oneTBB branch at all, `install.libs.R` fills `RcppParallel/lib` with absolute symlinks into the image's library dir, and the plain `-L` from #145 just points at the system TBB instead of the bundled one. Nothing on the pipeline side can undo this: R reads `~/.Renviron` **after** the process environment, so neither a Crow `environment:` block nor the registry `env` field wins. ``` # the image, not the Containerfile docker run --rm reg.devxy.io/rpkgs/build-env-ubuntu:jammy sh -lc 'grep TBB ~/.Renviron' TBB_INC=/usr/include/oneapi TBB_LIB=/usr/lib/x86_64-linux-gnu ``` Rather than couple correctness to image freshness for a third round-trip, the patch now forces the bundled TBB itself. ## Changes - `local/patches/RcppParallel/bundled-tbb-link-order.patch` -> `local/patches/RcppParallel/force-bundled-tbb.patch`. Keeps the `-Wl,-L` -> `-L` link-order hunk (the bundled branch has the same problem when a distro libtbb is installed) and adds an opt-out, `BINCRAFT_ALLOW_SYSTEM_TBB=TRUE`, for the ambient `TBB_ROOT`/`TBB_LIB`/`TBB_INC`. - All four read sites have to move together; patching only `configure.R` builds the bundled TBB and then **segfaults R on the post-install load test**: - `tools/config/configure.R`: branch selection. - `src/install.libs.R`: `R CMD INSTALL` runs it outside the `tbb` rule in `src/Makevars` that passes the configured values, so at install time it still sees the image environment. - `R/tbb.R` `tbbLibraryPath()`: run time. `.onLoad()` would `dyn.load` the system TBB into the global scope next to the bundled one that `RcppParallel.so` needs - two copies of the same symbols, hence the segfault. - `R/tbb.R` `tbbCxxFlags()` / `tbbLdFlags()`: otherwise dependents such as `rstan` get the system TBB handed back to them. - `local/patches/registry.json`: new patch path and a `reason` describing both failure modes. ## Verification Built through `bincraft:::prepare_patched_repo()` on the **current, unfixed** images (all three still export `TBB_INC`/`TBB_LIB`), with bincraft v5.0.1, i.e. what CI deploys: | image | `NEEDED` | rpath | `RcppParallel/lib` | load, all system `libtbb*` moved away | | --- | --- | --- | --- | --- | | `build-env-ubuntu:jammy` | `libtbb.so` | `$ORIGIN/../lib` | real `libtbb.so.2`, 5.0 MB | OK, 12 threads | | `build-env-redhat:9` | `libtbb.so` | `$ORIGIN/../lib` | real `libtbb.so.2`, 5.1 MB | OK, 12 threads | | `build-env-alpine:3.24` | `libtbb.so` | `$ORIGIN/../lib` | real `libtbb.so.2`, 4.7 MB | OK, 12 threads | For contrast, the same jammy build without this change: ``` NEEDED libtbb.so.12 RUNPATH /usr/lib/x86_64-linux-gnu:$ORIGIN/../lib lib/ libtbb.so.12.5 -> /usr/lib/x86_64-linux-gnu/libtbb.so.12.5 (dangling off the image) load libtbb.so.12: cannot open shared object file: No such file or directory ``` With `TBB_LIB` still exported, `RcppParallel::tbbLibraryPath()` resolves to the package's own `lib`, and `CxxFlags()` emits the package's own `include` plus `-DTBB_INTERFACE_NEW` - not `/usr/include/oneapi`. ## Behaviour change Published `RcppParallel` binaries carry their own oneTBB on every platform, whichever image version CI pulls. The patch content changed, so the patched-binary cache key changes with it and no stale entry is reused. The build-env-images fix is still worth rolling out (those images also carry the broken el8 `uvr` and uvr 0.4.1), but RcppParallel no longer waits on it. ## Follow-up, not in this PR The binaries already in B2 are the broken ones; they need a rebuild, and a Bunny `/purge` does not evict Perma-Cache. Reviewed-on: #146	2026-07-31 09:48:41 +00:00
pat-s	98371fb9c5	fix(patches): replace the RcppParallel system-TBB patch with a link-order fix (#145) ... ## Why Every build logs: ``` ! Patch for RcppParallel 6.2.0 did not apply cleanly; skipping patched build. ``` RcppParallel 6.x rewrote its build system. `src/Makevars.in` no longer contains the `USE_TBB=Linux` block that `system-tbb.patch` edited (it is now a short `@VAR@` template driven by `tools/config/configure.R`), so the patch can never apply again. The workaround it implemented is obsolete too. 6.x bundles **oneTBB 2022** and builds it with cmake, which works on musl and with g++ 8-15, so the 5.x reason for linking a system TBB is gone. Linking one is now harmful: `install.libs.R` symlinks the system libraries into `RcppParallel/lib`, so the published binary depends on a TBB the consumer does not have — the same failure mode as `fs` and libuv. What is still broken upstream is the **link order**. `configure.R` names the TBB directory with `-Wl,-L`, and gcc expands its own search dirs into `-L` options ahead of anything forwarded verbatim with `-Wl,`: ``` $ gcc -v -o t t.c -Wl,-L/usr/local/lib64 -ltbb -L/usr/lib/gcc/x86_64-redhat-linux/8 -L/lib/../lib64 -L/usr/lib/../lib64 ... -L/usr/local/lib64 ``` So on any build host with a distro TBB installed, `-ltbb` resolves to that library and `RcppParallel.so` records *its* SONAME (`libtbb.so.12`, or `libtbb.so.2` for the classic Intel TBB on el8/el9) instead of the bundled `libtbb.so`. ## Changes - Replace `RcppParallel/system-tbb.patch` with `RcppParallel/bundled-tbb-link-order.patch`, which changes the three `-Wl,-L` occurrences in `tools/config/configure.R` to plain `-L`. - Scope the RcppParallel entry to `>=6.0.0` and widen `platforms` to `*`. - Drop the `rstan` entry: its `-DTBB_INTERFACE_NEW` is already emitted by `RcppParallel::CxxFlags()` once the bundled oneTBB is used, and its `-I/usr/local/include` was an el8/el9 path applied on every platform. - `.pre-commit-config.yaml`: exclude `local/patches/*.patch` from `trailing-whitespace`, `end-of-file-fixer` and `editorconfig-checker`. They rewrite blank context lines (` ` -> ``) in every diff in the registry; `git apply` happens to tolerate it today, but a patch with meaningful trailing whitespace would be silently corrupted. The excludes are per-hook so `validate patch registry` still runs. - README: the RcppParallel example described the 5.x problem. ## Verification Built in the published images: | image | NEEDED | RPATH | loads with system libtbb removed | | --- | --- | --- | --- | | `build-env-alpine:3.24` | `libtbb.so` | `$ORIGIN/../lib` | yes | | `build-env-redhat:8` | `libtbb.so`, `libtbbmalloc.so` | `$ORIGIN/../lib` | yes | | `build-env-ubuntu:noble` | `libtbb.so` | `$ORIGIN/../lib` | yes | Without the patch the same builds record `libtbb.so.12` (alpine, ubuntu) or `libtbb.so.2` (el8, the classic 2018 TBB) and fall back to the system library at load time. rstan 2.32.7 compiles and loads against the patched RcppParallel on ubuntu noble with no makevars override and with the system libtbb moved away. On el8 it also compiles; loading it there is blocked by an unrelated image bug (see below). ## Behaviour change Requires the paired build-env-images PR (drops `TBB_INC`/`TBB_LIB`) and an image rebuild — with those env vars set, configure still takes the system-TBB branch. After that, RcppParallel binaries ship their own oneTBB and are self-contained. ## Also found, not fixed here - The `uvr lock failed ... GLIBC_2.29 not found` errors in the same log are the `-gnu` uvr artifact on el8's glibc 2.28; fixed in build-env-images. - `build-env-redhat:8` R 4.5.3/4.6.0 cannot load `stats` (`libRlapack.so: undefined symbol: dgemmtr_`): the el8 R RPM symlinks `libRblas.so` to openblas 0.3.15, which predates that symbol. R 4.4.3 and el9 are fine. Belongs in the R RPM build. Reviewed-on: #145	2026-07-31 06:41:46 +00:00
automation-bot	86707e3de4	chore(deps): update terraform bunnynet to ~> 0.16	2026-07-31 00:32:20 +00:00
pat-s	8d0c4b7ccd	fix(patches): widen RcppParallel system-tbb scope to the whole redhat family (#139) ... Extends #137. el8/el9 were excluded because they ship only classic Intel TBB. build-env-images now vendors oneTBB 2021 into `/usr/local` on el8/el9, so every platform has oneTBB 2021+. Widen the scope `redhat-10` → `redhat` (family token covers el8/el9/el10) and fix the reason text (no longer 'left on bundled'). **Requires the el8/el9 images rebuilt with the vendored oneTBB (build-env-images PR).** Verified: patched RcppParallel builds + links the vendored oneTBB on both el8 (g++ 8.5) and el9 (g++ 11.5). Reviewed-on: #139	2026-07-28 06:57:08 +00:00
automation-bot	463054625a	chore(deps): update pre-commit hook posit-dev/air-pre-commit to v0.11.0	2026-07-28 02:34:05 +00:00
automation-bot	c7293aefa1	chore(deps): update terraform bunnynet to v0.15.2	2026-07-28 02:33:47 +00:00
automation-bot	adcc74ba4f	chore(deps): update pre-commit hook rbubley/mirrors-prettier to v3.9.6	2026-07-28 00:32:55 +00:00
automation-bot	36c963eac9	chore(deps): update pre-commit hook davidanson/markdownlint-cli2 to v0.23.2	2026-07-28 00:32:39 +00:00
pat-s	736564a4ff	fix: prevent renovate bumps for fixed matrix assignments	2026-07-24 11:18:28 +02:00
pat-s	71f3f0c601	fix(audit): replace arch subsection in place instead of appending duplicates (#138) ... ## Problem A fresh `weekly-audit-missing` run followed by `weekly-rebuild-missing` reported that almost every package "already exists in the remote bucket", even though the audit had just flagged them as missing. Root cause: the audit's Forgejo-issue update matched the existing `### <arch>` subsection by its **bare** header (`### arm64`) while writing headers with a ` (N missing, M to rebuild)` suffix. The equality test never matched, so every run **appended** a new block instead of replacing the old one. Issue #63 had accumulated 38 arch subsections under `## alpine-323` where 2 are expected (75 total across the alpine platforms; body ~77k lines). `fetch-rebuild-packages-from-issue.R` reads the **first** matching block, which was the oldest snapshot. So the rebuild kept re-checking a months-old list (180 packages, mostly already built), while the genuinely-missing packages, ~1921 for alpine-323/arm64 in the freshest block, were never fed to the rebuild and the backlog grew silently. ## Changes - **`local/weekly-missing-binaries-audit.R`**: match arch subsections by prefix (`^### <arch>( |$)`) and remove **all** blocks for that arch before writing one fresh block. Accumulation now self-heals on every run. - **`local/dedupe-audit-issue.R`** (new): one-off cleanup that collapses each `## platform` section to the freshest block per arch across the three OS-family issues. Supports `DRY_RUN=1`. - **`local/fetch-rebuild-packages-from-issue.R`**: prefer the audit's freshly-written RDS (overwritten each run, immune to issue-body drift), falling back to issue parsing when absent. ## Validation Simulated the dedupe logic against the live #63 body: **75 → 8** arch subsections, body 77k → 28k lines, and the kept alpine-323/arm64 block correctly resolves to `GARCH.X (3.0)` (the stale first block held `2.0`). ## Follow-ups (not in this PR) - `alpine-321` is audited but has no row in the rebuild matrix (7,230 missing, never rebuilt). - An `alpine-324` section exists in the issue but is in neither matrix. Reviewed-on: #138	2026-07-21 08:51:23 +00:00
pat-s	3cc323d99e	fix(patches): widen RcppParallel system-tbb scope to all oneTBB-2021 platforms (#137) ... ## What Widen the RcppParallel `system-tbb` registry patch scope from `[alpine, ubuntu-2604]` to `[alpine, ubuntu, redhat-10]`. ## Why #135 fixed the TBB cluster on alpine + ubuntu-2604 only. A per-image audit showed the problem is broader: RcppParallel's bundled Intel TBB build fails on **every** build image with a modern toolchain (alpine/musl, ubuntu jammy/noble/resolute g++ 11/13/15, redhat-10 GCC 14), and only redhat-8/9 (older toolchains + pre-oneTBB system TBB) tolerate the bundled build. The system-TBB fix applies wherever the image ships **oneTBB 2021+** (has the `tbb::detail::r1` namespace rstan needs): | platform | system TBB | in scope? | |---|---|---| | alpine (all) | oneTBB 2022 | ✅ | | ubuntu 2204/2404/2604 | oneTBB 2021.5 / 2021.11 / 2022 | ✅ (via `ubuntu` family token) | | redhat-10 | oneTBB 2021.11 | ✅ | | redhat-8 | TBB 2018 (classic) | ❌ pre-oneTBB → bundled | | redhat-9 | TBB 2020.3 (classic) | ❌ pre-oneTBB, bundled works → bundled | ## Requires The matching build-env-images env (`TBB_INC`/`TBB_LIB`) extended to all ubuntu + el10: build-env-images #13. Ship together. ## Verified RcppParallel + rstan build and link (no `symbol not found`) on alpine, redhat-10, ubuntu noble and ubuntu resolute; RcppParallel builds on ubuntu jammy. Reviewed-on: #137	2026-07-20 20:59:47 +00:00
pat-s	61e1c0188d	chore: adjust weekly audit runs	2026-07-20 19:09:50 +02:00
pat-s	6df945d147	fix(patches): build RcppParallel against system oneTBB instead of disabling TBB (#135) ... ## What Replace RcppParallel's `disable-tbb` registry patch with `system-tbb`: build RcppParallel against the system oneTBB instead of stripping TBB entirely. ## Why `disable-tbb` skipped the bundled Intel TBB build by forcing the TinyThread backend, which also removed RcppParallel's TBB linkage. That broke every dependent that links TBB through `RcppParallelLibs()` -- `rstan` and the whole Stan cluster -- with `symbol not found: tbb::detail::r1::observe`. This is the root cause behind the large "blocked on RcppParallel" / rstan clusters (issues #115, #120): the packages were not individually broken, they were all waiting on one TBB-linkage regression. ## Change `system-tbb.patch` leaves `USE_TBB` unset (so the bundled build is still skipped on musl and g++ 15) but keeps the TBB backend and links the system oneTBB now shipped in the build-env images: ``` PKG_CXXFLAGS += -DRCPP_PARALLEL_USE_TBB=1 -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -DTBB_INTERFACE_NEW PKG_LIBS += -ltbb -ltbbmalloc ``` Scoped to `alpine` + `ubuntu-2604`, the only platforms where the bundled build fails; redhat and older ubuntus keep the bundled TBB. ## Requires The companion image change that exports `TBB_INC`/`TBB_LIB` so `RcppParallelLibs()` hands the system-TBB flags to dependents: build-env-images PR #12. Both must ship together. ## Verified In `build-env-alpine:3.24` with the new images: - RcppParallel builds against oneTBB 2022, no ABI errors, patch applies cleanly to the target clone. - `RcppParallelLibs()` returns `-L/usr/lib -Wl,-rpath,/usr/lib -ltbb -ltbbmalloc`. - rstan links (`-ltbb -ltbbmalloc`) and loads with no missing symbol -- `* DONE (rstan)`, trial build exit 0. Reviewed-on: #135	2026-07-20 09:56:21 +00:00
pat-s	b4b7d53a83	fix(local): make trial-build-patch.R detect non-throwing build failures (#134) ... `local/trial-build-patch.R` -- the single-package acceptance gate the proposer prints for humans to run (`PGPASS=... Rscript local/trial-build-patch.R <pkg>`) -- reported **`Trial build OK`** for rstan even though rstan.so failed to load (`symbol not found: tbb::detail::r1::observe`). Same false-green as the registry gate before #130: `build_binary_package()` catches failures internally and returns `"error"` rather than throwing, so a `tryCatch` that only treats a thrown exception as failure passes a broken build. Now inspects the return value (mirrors trial-build-registry.R). Found while verifying the rstan/oneTBB unblock. Reviewed-on: #134	2026-07-18 08:37:04 +00:00
pat-s	a30c6f9532	fix(crow): probe the auto-patch branch with git ls-remote to avoid a scary fatal (#133) ... Follow-up to #132. The no-op path already worked (the `else` branch runs and `exit 0`s), but using `git fetch` as the *existence check* printed `fatal: couldn't find remote ref auto/registry-patch-proposals` to stderr — which looks like a failure even though the step succeeds. Probe with `git ls-remote --exit-code --heads origin ${patch_branch}` instead: verified against the live repo it exits 0 for an existing branch and 2 for a missing one, **silently** (no `fatal`). Only fetch + checkout when the branch actually exists. Net: same behavior, clean log — a run with no pending auto-patch branch prints just `No ... branch; no pending auto-patch proposals to verify.` and exits 0. Reviewed-on: #133	2026-07-17 11:52:07 +00:00
pat-s	13fcad6dc4	fix(crow): make the trial-build gate no-op when the auto-patch branch is absent (#132) ... ## Problem The trial pipeline errored: ``` git clone -q --branch auto/registry-patch-proposals ... fatal: Remote branch auto/registry-patch-proposals not found in upstream origin ``` I deleted the stale `auto/registry-patch-proposals` branch during cleanup, and — now that cascade detection (#128) makes the proposer correctly find **no candidates** — the proposer exits before ever recreating it. So the branch is legitimately absent, and the gate's hard `git clone --branch` fails. ## Fix "Branch absent" = "no pending auto-patch proposals" = **nothing to verify**, which should be a clean no-op, not a failure. Clone `main`, then fetch + checkout the patch branch only if it exists; otherwise log and `exit 0`. A present branch is verified exactly as before (checkout its tip, diff registry vs `main`). This also means the gate now runs `main`'s `trial-build-registry.R` (with the #130 fix) rather than a possibly-stale copy on the branch. Reviewed-on: #132	2026-07-17 10:27:58 +00:00
pat-s	6c03f278ec	feat(local): aggregate blocked-on-dependency reporting by dependency (#131) ... ## Why With cascade detection (#128) live, the latest `auto-apply-patches` run did exactly the right thing — **proposed nothing** (`No auto-proposable candidates`) because every failure is a dependency cascade, and it surfaced the ~30 root-cause dependencies to fix. But the "Blocked on a dependency" list printed **one line per fingerprint group**, so the same dependency repeated (rstan ×4, lpsymphony ×4, salso ×2, BH ×2, GO.db ×2, RcppCWB ×2, …), burying the priority. ## What Aggregate blocked packages across all groups **by the dependency they wait on**: - Expose `blocked_map` (package → dependency) from `build_triage_report()`. - Add `blocked_by_dependency()` — dedupes dependents (a package in two groups counts once) and ranks dependencies by how many distinct dependents they block. - Proposer and tracker (log + issue) now print one line per dependency, sorted by impact. Replaces the per-group `blocked_summary`. ## Result (same data, aggregated) ``` Blocked on a dependency (3 dependencies block 6 dependents; fix the dependency, not each dependent): RcppParallel 3 dependent(s) rstan 2 dependent(s) sf 1 dependent(s) ``` So the real run becomes a crisp, ranked worklist: RcppParallel (894), sf (128), rstan (~96), Rfast (33), clarabel/DescTools (26), Rglpk (22), xgboost (18), … ## Verified New test covers cross-group aggregation, dedup (a dependent in two groups counted once), the example cap, and ranking. 112 tests pass; hooks pass. Reviewed-on: #131	2026-07-16 21:24:03 +00:00
pat-s	118a92889f	fix(local): make the trial-build gate detect non-throwing build failures (#130) ... ## Critical: the gate was false-green The latest run printed **`3/3 passed on alpine-324`** while all three builds actually **failed** (their `rstan` dependency won't compile). A false-green gate would let broken registry entries merge — worse than no gate. Root cause: `bincraft::build_binary_package()` catches build failures internally and **returns `"error"`** for the failed tag rather than throwing. The gate's `tryCatch` only treated a *thrown* exception as failure, so every non-throwing failure looked like a pass. Fix: inspect the return value. A tag passes only if the flattened result is non-empty and contains no `"error"` sentinel; a thrown error still counts as failure. Verified the verdict against `error`/`skipped`/`TRUE`/`list(success=TRUE)`/`NULL`/mixed inputs. With this, the current rstan-blocked entries will correctly show **0/3 (red)** — which is the right answer until rstan builds. Reviewed-on: #130	2026-07-16 08:24:03 +00:00
pat-s	f9d399fac0	feat(local): detect dependency-cascade failures generally, not just RcppParallel (#128) ... ## Why (from the #127 trial-build gate) The gate did its job: 0/3 passed, merge blocked. The log showed *why* -- BFpack, BayesERtools, GMLTM all fail while building their shared dependency **`rstan`**, not in their own code: ``` Failed to build source package rstan. .../StanHeaders/include/stan/math/prim/core/init_threadpool_tbb.hpp:9:10: fatal error: tbb/tbb_stddef.h: No such file or directory ``` So the per-package `-DTBB_INTERFACE_NEW` makevars entries the classifier proposed are useless for these packages -- they're blocked on `rstan` (which already has a registry entry). This is the **same dependency cascade** the RcppParallel `applies_to` guard catches, but `tbb-stddef-removed` is a generic signature with no such pin, so ~73 Stan packages kept getting proposed. ## What Generalise cascade detection beyond the RcppParallel special case: - `failing_dependency(error_text, package)` -- when the log names a **different** package as the one that failed to compile (`Failed to build source package X`, `compilation failed for package 'X'`, `dependency 'X' ... not available`), that package is the real cause. - `build_triage_report()` now blocks any package whose **every** failing build is such a cascade: reported as `blocked_on` that dependency, never proposed a bogus per-package entry. A package that fails in its **own** compilation is still proposed. - The `applies_to` (RcppParallel) and data-driven (rstan) cases are unified into one `blocked_packages` / `blocked_on` model; the report, proposer, and `blocked_summary` count the actually-blocked packages, and the blocked note shows even when a group also has genuine proposals. ## Effect Next auto-apply run will stop proposing the rstan-blocked Stan packages (and any future dependency cascade) and surface them as "blocked on rstan" instead. Fixing `rstan` once clears the whole cluster. ## Verification - New tests: `failing_dependency` (cascade vs own-compile vs none), and an end-to-end split where BFpack/GMLTM (blocked on rstan) are not proposed while an own-compile package still is. - Full suite: 112 tests pass; all pre-commit hooks pass. Refs #120, #127. (Separate follow-ups: fixing rstan's build itself, and quieting the gate's metadata-DB retry storm -- both root-caused to bincraft.) Reviewed-on: #128	2026-07-16 08:23:54 +00:00
pat-s	875b086122	ci(crow): enable verbose patched builds in the trial-build gate (#129) ... Companion to bincraft codefloe #65. Sets `BINCRAFT_VERBOSE_PATCH_BUILD=TRUE` in the trial-build gate so a failed isolated patched build prints the real compiler error instead of `System command 'R' failed`. Harmless on bincraft versions without the flag (unknown env var is ignored). Takes effect once bincraft #65 is released and the build-env images pick it up. Reviewed-on: #129	2026-07-16 08:23:44 +00:00
pat-s	ff9f5f5177	fix(local): shell-quote git args in the auto-patch push and trial-build gate (#126) ... ## Problem The first real `auto-apply-patches` run classified, limited to the top-10, and validated the candidate registry cleanly, then died at the push step: ``` Validating candidate registry: Patch registry OK (13 entries). sh: syntax error: unexpected "(" Error in git("commit", "-m", ...): git commit -m feat(patches): auto-propose 10 registry entries ... failed ``` Root cause: R's `system2()` with captured output (`stdout=TRUE`) runs the command through `/bin/sh`, and the arguments were passed **unquoted**. The commit message `feat(patches): ...` contains `()`, which the shell tried to interpret. The same class of bug affects the `^{commit}` and `ref:path` git refs in the trial-build gate. ## Fix `shQuote()` every git argument: - `propose-patches.R` -- the `git()` helper used by `--open-pr` (commit, push, checkout). - `trial-build-registry.R` -- the base-ref reads (`rev-parse ... ^{commit}`, `ls-tree`, `show ref:path`). ## Verification - Reproduced against a real git repo: the unquoted call fails with status 2 (the same `unexpected "("`); the `shQuote`d call commits successfully, and `rev-parse HEAD^{commit}` resolves. - 105 tests pass; all pre-commit hooks pass. Everything else in that run was correct: 842,659 failing builds classified, RcppParallel's 895 dependents correctly reported as blocked (not proposed), top-10 tbb-stddef candidates selected, 69 deferred, registry validated. Only the shell quoting was broken. Reviewed-on: #126	2026-07-15 15:43:02 +00:00
pat-s	ed1485c542	ci: remove evaluate conditions	2026-07-15 16:51:15 +02:00
pat-s	1bca6004fd	chore(local): reuse FORGEJO_TOKEN for the auto-patch push (#125) ... Follow-up to #124 (merged): the auto-patch pipeline required a separate write-scoped `REPO_RW_TOKEN` to push the branch. Reuse the existing `FORGEJO_TOKEN` instead. - `propose-patches.R --open-pr` now pushes `auto/registry-patch-proposals` over HTTPS with `FORGEJO_TOKEN` (the same token used for the PR API); the read-only `origin` clone URL can't push, so it builds an authenticated URL explicitly. - Drop `REPO_RW_TOKEN` from `.crow/auto-apply-patches.yaml` and the docs. No new secret needed: the pipeline's secrets are now `PGPASS`, `REPO_RO_TOKEN`, and `FORGEJO_TOKEN` (all existing). `FORGEJO_TOKEN` must have repository write scope for the push to succeed. 105 tests pass; all pre-commit hooks pass. Reviewed-on: #125	2026-07-15 14:50:19 +00:00
pat-s	4bca17e4ac	feat(local): auto-apply registry patches with a build-env trial-build gate (#124) ... Closes the classifier loop (issue #115, step 3): turns the auto-proposable candidates into an actual PR, gated by a real trial build **in our own build-env images**. Chosen model (from the design discussion): **autonomous PR**, **PR-first with a CI trial-build gate**, **bounded top-N batch** per run. ## Creating the patch PR - `propose-patches.R` gains: - `--limit N` -- act on the top-N candidates by failure volume; the rest defer to the next run (logged). - `--open-pr` -- write the entries onto the reused `auto/registry-patch-proposals` branch, push (with `REPO_RW_TOKEN`), and open/update **one** PR via the Forgejo API (so re-runs update the same PR instead of piling up). - `.crow/auto-apply-patches.yaml` -- a single job that runs `--open-pr --limit` on a cron/manual trigger. Needs `FORGEJO_TOKEN` + a write-scoped `REPO_RW_TOKEN`. Novel source diffs and unknown signatures are still never proposed; nothing merges. ## The merge gate (our build-env images) - `.crow/trial-build-registry.yaml` -- matrixed over the real `OS/IMG` build-env matrix (alpine:3.24, redhat:8/9/10, ubuntu:jammy/noble/**resolute** for ubuntu-2604). Each platform runs `local/trial-build-registry.R`, which diffs the branch registry against `main` and trial-builds **only the entries the branch adds** that apply to that platform, inside `reg.devxy.io/rpkgs/build-env-*`. Green only if every new entry builds; a platform with no new entries is a fast no-op. Nothing is uploaded/archived/recorded. - The base-registry read **fails loud** if it can't read `registry.json` at `main`, rather than silently treating the base as empty and trial-building the whole registry. ## Notes / follow-up - The repo uses **no `pull_request` triggers**, so the gate runs manually or on a cron against the branch (`--var patch_branch=...`). Wiring it to fire automatically on the PR needs `event: pull_request` enabled on the Forgejo webhook -- a one-line addition once that's on. - Two new crons to register in the crow UI: `auto-apply-patches` and `trial-build-registry`. New secret needed: `REPO_RW_TOKEN` (write scope) for the push. ## Verification - New pure helpers `entry_applies_to_os()` / `new_registry_packages()` covered by tests (platform codename/family/wildcard matching; added-vs-unchanged entry detection; per-platform filtering). - `--limit` smoke (stubbed DB): top-2 by volume proposed, 3 deferred, candidate registry validates. - Full suite: 105 tests pass; all pre-commit hooks pass (air, prettier, markdownlint, yamllint, validate-patches). Reviewed-on: #124	2026-07-15 08:28:15 +00:00
pat-s	21a2fe9e6c	feat(local): report unclassified and dependency-blocked failures for discovery (#122) ... ## Why Issue #120 (the auto-proposed-patches issue) only lists **auto-proposable** fixes -- currently just the TBB signatures. So a reasonable read of it was "TBB is our only build failure", when in fact three whole categories are simply not shown there: - **Unclassified failures** -- anything that doesn't match a seeded signature is routed to human triage and never appears (we've only seeded TBB and libuv signatures). - **Dependency-blocked failures** -- the ~800 RcppParallel dependents (post #121) are still failing; they only show as a log line. - Human-only signatures (libuv). These blind spots are exactly where the *next* signatures should come from, so they deserve the same visibility as the proposals. ## What Extend the feedback-loop tracker to surface the classifier's blind spots: - **`unclassified_summary()`** -- groups every unknown-signature failure by normalised fingerprint, ranked by build count, capped with an explicit `dropped_groups` count (no silent truncation), each with example packages + platforms. These are the candidates for new `build_signatures()` rules. - **`blocked_summary()`** -- lists each dependency (e.g. RcppParallel) and how many dependents wait on it. - `proposal-tracking.R` prints both sections, and a new **`--open-issue`** mode posts/updates a *"Unclassified build failures (needs signatures) (#115)"* Forgejo issue. - The weekly crow pipeline now runs the tracker with `--open-issue`, so it maintains a second tracking issue alongside the proposals one. Read-only on the DB; the only writes are the two issues. ## Verification - New tests cover `unclassified_summary` (ranking + both caps) and `blocked_summary`. - Tracker smoke with a stubbed DB (proposable + blocked + unclassified mix) prints the hit rate, `Blocked on a dependency: RcppParallel: 2 dependent(s)`, and `Unclassified failures ... [2 builds | 2 pkgs] ld: undefined reference ...`. - Full suite: 95 tests pass; all pre-commit hooks pass (air, prettier, markdownlint, yamllint, validate-patches). Reviewed-on: #122	2026-07-15 07:46:29 +00:00
pat-s	9bc4973159	fix(local): stop proposing per-dependent patches for dependency-cascade failures (#121) ... ## Problem The scheduled `--open-issue` run (issue #120) posted **800+** proposed entries, all `rcppparallel-bundled-tbb` on `ubuntu-2604`, each pre-filling `RcppParallel/disable-tbb.patch` for an unrelated package (`ACEsimFit`, `AovBay`, `AdaptGauss`, ...). Root cause: those packages fail on ubuntu-2604 only because their **RcppParallel dependency** does not build there. bincraft records RcppParallel's TBB error text against each dependent, so every one of them matches the `rcppparallel-bundled-tbb` signature. Proposing a RcppParallel source patch for `ACEsimFit` is not just noise -- it is **broken**: the diff targets RcppParallel's `Makevars.in`, so it cannot apply to a dependent's source. ## Fix Add an optional `applies_to` field that pins a package-specific fix to the package it targets: - A signature whose fix is a curated per-package source patch (`rcppparallel-bundled-tbb` -> `RcppParallel`) is only ever proposed **for that package**. - Other packages matching the signature are downstream failures, reported as **"blocked on `<dependency>`"** and never proposed a bogus entry. Fix the dependency once and the whole cascade clears. Generic levers (the `tbb-stddef-removed` makevars fix) are unaffected -- they carry no `applies_to`, so they still apply to any matching package. ## Changes - gate proposal generation in `build_triage_report()` on `applies_to`; add a `blocked_on` field to group records - surface blocked groups in both `failing-builds-report.R` and `propose-patches.R` instead of the misleading "already registered" note - document `applies_to` in the signature-table header ## Verification - New tests: RcppParallel dependents are blocked (no `proposed_entries`, `blocked_on = "RcppParallel"`), while RcppParallel *itself* is still proposed when it is the failing package. - Reproduced the avalanche end-to-end with a stubbed DB: 5 dependents now report "Blocked on RcppParallel: 5 package(s) fail because RcppParallel does not build" and **zero** candidates are emitted; a genuine `tbb-stddef` proposal in the same run is unaffected. - Full suite: 84 tests pass; all pre-commit hooks pass. Refs #120. Note: the underlying build problem (RcppParallel failing on ubuntu-2604 despite its registry entry) is real and separate -- this PR stops the classifier from spamming broken per-dependent proposals about it. Reviewed-on: #121	2026-07-14 18:40:41 +00:00
pat-s	f1dd661213	fix(local): route multi-signature packages to human triage in the proposer (#119) ... ## Problem A live `propose-patches.R --open-issue` run aborted with: ``` Patch registry validation FAILED: - ambiguous duplicate entries: hmmTMB|ubuntu-2604|*, imt|ubuntu-2604|*, refundBayes|ubuntu-2604|* Error: Candidate registry failed validation; not writing or proposing. ``` Root cause: a package whose failing builds match **more than one** auto-proposable signature (e.g. some logs hit `tbb-stddef-removed`, others `rcppparallel-bundled-tbb`) landed in two separate signature groups, so the proposer emitted two candidate entries with the same `package|platforms|versions` key. The validator's ambiguous-duplicate check then failed the whole candidate set, aborting the run — so even the unambiguous candidates (e.g. `vacalibration`) never got proposed. ## Fix Add a pure, tested `dedupe_candidates()` that: - keeps a package's candidate only when it maps to a **single** signature (collapsing an accidental repeat under the same signature), and - routes a package that maps to **multiple** signatures (conflicting fix tiers -- makevars vs source patch) to **human triage** instead of guessing between them. The proposer prints the skipped ambiguous packages and proceeds with the clean candidates, so one ambiguous package no longer blocks the rest. This matches the issue #115 guardrail: don't auto-pick when the fix is ambiguous. ## Verification - New unit tests in `test-proposal-tracking-lib.R` cover the single-signature, multi-signature (conflict), duplicate-under-one-signature, and empty cases. - Reproduced the original failure end-to-end with a stubbed DB (`hmmTMB` ambiguous + `vacalibration` clean): `hmmTMB` is now listed under "Ambiguous -> human triage", `vacalibration` is proposed, and the candidate registry validates (`Patch registry OK`). - Full suite: 78 tests pass; all pre-commit hooks pass. Reviewed-on: #119	2026-07-14 17:11:54 +00:00
pat-s	77508f8fc5	ci: clean weekly-patch-proposals trigger	2026-07-14 18:17:28 +02:00
pat-s	af481f5129	ci(crow): add weekly patch-proposal + feedback-loop pipeline (#118) ... Adds the scheduled pipeline for the failure-triage proposer/tracker landed in #117 (issue #115, steps 3 + 4). ## `.crow/weekly-patch-proposals.yaml` A single (non-matrix) job -- the classifier groups over all of `single_builds`, so unlike the per-platform `weekly-audit-missing` it runs once globally. Weekly it: 1. `propose-patches.R --open-issue` -- posts/updates a Forgejo tracking issue with the auto-proposable registry entries (human reviews, trial-builds, opens the PR; nothing merges). 2. `proposal-tracking.R` -- logs the feedback-loop metrics (signature hit rate, proposed-vs-merged, retirement candidates). Low blast radius: clones read-only (`REPO_RO_TOKEN`); the only write is the tracking issue via `FORGEJO_TOKEN`. It builds nothing (no B2 keys, no bincraft). Triggers: `cron: weekly-patch-proposals` (register the cron in the crow UI) or manual with `task=weekly-patch-proposals`, matching the `archive-missed-packages` convention. Documented under "Scheduled run" in `local/patches/README.md`. `yamllint` and the other pre-commit hooks pass. Reviewed-on: #118	2026-07-14 15:17:16 +00:00
pat-s	1c297c01bf	feat(local): auto-propose registry patches and track the feedback loop (#117) ... Implements steps 3 + 4 of #115, building on the classifier merged in #116. Now that bincraft **v4.4.3** applies registry `patch`/`makevars`/`configure_args` to the *target* package build (previously deps-only), a trial patched build is a meaningful acceptance gate, so the "propose" half is viable. ## Step 3 — propose, do not apply - **`local/propose-patches.R`** — for each classified, safe fix affecting a package with no current registry entry, emits a pre-filled `registry.json` entry and validates the candidate set against a *temporary* merged registry (the real one is never touched unless asked). - default: print candidates + validation, **take no action** - `--write`: append entries to `registry.json` + the proposals ledger (you commit + open the PR) - `--open-issue`: post/update a Forgejo tracking issue (reuses the weekly-audit `httr2` + `FORGEJO_TOKEN` pattern) - **`local/trial-build-patch.R`** — isolated bincraft build of one package with the registry applied (no upload/archive/metadata; `patchhash` keeps it out of the real cache). Exit 0/1, so it gates a CI step or manual pre-merge check. The human gate stays: nothing merges. Acceptance = `validate-patches.R` passes (checked automatically) **and** the trial build succeeds. Novel source diffs and unknown signatures are never proposed (they carry `auto = FALSE`). ## Step 4 — feedback loop - **`local/proposal-tracking.R`** (read-only) — signature hit rate (builds/pkgs/addressed/open per signature), proposed-vs-merged (a proposal counts merged once its package is in the registry), and retirement candidates (registry entries whose package no longer fails, i.e. likely fixed upstream). - **`local/proposal-tracking-lib.R`** — the pure metric/ledger helpers. ## Supporting changes - Refactored the classify helpers to expose a pure `build_triage_report()` + a list-returning entry builder; `failing-builds-report.R` now renders from the shared function (no behaviour change). - `validate-patches.R` gains optional `PATCH_DIR`/`REGISTRY_FILE` overrides (backward-compatible) so a candidate registry can be validated in isolation. - Documented the propose/trial-build/tracking workflow in `local/patches/README.md`. ## Verification - 71 unit tests pass (incl. new `test-proposal-tracking-lib.R`) under the Dockerized R 4.5.3 build env. - All pre-commit hooks pass (`air-format`, `validate-patches`, prettier, etc.). - Smoke-tested all three entrypoints end-to-end with a stubbed DB: dry-run, `--write` (produces a registry that passes the canonical validator + a valid ledger, then reverted), and the tracker. Closes #115 Reviewed-on: #117	2026-07-14 15:03:24 +00:00
pat-s	f11ba7172f	feat(local): classify failing binary builds and pre-fill registry suggestions (#116) ... Implements steps 1 + 2 of #115: turn recorded build failures into triaged patch suggestions instead of hand-scraping Crow logs. ## What this adds A **read-only** reporting pipeline over the `single_builds` metadata table. It never writes to the DB or the registry. - `local/failing-builds-classify.R` — pure, DB-free helpers: - `normalise_error()` strips temp paths, version numbers, hex addresses, and the package name so the same root cause collapses to one fingerprint. - `fingerprint_error()` extracts the salient error line and normalises it. - `classify_error()` matches against a seed signature set; unmatched errors are never guessed at. - `propose_registry_entry()` renders a schema-valid `registry.json` entry. - `local/failing-builds-report.R` — entrypoint: queries `single_builds WHERE error_occurred = TRUE AND removed = FALSE`, groups by root cause (signature when classified, fingerprint otherwise), classifies each group, and prints a triaged report. Flags: `--platform`, `--arch`, `--min`, `--json`; `PLATFORM`/`ARCH` env fallbacks. - `local/tests/test-failing-builds-classify.R` — unit tests for the helpers. - `local/patches/README.md` — documents the workflow. ## Seed signatures Each rule carries a fix tier, confidence, and an auto/human-only flag: | Signature | Fix | Disposition | | --- | --- | --- | | `tbb/tbb_stddef.h: No such file` | makevars `-DTBB_INTERFACE_NEW` | auto-proposable | | RcppParallel bundled TBB (musl / new g++) | curated `disable-tbb.patch` | auto-proposable | | system `libuv.so` link leak | force vendored/static lib | **human triage** (novel source diff) | | unmatched | none | **human triage** | ## Guardrails honored - No autonomous novel source diffs: only known env/makevars levers and already-curated package patches are auto-proposable; anything needing a brand-new diff, and any unknown signature, is routed to human triage. - No DB or registry writes; no change to the public `src/contrib` index. - Reuses `single_builds.error_text`; no new failure-capture pipeline. ## Verification - All helper unit tests pass under the Dockerized R 4.5.3 build env. - Pre-commit hooks pass (`air-format`, `validate-patches`, prettier, etc.). - Smoke-tested the full report path with a stubbed DB; generated proposals pass the real `local/validate-patches.R`. Steps 3 (auto-open PRs) and 4 (feedback loop) are intentionally deferred, per the issue's suggestion to validate the signature set first. Closes #115 Reviewed-on: #116	2026-07-14 14:45:25 +00:00
pat-s	04059f20e3	fix(patches): add rstan entry to skip removed tbb_stddef.h (#114) ... ## Motivation `rstan` fails to compile when it is pulled in as a transitive dependency (e.g. of `ctsem`), aborting the dependency install: ``` /mnt/cache/R-pkgs/StanHeaders/include/stan/math/prim/core/init_threadpool_tbb.hpp:9:10: fatal error: tbb/tbb_stddef.h: No such file or directory 9 | #include <tbb/tbb_stddef.h> ``` StanHeaders' `init_threadpool_tbb.hpp` unconditionally includes the legacy `<tbb/tbb_stddef.h>` to detect the TBB version, but that header was removed in oneTBB 2021+, which is the TBB that StanHeaders now bundles. ## Fix Add an `rstan` entry to `local/patches/registry.json` that pre-defines `TBB_INTERFACE_NEW` via `CPPFLAGS`. The header's own guard skips the removed include when that macro is already defined and selects the modern `tbb/global_control.h` + `tbb/task_arena.h` path that the bundled oneTBB actually provides: ```cpp #ifndef TBB_INTERFACE_NEW #include <tbb/tbb_stddef.h> // removed in oneTBB 2021+ #if TBB_VERSION_MAJOR >= 2020 #define TBB_INTERFACE_NEW #endif #endif #ifdef TBB_INTERFACE_NEW #include <tbb/global_control.h> // taken when we pre-define the macro #include <tbb/task_arena.h> ``` This is a makevars-tier override (no source diff), applied through bincraft's `R_MAKEVARS_USER`, so `CPPFLAGS` stays additive and does not clobber rstan's own `PKG_CPPFLAGS`. `-I/usr/local/include` is kept because the override replaces the default `CPPFLAGS`. ## Files changed - `local/patches/registry.json`: add the `rstan` entry (`versions: "*"`, `platforms: ["*"]`, `makevars.CPPFLAGS = "-DTBB_INTERFACE_NEW -I/usr/local/include"`). ## Behaviour change When bincraft resolves `rstan` (direct or transitive) it builds a patched binary with `TBB_INTERFACE_NEW` defined, so `rstan` (and dependents such as `ctsem`) compile against the bundled oneTBB. The scope is `["*"]` because the failure is bound to StanHeaders' bundled TBB version, not the OS or toolchain. Reviewed-on: #114	2026-07-14 13:52:17 +00:00
pat-s	a4b274f281	fix(ci): bound pkgcache _metadata growth to stop macmini disk-fill (#110) ... ## Problem The arm64 `build-all` pipeline fills the macmini (gaia) host disk despite an 8h prune. Root cause is not images or job volumes: it is the persistent dep-cache volume, specifically `pkgcache/R/pkgcache/_metadata`, which grew to ~165 GB. `{pkgcache}` mints a new content hash for the "patched" binaries repo on every PACKAGES change, so each per-package build writes a fresh ~70 MB `pkgs-<hash>.rds` (+ `patched-<hash>/`) that is never evicted (2407 snapshots observed). When the disk hits 100% OrbStack stops and the on-host prune can no longer connect to the daemon, so it never self-heals. ## Change (Workstream A of the disk-fill fix) - Add `trim_pkgcache_metadata()` to `local/r-minor-helpers.R`: keeps the newest `keep` (default 20) `patched-*`/`pkgs-*.rds` entries under `_metadata`, deleting only entries older than `min_age_secs` (default 600s) so it never races the up-to-4 concurrent split-jobs sharing the volume. Preserves `pkg/` downloads and the stable CRAN/BioC/INLA repo dirs. No-op when `R_PKG_CACHE_DIR` is empty (amd64) or `_metadata` is absent (first run). - Call it every 25 packages inside the build loop in `local/build-all.R`. - Add a defensive start-of-run cleanup of `_metadata/patched-*` + `pkgs-*.rds` to the two workflows that mount the persistent volume (`build-all-versions.yaml`, `build-all-versions-install-deps.yaml`). Only these paths are touched; `process-updates.yaml`/`weekly-rebuild-missing.yaml` (no persistent volume) are unchanged. Follow-ups (separate workstreams): on-host self-healing prune watcher + OrbStack disk cap (ansible), and Prometheus/Grafana alerting (k8s-talos). Upstream: bincraft patched-repo hash churn is the true source fix. New unit tests (6) for the helper; full suite 24/24 green. Reviewed-on: #110	2026-07-13 09:31:29 +00:00
pat-s	ffc2319558	fix(build-all): exclude previously-errored versions in prefilter (#113) ... ## Motivation Build jobs were cycling through hundreds of packages that were only ever printed as `Skipping … due to previous build error recorded in metadata DB`, wasting wall-clock on per-package preparation before dropping each one. ## Cause The prefilter query in `local/build-all.R` selected only successfully-built versions (`error_occurred = FALSE`) into `built`, so line 112 removed only those from the chunk. Every previously-errored version stayed in the work list and was walked one-by-one, each hitting the internal skip in `build_binary_package()`. This also explains the misleading `Skipped 0 already-built package versions` line for alphabetical chunks whose leading packages only have error records. ## Changes - `local/build-all.R`: drop the `AND error_occurred = FALSE` clause so `built` holds every version already attempted (built or errored) for this platform/arch; the existing filter then removes all of them up front. - Rename the log line to `already-attempted` so the reported count reflects successes and errors. - Update the surrounding comment to explain why errored versions are excluded. ## Behaviour change Previously-errored versions are now dropped before the build loop instead of being iterated and individually skipped. No package that would otherwise build is affected, `build_binary_package()` already skipped these internally. Retrying errored versions is out of scope and would need a separate opt-in flag on both the prefilter and the in-loop skip. Reviewed-on: #113	2026-07-13 09:28:45 +00:00
pat-s	09536cddd6	feat(build): raise BuildKit cache-mount budget for remote builders (#112) ... BuildKit's default GC caps the ephemeral cache tier — `RUN --mount=type=cache` mounts, local build context, git checkouts — at a hardcoded **512 MB** (shown as `488.3 MiB` in `buildx inspect`). Across our 7-distro build matrix that fills instantly and forces re-downloads of system + R packages on every rebuild. ## Changes - **`docker/buildkitd.toml`** (new) — GC config passed via `--config` when creating the `docker-container` builders `artemis` (amd64) and `gaia` (arm64): - cache-mount tier: **512 MB → 8 GB**, retained 7 days - total cache bounded at **40 GB** with **20 GB min-free** - **`justfile`** — document the `--config docker/buildkitd.toml` flag on the builder-create commands so a recreate does not silently revert to the 512 MB default. ## Notes - Limits are absolute (not `%`) because the two hosts differ ~6× in free space (Hetzner ~42 GiB free vs Mac mini ~279 GiB). The 20 GB min-free is the safety valve on the disk-tight Hetzner host; the old default wanted 64 GiB free, which does not exist there. - Both live builders were already recreated with this config and verified running. Reviewed-on: #112	2026-07-08 08:22:03 +00:00
pat-s	36d3bc8604	fix(patches): force fs to build vendored static libuv (#111) ... ## Problem The `fs` 2.1.0 binary links **system libuv** (`readelf -d fs.so` shows `NEEDED libuv.so.1`). fs's `configure` prefers system libuv whenever `pkg-config` resolves it, and our build images ship `libuv-devel` (installed as a pak build-time system requirement), so the resulting binary is dynamically linked against `libuv.so.1`. That binary fails to load on any consumer machine without runtime libuv: ```text unable to load shared object '.../fs/libs/fs.so': libuv.so.1: cannot open shared object file: No such file or directory ``` `install.packages()`/renv do **not** install `SystemRequirements` (only `pak` does, and only inside the build container), so most consumers hit this. Older fs 1.6.x always vendored libuv, so only the 2.x binaries regressed. Reproduced in a clean `reg.devxy.io/r/r-alma:4.5-9`. ## Fix Add `local/patches/fs/force-vendored-libuv.patch`, registered for all platforms. It short-circuits `configure` to `cp -f src/Makevars.vendor src/Makevars; exit 0` before the pkg-config detection, forcing the bundled static libuv build (`tools/libuv-v1.52.0.tar.gz`, built via cmake). An env/pkg-config override (`PKG_CONFIG_LIBDIR`) was tried first but the rebuilt binary still linked `libuv.so.1` (the registry `env` tier does not reach fs's configure step), so a source patch is used instead. ## Verification Built end-to-end inside the real `build-env-redhat:9` image (system libuv present): - patch fires (`Building static libuv (bincraft: forced vendored)`), - cmake compiles the vendored libuv, - resulting `fs.so` has **no `libuv.so.1`** in `NEEDED` (only libR, libstdc++, libm, libgcc_s, libc). `Rscript local/validate-patches.R` passes (2 entries). cmake confirmed present in the build-env images. ## Follow-up (not in this PR) - Rebuild `fs 2.1.0` on every affected platform (rhel8/9/10, ubuntu jammy/noble, alpine 3.22/3.23; amd64 + arm64) and purge the CDN binary paths. - CDN delivery gap: `purge_cdn_cache.sh` only purges `PACKAGES*`, never package binaries, so rebuilt binaries stay masked until their `.tar.gz` path is purged. Reviewed-on: #111	2026-07-07 18:12:36 +00:00
pat-s	1e06576084	docs: add CLAUDE.md with B2 storage gotchas and build pipeline conventions for agents	2026-07-02 08:43:37 +02:00