Author	SHA1	Message	Date
pat-s	e9782bb529	fix(ci): install RPostgres for metadata updates	2026-08-28 08:48:58 +00:00
pat-s	50495f5c3b	Revert "fix(ci): split oversized weekly rebuild matrix" ... This reverts commit b75fd2f1c4.	2026-08-14 07:01:56 +00:00
pat-s	b75fd2f1c4	fix(ci): split oversized weekly rebuild matrix	2026-08-14 06:56:41 +00:00
pat-s	132d1d2d3c	docs(ci): clarify parallel manual matrix runs	2026-08-14 06:44:10 +00:00
pat-s	9bded261ee	fix(cdn): restore Alliance pull-zone hostname (#166) ... ## Motivation Applying #165 recreated the Alliance SwissPass pull zone without its custom hostname because the hostname association was not represented in OpenTofu. The recreated zone also received a new numeric ID, making the weekly purge configuration stale. ## Changes - Manage `cran.allianceswisspass.devxy.io` as a pull-zone hostname with TLS and forced HTTPS. - Resolve the Alliance pull-zone ID from its hostname before purging instead of persisting a replaceable numeric ID. - Install `jq` in the purge step for the Bunny API lookup. ## Verification - Targeted `prek` hooks pass. - `tofu validate` passes. - `crow lint .crow/` passes. - `just edge-test` passes all 14 routing steps. - `bash -n scripts/purge_cdn_zone.sh` passes. ## Deployment Run `tofu apply` to restore the Alliance hostname on the recreated pull zone. Reviewed-on: #166	2026-08-13 14:13:04 +00:00
pat-s	a1c1f5e78f	fix(cdn): align repository routing across pull zones (#165) ... ## Motivation `cran.rpkgs.com` and `cran.allianceswisspass.devxy.io` serve the same B2 repository through separate Bunny pull zones, but only the first zone was managed and purged after weekly reindexing. This allowed the Alliance endpoint to retain stale repository metadata and left locked `renv` restores unable to retrieve versions whose binary archive object was absent. ## Changes - Adopt the Alliance SwissPass pull zone `3265648` into OpenTofu and configure it with the shared B2 origin and middleware script. - Purge both Bunny pull zones after the weekly rebuild reindex. - Preserve the requested public hostname in middleware redirects. - Redirect missing archived binaries to the corresponding CRAN source package, checking whether the version is archived or still current. - Cover the existing archived-binary passthrough behavior in the edge routing matrix. ## Verification - `prek run -a` - `just edge-test` - `crow lint .crow/` - `tofu validate` - `bash -n scripts/purge_cdn_zone.sh` ## Deployment Run `tofu apply` to adopt pull zone `3265648`, publish the middleware release, and align both pull zones. After the apply, rerun the Alliance SwissPass CI restore that requested `cli 3.6.5` and `AzureStor 3.7.1`. Reviewed-on: #165	2026-08-13 14:08:10 +00:00
pat-s	aa4c95457f	fix(rebuild): harden split workflow setup (#164) ... ## Motivation Weekly rebuild shards can all hit a transient CRAN DNS/index outage at once, and the dependent CDN purge always fails because it tries to clone over the checkout preserved from the re-index step. ## Changes - Retry `uvr add` resolution up to four times with bounded backoff. - Reuse the existing Crow workspace checkout in the CDN purge step. - Remove the purge step's unused Git package and repository token. ## Validation - `crow lint .crow/` - `shellcheck local/uvr-install.sh scripts/purge_cdn_zone.sh` - `git diff --check` Reviewed-on: #164	2026-08-13 13:38:28 +00:00
pat-s	4b7dc28cc8	feat(rebuild): shard the weekly rebuild and make each shard resumable (#163) ... ## Problem `weekly-rebuild-missing` runs one job per `<os>-<arch>` and walks that slot's list serially in a single `R -q -e` argument. That was cheap while every source fallback was skipped as "already built". Since bincraft #105/#106/#107 and #159 the gate works, and the lists are large: 8 917 source-served records on `amd64/alpine324`, 15 023 on `amd64/resolute`. Pipeline 10910 (`weekly_rebuild_missing:alpine-324-amd64`) ran for two days, reached `[8692/23885] cholera`, and was killed there. Two failures follow from that shape: - **No parallelism.** The work is embarrassingly parallel across packages; one job does all of it. - **No resumability and no clean stopping point.** The loop ends only by exhausting the list, so the only way to stop it is a kill. A restart re-walks from the first entry, paying a CRAN version resolution and an S3 `HEAD` per package before reaching new work. And a kill matches neither `success` nor `failure`, so the `Purge CDN cache` step never ran: the ~4 600 binaries 10910 did publish stayed hidden behind stale edge copies. ## What this changes **Three shards per slot.** Each of the 18 `OS`/`ARCH` rows gains `SPLIT_INTO`/`SPLIT_INDEX`, mirroring `build-all-versions.yaml`. Cron and manual routing are unchanged: both filters already match on `${OS}-${ARCH}`, so they now match all three shards of a slot. **`local/rebuild-missing.R`** replaces the ~1 500-character inline one-liner. The slice is interleaved rather than contiguous, because the list is alphabetical and cost clusters by name (`Rcpp*`, `Bioc*`, `rstan*`). **Resume by re-deriving state from the bucket.** One `s3_dir_info()` listing gives ETags for the slot; a package is outstanding iff its object's ETag equals CRAN's published `MD5sum`, i.e. it is still byte-identical to CRAN's source. That is `check_s3_root_package()` evaluated in bulk. No progress file, no volume, no DB cursor, and correct when a sibling shard or a `process-updates` run completes something concurrently. It reads ETags rather than the index's `Built` field the way `packages-to-build.R` does, because the index is no longer rewritten until the dependent pipeline runs and so cannot reflect the current run's progress. Unknown always means "already a binary", never "rebuild it": a multipart ETag, an unreadable CRAN index or an empty listing can never mass-schedule work. **A 20 h wall-clock budget** per shard. It exits 0, so the re-index and purge always fire and the remainder is picked up next run with no bookkeeping. **`.crow/weekly-rebuild-reindex.yaml`** takes over re-indexing and the purge, with `depends_on: [weekly-rebuild-missing]` and `runs_on: [success, failure]`. Three shards writing one slot's `PACKAGES` concurrently would race: `update_PACKAGES()` lists the live bucket, so an early lister that uploads last publishes an index missing its siblings' work. ## Verification `crow lint .crow/` passes on all 11 pipelines. `prek run` passes. 19 assertions in `local/tests/test-rebuild-missing.R`, 0 failures, covering the partition (disjoint, covering, deterministic, short lists, out-of-range index) and the outstanding filter (source ETag kept, binary ETag dropped, absent object kept, multipart and missing-from-CRAN treated as built). One of those tests caught a real bug before it shipped: an empty ETag table indexed to zero length rather than to `NA`, which recycled the result away and reported "nothing to build" — the dangerous direction. Fixed with an explicit `lookup()`. The filter run against the live `amd64/alpine324` index, using its `MD5sum` column as the ETag (established to match the objects): ``` index packages: 24343 outstanding (filter): 8950 no Built stamp: 8917 filter vs no-Built agreement: 8917 of 8917 outstanding but stamped Built: 33 (version drift vs CRAN) shard sizes: 2984/2983/2983 (sum 8950, unique 8950) ``` It reproduces the source-served set exactly. The extra 33 are packages whose slot version differs from CRAN's current one, so no object exists at the CRAN version key: correctly outstanding. ## Notes for review - The 20 h budget is a chosen default, exposed as `REBUILD_BUDGET_HOURS` in the pipeline. - `depends_on` is file-level, not row-level, so on a full cron run no slot is re-indexed until the slowest of all 54 jobs finishes. The budget bounds that at roughly a day. - An explicit cancel still skips the re-index. Recovery is to trigger `weekly-rebuild-reindex` on its own. - The purge runs on every re-index row rather than one designated slot: a cron fires only its own slot's row, so gating on a named slot would leave every other slot unpurged. - Out of scope: `build-all-versions` still cannot rebuild source fallbacks, because `local/build-all.R:113-122` drops every version with any `single_builds` row, which is precisely the source-fallback set. Design: `specs/2026-08-12-shard-weekly-rebuild-design.md` Reviewed-on: #163	2026-08-12 08:30:29 +00:00
pat-s	01b8ab43df	fix(rebuild): re-index and purge the CDN after a rebuild (#160) ... ## Problem The rebuild now works — `AATtools 0.0.3` was detected as a source fallback, built, and published: ``` ℹ `upload_single_binary()`: Replacing the CRAN source published for AATtools 0.0.3 … with the binary. ✔ Successfully uploaded package AATtools with tag 0.0.3. ``` But clients still get the source, and will for about a year: ``` $ curl -sI .../src/contrib/AATtools_0.0.3.tar.gz etag: "ea8127d953ca6a2f118ea49441772af6" # CRAN's source MD5 cdn-cache: HIT cdn-cachedat: 08/09/2026 16:43:32 # predates the 18:12 upload ``` Two causes, both specific to a rebuild: 1. **The slot is never re-indexed.** `weekly-rebuild-missing` has no `upload_package_index` step, so the index keeps the old MD5 and — for anything that had been served from source — no `Built` stamp. This one self-heals at the next `process-updates` run. 2. **The tarball URL is never purged.** A normal update publishes new packages at *new* URLs, so `purge_cdn_cache.sh` only needs the five index files. A rebuild replaces an object *in place*, and the zone caches tarballs for `cache_expiration_time = 31919000` (~370 days). This does not self-heal. Nothing about a stale package looks wrong from the outside, which is what makes it worth fixing rather than documenting. ## What this changes **Re-index at the end of a rebuild**, flat and per-minor, mirroring the tail of `process-updates`. The codename is detected from the image's `/etc/os-release` (as `local/packages-to-build.R` already does) rather than adding `OS_ID` to all 18 matrix rows. **Purge the zone afterwards**, via a new `scripts/purge_cdn_zone.sh`. One call to `POST /pullzone/{id}/purgeCache` covers every replaced object, and all three hostnames — `cran.devxy.io`, `cran.allianceswisspass.devxy.io`, `cran.rpkgs.com` — share pull zone `3857050`, confirmed from the `cdn-pullzone` response header. Purging per URL was the alternative and is worse here: ~13.5k rate-limited calls per arch, where a single missed call leaves a package silently stale. The cost of the zone purge is a cold cache for everything else, which is why it stays out of the daily update path — `purge_cdn_cache.sh` is untouched. The purge runs on failure too (`when: status: [success, failure]`): a rebuild that died part-way still replaced objects, and those are exactly the ones a stale edge keeps hiding. ## Verification `crow lint .crow/` reports all ten configs valid; `bash -n` on the new script passes; prek hooks pass. Not yet exercised against Bunny — it needs `BUNNYNET_API_KEY`, which is a CI secret. The failure mode is explicit rather than silent: any status other than 200/204 prints the response body and exits non-zero. Reviewed-on: #160	2026-08-10 06:30:35 +00:00
pat-s	f8e31af75b	fix(ci): make every manual gate default to a value that matches nothing (#158) ... ## Problem A manual `crow pipeline create` instantiates **every** file in `.crow/`, and a declared variable default is applied even when the run never passed that variable. A gate is therefore only a gate if its default matches nothing. #155 fixed the three pipelines that had no manual gate at all. It missed that a *permissive default* leaves a pipeline just as exposed. Demonstrated the expensive way: creating a pipeline with only ``` --var weekly_audit_missing=alpine-324-amd64 ``` also started `build-all-versions` — because its gate `target_arch` defaults to `amd64`, which matches its own amd64 matrix rows — and `process-updates` across every row, because that gate defaults to `all`. The run was killed before any `Upload package indexes` step produced output and both alpine324 indices were verified unchanged, but `build-all-versions` uploads binaries and rewrites indexes, so the next one might not be caught in time. Before: | pipeline | gate | default | fired on an unrelated manual run | | --- | --- | --- | --- | | `build-all-versions` | `target_arch` | `amd64` | amd64 rows — builds and uploads | | `build-all-versions-install-deps` | `target_arch` | `amd64` | amd64 rows | | `weekly-rebuild-missing` | `weekly_rebuild_missing` | `all` | every row | | `weekly-audit-missing` | `weekly_audit_missing` | `all` | every row | | `process-updates` | `process_cran_updates` | `all` | every row | | `repair-built-stamp` | `repair_built_stamp` | `arm64` | arm64 rows | `archive-missed-packages` was the one that behaved, because its gate variable is never declared and so matches nothing. That is the property this restores everywhere. ## What this changes Each of the six gets a `none` option on its gate variable and defaults to it, so a manual run has to name its target explicitly. The reason is recorded next to the default, where someone would go to change it. `none` is used rather than dropping the default so the expression always has a defined value to compare, instead of relying on undefined-variable semantics. Cron triggers are untouched — they match on the `cron:` name, not the variable. ## Verification `crow lint .crow/` reports all ten configs valid. Auditing every pipeline that accepts a manual event: ``` archive-missed-packages.yaml: gate=task default=<none> auto-apply-patches.yaml: gate=auto_apply_patches default='false' build-all-versions-install-deps.yaml gate=target_arch default=none build-all-versions.yaml: gate=target_arch default=none process-updates.yaml: gate=process_cran_updates default=none repair-built-stamp.yaml: gate=repair_built_stamp default=none trial-build-registry.yaml: gate=trial_build_registry default='false' weekly-audit-missing.yaml: gate=weekly_audit_missing default=none weekly-patch-proposals.yaml: gate=weekly_patch_proposals default='false' weekly-rebuild-missing.yaml: gate=weekly_rebuild_missing default=none ``` Every gate now defaults to something that matches no matrix row. Reviewed-on: #158	2026-08-09 15:31:14 +00:00
pat-s	1e843523a0	fix(ci): gate the three ungated pipelines on their own variable (#155) ... ## Problem A manual `crow pipeline create` instantiates **every** pipeline in `.crow/`, and each one decides for itself whether to run. Three had nothing to decide with — their only manual condition was a bare `event: manual`: - `auto-apply-patches` — pushes to `auto/registry-patch-proposals` and opens/updates a PR - `weekly-patch-proposals` — posts and edits two Forgejo issues - `trial-build-registry` — starts a build per matrix row, on both arches So they fired on *any* manual trigger in this repo, whatever it was for. That is how they came to run alongside a manual `process-updates` run for `alpine-324-arm64` (#10723), which is also why that pipeline is marked failure. `repair-built-stamp.yaml` already documents the rule this breaks: > The gate variable is `repair_built_stamp`, not `target_arch` … Every pipeline here gates on a variable named after itself for exactly that reason. ## What this changes Each of the three gets a gate variable named after the pipeline, `evaluate`d on the manual event, defaulting to off: ```yaml variables: auto_apply_patches: description: 'Run the auto-patch proposer. Also gates this pipeline.' options: ['true', 'false'] default: 'false' when: - event: manual evaluate: 'auto_apply_patches == "true"' - event: cron cron: auto-apply-patches ``` Cron triggers are untouched, so the scheduled runs behave exactly as before. The run-manually comments in all three headers were also stale: they documented `--var task=<name>` with `woodpecker-cli`, and no pipeline evaluates a `task` variable. They now show the real invocation. ## Note on the sibling pipelines The already-gated pipelines use `default: all` (e.g. `weekly_rebuild_missing`). If Crow applies a declared default to a variable that an API-created pipeline never passed, those would match on an unrelated manual run too — `weekly-rebuild-missing` would be an expensive way to find out. I could not settle that from #10723 because its step logs have since expired, so I left them alone rather than guess. The three fixed here default to `'false'`, which is safe under either semantics. ## Verification `crow lint .crow/` passes. Auditing every pipeline that accepts a manual event now reports a gate on all ten: ``` build-all-versions-install-deps.yaml: gated auto-apply-patches.yaml: gated weekly-patch-proposals.yaml: gated weekly-audit-missing.yaml: gated repair-built-stamp.yaml: gated archive-missed-packages.yaml: gated weekly-rebuild-missing.yaml: gated trial-build-registry.yaml: gated build-all-versions.yaml: gated process-updates.yaml: gated ``` Reviewed-on: #155	2026-08-09 10:02:05 +00:00
pat-s	77ce1d212d	fix(ci): gate the Built-stamp repair on its own variable (#151) ... ## Problem `repair-built-stamp.yaml` gated on `target_arch`: ```yaml when: - event: manual evaluate: 'target_arch == "${ARCH}"' ``` That is the same variable `build-all-versions.yaml` and `build-all-versions-install-deps.yaml` gate on. A manual run passing `target_arch=arm64` to reach the repair therefore matched all three, so triggering a dry-run repair also queued a full arm64 rebuild. I hit this triggering the alpine 3.22 dry run (pipeline 10706), which I killed. `crow pipeline ps` renders empty states on this version, so I could not confirm from the CLI whether the rebuild workflows started before the kill or only sat queued behind the running cron jobs; no output was attributable to them. ## Change Gate on `repair_built_stamp` instead. Every other pipeline here already gates on a variable named after itself (`process_cran_updates`, `weekly_audit_missing`, `weekly_rebuild_missing`), and `target_arch` was the odd one out being shared by two. The header now records the collision and the exact invocation, so the next pipeline added here does not repeat it. ## Verification `crow lint` passes; `prek` clean. Grep of every trigger condition in `.crow/` confirms `repair_built_stamp` is unique and that no other pipeline gates on `OS`, `OS_VERSION`, `R_VERSION` or `dry_run` alone: ``` archive-missed-packages.yaml: task == "archive-missed-packages" build-all-versions.yaml: target_arch == "${ARCH}" build-all-versions-install-deps.yaml: target_arch == "${ARCH}" weekly-audit-missing.yaml: weekly_audit_missing == ... repair-built-stamp.yaml: repair_built_stamp == "${ARCH}" process-updates.yaml: process_cran_updates == ... weekly-rebuild-missing.yaml: weekly_rebuild_missing == ... ``` ## Note for whoever runs it `crow pipeline create` against this instance returned HTTP 504 while still creating the pipeline. Retrying on that error duplicates the run: I created four before noticing. Verify with `pipeline ls` rather than trusting the exit status. Reviewed-on: #151	2026-08-07 09:41:53 +00:00
pat-s	0da29ab3f6	fix(index): add a slot repair for a broken Built stamp and retire alpine 3.21 (#150) ... ## Problem `arm64/alpine321` and `arm64/alpine322` advertise a broken stamp: ``` arm64/alpine321 :: 22930 Built: R 4.4.0; NA; 2026-07-31 13:35:52 UTC; unix arm64/alpine322 :: 24696 Built: R 4.5.0; NA; 2026-07-31 13:51:54 UTC; unix ``` The per-minor sub-slots (`contrib/4.4`, `4.5`, `4.6`) are affected too. All 18 other slots are correct. uvr matches the stamp's platform triple plus R minor to pick binary over source, so nothing matches `NA` and both slots silently serve as source-only, which is exactly the regression bincraft#85 added the stamp to prevent. `install.packages()` is unaffected, since it reads `Built:` from each tarball's own `DESCRIPTION`. The tarballs are fine (`arm64/alpine322/.../dress.graph_0.8.3.tar.gz` carries `aarch64-unknown-linux-musl`), and so is the R that built them (`r-4.5.0_1_aarch64.apk` ships `R_PLATFORM='aarch64-unknown-linux-musl'`). Only the index is wrong. bincraft#96 stops a stamp like this being written again, but it cannot repair what is already there. ## Why not just re-run the index update `upload_package_index()` reuses the slot's remote `PACKAGES.db`, and cranlike's `update_db()` only reparses files whose md5 changed, so entries already in the database keep the stamp they were written with. Dropping `PACKAGES.db` to force a full reparse does work, and it is what bincraft#85's rollout note suggested, but for an S3 repo cranlike reads each package's metadata from the CRAN *source* mirror on GitHub. A 25k-entry slot is then 25k requests to raw.githubusercontent.com, with a real risk of being rate-limited part-way through and leaving the slot half-written. Only the `Built` column is wrong, so this corrects it in place instead: patch the column in `PACKAGES.db`, put the database back, and let `upload_package_index()` re-emit `PACKAGES*` from it. `update_db()` always rewrites the index files even when nothing was reparsed, so no tarball is re-read and nothing is fetched from GitHub. ## Change - `local/repair-built-stamp.R` — repairs the generic slot plus every per-minor sub-slot. Dry-run by default; `--apply` writes. The replacement comes from `bincraft::built_stamp()` under the R running the script, so it is exactly what a healthy run would have written, and bincraft#96's guard makes a broken build image fail rather than write a second bad stamp. - `.crow/repair-built-stamp.yaml` — manual pipeline, routed by `target_arch` to the matching agent group and platform image, with `dry_run` defaulting to `true`. - `.crow/archive-missed-packages.yaml` — drop the two `alpine321` matrix entries. Alpine 3.21 is EOL: the website advertises only 3.23/3.24 and `process-updates.yaml` already dropped it, so that slot is retired rather than repaired. ## Verification `crow lint .crow/` passes on all nine pipelines. `air format` and `jarl check` are clean; the script parses, and the `/opt/R` minor-version derivation was checked against `4.4.3 / 4.5.3 / 4.6.0 / current` → `4.4 4.5 4.6`. The repair itself is unrun by design — it needs B2 credentials and an arm64 agent. ## Rollout 1. Cut a bincraft release so `local/install-bincraft.R` picks up #96 (it resolves the latest `vX.Y.Z` tag, and #96 is only on `main`). 2. Run this pipeline with `target_arch=arm64`, `OS=alpine`, `OS_VERSION=3.22`, `R_VERSION=4.5.3`, `dry_run=true` and check the reported counts. 3. Re-run with `dry_run=false`. 4. Confirm: `curl -sS https://cran.devxy.io/arm64/alpine322/latest/src/contrib/PACKAGES | grep '^Built:' | sort | uniq -c` `arm64/alpine321` is deliberately left alone. Reviewed-on: #150	2026-08-07 09:15:04 +00:00
pat-s	ee15c50f53	refactor: migrate package installation from pak to uvr (#147) ... `bincraft` dropped pak in favour of uvr (5.0.x, "Dependencies and their system requirements are now installed with `uvr` instead of pak during `build_binary_package()`"), so the pipelines, helper scripts and images in this repo move with it. ## Approach uvr is project-scoped in a way pak is not: `uvr add` refuses to run outside a project and always installs into `.uvr/library/`, and only `uvr sync` honours `--library`. So there is no one-line `pak::pak(...)` equivalent. `local/uvr-install.sh` encapsulates the dance — bootstrap a pinned uvr, mint a throwaway project under `TMPDIR`, `uvr add --no-install`, then `uvr sync --library <target>`. Keeping the project outside the checkout also keeps `uvr init`'s `.Rprofile` from hijacking `.libPaths()` for every other R call in the pipeline. This matches what bincraft itself does (`uvr sync --install-system-deps --library <lib>`), and bincraft requires `uvr` on `PATH`, which the bootstrap provides: every pipeline that calls `bincraft::` runs `install-bincraft.R` (and therefore the bootstrap) first. ## Changes | File | Change | | --- | --- | | `local/uvr-install.sh` | **New.** The single replacement for `pak::pak(...)`. Bootstraps uvr `v0.4.4`, resolves the R interpreter from `UVR_R_BIN`/`R_VERSION`/`PATH`, pins the manifest to that R's exact version, and syncs into `UVR_TARGET_LIB`/`R_LIBS_USER`. | | `local/install-bincraft.R` | Installs `forgejo::codefloe.com/rpkgs/bincraft@<tag>` instead of a `git::` URL; keeps the `git ls-remote` tag resolution. Exports `UVR_R_BIN`/`UVR_TARGET_LIB` from `R.home()`/`.libPaths()[1]` so the per-R-minor passes target their own R and library. | | `.crow/auto-apply-patches.yaml`, `.crow/weekly-patch-proposals.yaml`, `.crow/weekly-audit-missing.yaml`, `.crow/weekly-rebuild-missing.yaml`, `.crow/build-all-versions-install-deps.yaml` | `pak::pak(...)` → `UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh ...`. The explicit `UVR_R_BIN` matters in `build-all-versions-install-deps.yaml`, which has no `R_VERSION` in its step environment. | | `.crow/build-all-versions.yaml`, `.crow/process-updates.yaml`, `.crow/weekly-rebuild-missing.yaml` | `R_PKG_CACHE_DIR` → `UVR_CACHE_DIR` + `UVR_PACKAGES_DIR` on the same `/mnt/cache` volume, preserving the amd64-off/arm64-on split. Drops the `rm -rf .../pkgcache/_metadata/...` cleanup. | | `local/r-minor-helpers.R`, `local/build-all.R`, `local/tests/test-trim-pkgcache.R` | Removes `trim_pkgcache_metadata()`, its every-25-packages call and its tests. uvr's cache does not mint a fresh ~70 MB snapshot per `PACKAGES` change. | | `.crow/build-all-versions-install-deps.yaml` | Drops `pak::sysreqs_db_update()`; uvr resolves sysreqs from its vendored `r-system-requirements` rules via `--install-system-deps`. | | `docker/Containerfile-shiny-app` | Bootstraps uvr and drives both dependency installs through one uvr project with `UVR_LIBRARY` pointed at the image's R library. | | `docker/build-one.Dockerfile` | Ships `uvr-install.sh` at `/work/local/` so `install-bincraft.R` finds it. | | `docker/reprex/alpine.sh` | Replaces `pak::local_install_deps()` with DESCRIPTION parsing + `uvr add`. | | `local/test-package-loading.R` | Installs via the helper instead of `pak::pkg_install()`. | | `README.md` | Documents uvr for sysreq inference, archived-version installs and cache clearing. | | `renovate.json` | Tracks the `UVR_PIN` in `uvr-install.sh` via `github-releases`. | ## Behaviour notes - **`weekly-audit-missing` still takes bincraft from the default branch**, not the latest release tag, matching what the `git::` pak call did. Called out in a comment rather than silently changed. - **The uvr pin is repo-wide.** bincraft resolves `uvr` from `PATH` and pins no version of its own, so `UVR_PIN` in `uvr-install.sh` governs the whole pipeline. - **Persistent caches now also benefit bincraft**, which reads `UVR_CACHE_DIR`/`UVR_PACKAGES_DIR` from the inherited pipeline environment. - **`uvr sync` will not prune the shared library.** Pruning is disabled whenever `--library` is passed (`do_prune = prune && library_override.is_none()`), so `/mnt/cache/R-pkgs` keeps bincraft and its dependencies. The wipe-on-ABI-mismatch path is *not* similarly guarded, which is why the helper pins the manifest to the active R's exact version. - **`plans/` and `specs/` are untouched** — they are dated records of decisions made in June 2026 and describe bincraft's then-pak-based internals; rewriting them would misstate history. ## Verification `shellcheck`, all pre-commit hooks (on this commit's file range) and the `local/tests/` suite (100 assertions) pass. Not yet exercised in CI: the build-env images do not ship `uvr`, so the per-step `curl install.sh` bootstrap is untested against a real image. Worth a manual `build-all-versions-install-deps` run before merging. Reviewed-on: #147	2026-07-31 12:19:26 +00:00
pat-s	736564a4ff	fix: prevent renovate bumps for fixed matrix assignments	2026-07-24 11:18:28 +02:00
pat-s	61e1c0188d	chore: adjust weekly audit runs	2026-07-20 19:09:50 +02:00
pat-s	a30c6f9532	fix(crow): probe the auto-patch branch with git ls-remote to avoid a scary fatal (#133) ... Follow-up to #132. The no-op path already worked (the `else` branch runs and `exit 0`s), but using `git fetch` as the *existence check* printed `fatal: couldn't find remote ref auto/registry-patch-proposals` to stderr — which looks like a failure even though the step succeeds. Probe with `git ls-remote --exit-code --heads origin ${patch_branch}` instead: verified against the live repo it exits 0 for an existing branch and 2 for a missing one, **silently** (no `fatal`). Only fetch + checkout when the branch actually exists. Net: same behavior, clean log — a run with no pending auto-patch branch prints just `No ... branch; no pending auto-patch proposals to verify.` and exits 0. Reviewed-on: #133	2026-07-17 11:52:07 +00:00
pat-s	13fcad6dc4	fix(crow): make the trial-build gate no-op when the auto-patch branch is absent (#132) ... ## Problem The trial pipeline errored: ``` git clone -q --branch auto/registry-patch-proposals ... fatal: Remote branch auto/registry-patch-proposals not found in upstream origin ``` I deleted the stale `auto/registry-patch-proposals` branch during cleanup, and — now that cascade detection (#128) makes the proposer correctly find **no candidates** — the proposer exits before ever recreating it. So the branch is legitimately absent, and the gate's hard `git clone --branch` fails. ## Fix "Branch absent" = "no pending auto-patch proposals" = **nothing to verify**, which should be a clean no-op, not a failure. Clone `main`, then fetch + checkout the patch branch only if it exists; otherwise log and `exit 0`. A present branch is verified exactly as before (checkout its tip, diff registry vs `main`). This also means the gate now runs `main`'s `trial-build-registry.R` (with the #130 fix) rather than a possibly-stale copy on the branch. Reviewed-on: #132	2026-07-17 10:27:58 +00:00
pat-s	875b086122	ci(crow): enable verbose patched builds in the trial-build gate (#129) ... Companion to bincraft codefloe #65. Sets `BINCRAFT_VERBOSE_PATCH_BUILD=TRUE` in the trial-build gate so a failed isolated patched build prints the real compiler error instead of `System command 'R' failed`. Harmless on bincraft versions without the flag (unknown env var is ignored). Takes effect once bincraft #65 is released and the build-env images pick it up. Reviewed-on: #129	2026-07-16 08:23:44 +00:00
pat-s	ed1485c542	ci: remove evaluate conditions	2026-07-15 16:51:15 +02:00
pat-s	1bca6004fd	chore(local): reuse FORGEJO_TOKEN for the auto-patch push (#125) ... Follow-up to #124 (merged): the auto-patch pipeline required a separate write-scoped `REPO_RW_TOKEN` to push the branch. Reuse the existing `FORGEJO_TOKEN` instead. - `propose-patches.R --open-pr` now pushes `auto/registry-patch-proposals` over HTTPS with `FORGEJO_TOKEN` (the same token used for the PR API); the read-only `origin` clone URL can't push, so it builds an authenticated URL explicitly. - Drop `REPO_RW_TOKEN` from `.crow/auto-apply-patches.yaml` and the docs. No new secret needed: the pipeline's secrets are now `PGPASS`, `REPO_RO_TOKEN`, and `FORGEJO_TOKEN` (all existing). `FORGEJO_TOKEN` must have repository write scope for the push to succeed. 105 tests pass; all pre-commit hooks pass. Reviewed-on: #125	2026-07-15 14:50:19 +00:00
pat-s	4bca17e4ac	feat(local): auto-apply registry patches with a build-env trial-build gate (#124) ... Closes the classifier loop (issue #115, step 3): turns the auto-proposable candidates into an actual PR, gated by a real trial build **in our own build-env images**. Chosen model (from the design discussion): **autonomous PR**, **PR-first with a CI trial-build gate**, **bounded top-N batch** per run. ## Creating the patch PR - `propose-patches.R` gains: - `--limit N` -- act on the top-N candidates by failure volume; the rest defer to the next run (logged). - `--open-pr` -- write the entries onto the reused `auto/registry-patch-proposals` branch, push (with `REPO_RW_TOKEN`), and open/update **one** PR via the Forgejo API (so re-runs update the same PR instead of piling up). - `.crow/auto-apply-patches.yaml` -- a single job that runs `--open-pr --limit` on a cron/manual trigger. Needs `FORGEJO_TOKEN` + a write-scoped `REPO_RW_TOKEN`. Novel source diffs and unknown signatures are still never proposed; nothing merges. ## The merge gate (our build-env images) - `.crow/trial-build-registry.yaml` -- matrixed over the real `OS/IMG` build-env matrix (alpine:3.24, redhat:8/9/10, ubuntu:jammy/noble/**resolute** for ubuntu-2604). Each platform runs `local/trial-build-registry.R`, which diffs the branch registry against `main` and trial-builds **only the entries the branch adds** that apply to that platform, inside `reg.devxy.io/rpkgs/build-env-*`. Green only if every new entry builds; a platform with no new entries is a fast no-op. Nothing is uploaded/archived/recorded. - The base-registry read **fails loud** if it can't read `registry.json` at `main`, rather than silently treating the base as empty and trial-building the whole registry. ## Notes / follow-up - The repo uses **no `pull_request` triggers**, so the gate runs manually or on a cron against the branch (`--var patch_branch=...`). Wiring it to fire automatically on the PR needs `event: pull_request` enabled on the Forgejo webhook -- a one-line addition once that's on. - Two new crons to register in the crow UI: `auto-apply-patches` and `trial-build-registry`. New secret needed: `REPO_RW_TOKEN` (write scope) for the push. ## Verification - New pure helpers `entry_applies_to_os()` / `new_registry_packages()` covered by tests (platform codename/family/wildcard matching; added-vs-unchanged entry detection; per-platform filtering). - `--limit` smoke (stubbed DB): top-2 by volume proposed, 3 deferred, candidate registry validates. - Full suite: 105 tests pass; all pre-commit hooks pass (air, prettier, markdownlint, yamllint, validate-patches). Reviewed-on: #124	2026-07-15 08:28:15 +00:00
pat-s	21a2fe9e6c	feat(local): report unclassified and dependency-blocked failures for discovery (#122) ... ## Why Issue #120 (the auto-proposed-patches issue) only lists **auto-proposable** fixes -- currently just the TBB signatures. So a reasonable read of it was "TBB is our only build failure", when in fact three whole categories are simply not shown there: - **Unclassified failures** -- anything that doesn't match a seeded signature is routed to human triage and never appears (we've only seeded TBB and libuv signatures). - **Dependency-blocked failures** -- the ~800 RcppParallel dependents (post #121) are still failing; they only show as a log line. - Human-only signatures (libuv). These blind spots are exactly where the *next* signatures should come from, so they deserve the same visibility as the proposals. ## What Extend the feedback-loop tracker to surface the classifier's blind spots: - **`unclassified_summary()`** -- groups every unknown-signature failure by normalised fingerprint, ranked by build count, capped with an explicit `dropped_groups` count (no silent truncation), each with example packages + platforms. These are the candidates for new `build_signatures()` rules. - **`blocked_summary()`** -- lists each dependency (e.g. RcppParallel) and how many dependents wait on it. - `proposal-tracking.R` prints both sections, and a new **`--open-issue`** mode posts/updates a *"Unclassified build failures (needs signatures) (#115)"* Forgejo issue. - The weekly crow pipeline now runs the tracker with `--open-issue`, so it maintains a second tracking issue alongside the proposals one. Read-only on the DB; the only writes are the two issues. ## Verification - New tests cover `unclassified_summary` (ranking + both caps) and `blocked_summary`. - Tracker smoke with a stubbed DB (proposable + blocked + unclassified mix) prints the hit rate, `Blocked on a dependency: RcppParallel: 2 dependent(s)`, and `Unclassified failures ... [2 builds | 2 pkgs] ld: undefined reference ...`. - Full suite: 95 tests pass; all pre-commit hooks pass (air, prettier, markdownlint, yamllint, validate-patches). Reviewed-on: #122	2026-07-15 07:46:29 +00:00
pat-s	77508f8fc5	ci: clean weekly-patch-proposals trigger	2026-07-14 18:17:28 +02:00
pat-s	af481f5129	ci(crow): add weekly patch-proposal + feedback-loop pipeline (#118) ... Adds the scheduled pipeline for the failure-triage proposer/tracker landed in #117 (issue #115, steps 3 + 4). ## `.crow/weekly-patch-proposals.yaml` A single (non-matrix) job -- the classifier groups over all of `single_builds`, so unlike the per-platform `weekly-audit-missing` it runs once globally. Weekly it: 1. `propose-patches.R --open-issue` -- posts/updates a Forgejo tracking issue with the auto-proposable registry entries (human reviews, trial-builds, opens the PR; nothing merges). 2. `proposal-tracking.R` -- logs the feedback-loop metrics (signature hit rate, proposed-vs-merged, retirement candidates). Low blast radius: clones read-only (`REPO_RO_TOKEN`); the only write is the tracking issue via `FORGEJO_TOKEN`. It builds nothing (no B2 keys, no bincraft). Triggers: `cron: weekly-patch-proposals` (register the cron in the crow UI) or manual with `task=weekly-patch-proposals`, matching the `archive-missed-packages` convention. Documented under "Scheduled run" in `local/patches/README.md`. `yamllint` and the other pre-commit hooks pass. Reviewed-on: #118	2026-07-14 15:17:16 +00:00
pat-s	a4b274f281	fix(ci): bound pkgcache _metadata growth to stop macmini disk-fill (#110) ... ## Problem The arm64 `build-all` pipeline fills the macmini (gaia) host disk despite an 8h prune. Root cause is not images or job volumes: it is the persistent dep-cache volume, specifically `pkgcache/R/pkgcache/_metadata`, which grew to ~165 GB. `{pkgcache}` mints a new content hash for the "patched" binaries repo on every PACKAGES change, so each per-package build writes a fresh ~70 MB `pkgs-<hash>.rds` (+ `patched-<hash>/`) that is never evicted (2407 snapshots observed). When the disk hits 100% OrbStack stops and the on-host prune can no longer connect to the daemon, so it never self-heals. ## Change (Workstream A of the disk-fill fix) - Add `trim_pkgcache_metadata()` to `local/r-minor-helpers.R`: keeps the newest `keep` (default 20) `patched-*`/`pkgs-*.rds` entries under `_metadata`, deleting only entries older than `min_age_secs` (default 600s) so it never races the up-to-4 concurrent split-jobs sharing the volume. Preserves `pkg/` downloads and the stable CRAN/BioC/INLA repo dirs. No-op when `R_PKG_CACHE_DIR` is empty (amd64) or `_metadata` is absent (first run). - Call it every 25 packages inside the build loop in `local/build-all.R`. - Add a defensive start-of-run cleanup of `_metadata/patched-*` + `pkgs-*.rds` to the two workflows that mount the persistent volume (`build-all-versions.yaml`, `build-all-versions-install-deps.yaml`). Only these paths are touched; `process-updates.yaml`/`weekly-rebuild-missing.yaml` (no persistent volume) are unchanged. Follow-ups (separate workstreams): on-host self-healing prune watcher + OrbStack disk cap (ansible), and Prometheus/Grafana alerting (k8s-talos). Upstream: bincraft patched-repo hash churn is the true source fix. New unit tests (6) for the helper; full suite 24/24 green. Reviewed-on: #110	2026-07-13 09:31:29 +00:00
pat-s	1ad7a6bfe9	chore: resolve latest bincraft release dynamically (no hardcoded pins) (#107) ... ## Summary Stop hardcoding the bincraft version. Every `.crow` workflow and the build-one image pinned `@vX.Y.Z` (and a `packageVersion() != "X.Y.Z"` guard), so each bincraft release meant editing the version in ~8 places — and it was easy to miss one (the Dockerfile lagged at v4.2.1; v4.4.1 shipped without the empty-env fix because of exactly this churn). ## Change New `local/install-bincraft.R` resolves the **latest release tag dynamically**: - `git ls-remote --tags` on the public repo (no token), - keep `vX.Y.Z` tags, pick the highest version (filtered/sorted in R for portability, not via git `--sort`/refspec which behaved inconsistently under `system2()`), - `pak::pak("git::…@<latest>")` — idempotent on the git ref, so re-runs keep the package unless a newer tag exists. All call sites now invoke the helper instead of a pinned version: - `.crow/build-all-versions.yaml` (primary + per-minor pass) - `.crow/build-all-versions-install-deps.yaml` - `.crow/process-updates.yaml` (primary + per-minor pass) - `.crow/weekly-rebuild-missing.yaml` - `.crow/archive-missed-packages.yaml` - `docker/build-one.Dockerfile` (ships the helper into the image; `ensure_bincraft` sources it) ## Effect Tag a new bincraft release → the next CI run / `just rebuild` picks it up automatically. No more pin edits, and no more "forgot to bump the Dockerfile" drift. ## Verified - Resolver returns the current latest tag (`v4.4.2`) via `git ls-remote` + R-side version sort. - All five workflow YAMLs parse; helper R parses; air/editorconfig clean. Note: this tracks the latest **tag**, so cutting a release is still the deliberate gate — CI won't pick up un-tagged main. Reviewed-on: #107	2026-07-01 08:10:02 +00:00
pat-s	b4ec6291a9	chore: bump bincraft to 4.4.2	2026-06-30 16:09:44 +02:00
pat-s	d496029a79	chore: bump bincraft to 4.4.1	2026-06-30 15:51:07 +02:00
pat-s	b93d1ef6bd	chore: silence otelsdk warnings across build paths (#105) ... ## Summary Silences the recurring `OpenTelemetry error: there is no package called 'otelsdk'` warnings during builds. ## Root cause The `build-env-*` images configure an OTel exporter (traces/logs/metrics), but `otelsdk` (the R OTel SDK backend) isn't installed. pak's `otel` instrumentation therefore tries to load `otelsdk` on every run and logs the error, falling back to a no-op. `OTEL_SDK_DISABLED=true` (already set in `build-one.Dockerfile`) does **not** help — the R `otel` package ignores it and gates purely on `OTEL_R_<SIGNAL>_EXPORTER` (then the standard `OTEL_<SIGNAL>_EXPORTER`). When that resolves to a real exporter (`otlp`/`http`/…) with no SDK present, you get the error. ## Fix Set `OTEL_R_TRACES_EXPORTER`, `OTEL_R_LOGS_EXPORTER`, and `OTEL_R_METRICS_EXPORTER` to `none` so the R otel providers are clean no-ops: - `docker/build-one.Dockerfile` — exported alongside the existing OTel var. - `.crow/process-updates.yaml`, `weekly-rebuild-missing.yaml`, `build-all-versions.yaml`, `build-all-versions-install-deps.yaml` — added to each step's `environment` block. Using the R-specific variables (not the standard `OTEL_*_EXPORTER`) keeps OTel intact for any non-R tooling in the images. Reviewed-on: #105	2026-06-30 12:41:32 +00:00
pat-s	00db47398d	chore: 4.4.0 instead of 4.3.1	2026-06-30 14:07:22 +02:00
pat-s	55a18fd87d	feat: patch registry + wiring for per-package patching (#103) ... ## Summary Adds the curated **patch registry** and wiring that drives bincraft's new package-patching mechanism (see bincraft PR `feat/package-patching`). Lets specific packages be patched (env/configure/Makevars overrides or source diffs) before pak installs them — including as transitive dependencies — so compiler-/OS-specific failures like RcppParallel's bundled TBB stop cascading. ## What's included - `local/patches/registry.json` — initial entry: RcppParallel with `RCPP_PARALLEL_USE_TBB=0` for alpine / ubuntu-2604, plus `local/patches/README.md` schema docs. - `local/validate-patches.R` — validates schema, referenced patch files, and ambiguous overlaps; clean failure + exit 1 (no stacktrace). - `.pre-commit-config.yaml` — a `validate-patches` hook (re-runs when the registry or the validator changes). - `local/build-one.R` / `local/build-all.R` — pass `patches = "local/patches"` to `bincraft::build_binary_package()`. - `specs/2026-06-30-package-patching-design.md` and `plans/2026-06-30-package-patching-implementation.md`. ## ⚠️ Merge ordering (blocker) This PR adds a `patches = ...` argument to `build_binary_package()` calls. The `.crow/*.yaml` workflows currently pin bincraft **v4.2.3**, which does not accept that argument — CI will error with `unused argument (patches=...)` until: 1. bincraft **v4.3.0** is released (PR `feat/package-patching`), and 2. the pin is bumped in `.crow/build-all-versions-install-deps.yaml`, `.crow/build-all-versions.yaml`, and `.crow/process-updates.yaml`. The `.crow` pin bump will be added to this PR once bincraft v4.3.0 is tagged. Do not merge before then. Reviewed-on: #103	2026-06-30 08:50:19 +00:00
pat-s	a1d7064fb8	feat(ci): add process-updates rows for alpine-324 and ubuntu resolute ... Add matrix rows and manual dropdown options for alpine 3.24 (alpine-324, PROCESS_NEW=FALSE) and ubuntu resolute / 26.04 (ubuntu-2604, OS_ID resolute, R 4.5.3, PROCESS_NEW=TRUE).	2026-06-21 11:04:24 +02:00
pat-s	b46472b59e	fix(ci): pass Backblaze B2 credentials to process_unarchived_pkgs ... The function defaults to Hetzner S3 with HETZNER_S3_*_K3S env vars that are not set in these workflows, so paws failed with "No compatible credentials provided". Pass the B2 endpoint/region/bucket and B2_S3_* secrets explicitly, matching the other S3 calls. Also use set_codename(NULL) for the build-all-versions codename so it matches the S3 repo path.	2026-06-21 10:55:22 +02:00
pat-s	3420523849	chore: add all options for OS_VERSION	2026-06-18 13:40:13 +02:00
pat-s	83c761b736	feat(ci): OS/OS_VERSION manual dropdowns + restore lost crow fix (#97) ... ## Summary Two changes: 1. **`OS`/`OS_VERSION` manual-run dropdowns** — give these form variables explicit `options:` lists (like `target_arch` and `R_VERSION`), so the manual-run form shows dropdowns instead of free-text, in both `build-all-versions.yaml` and `build-all-versions-install-deps.yaml`. Crow form variables are independent (no cascading), so the operator still has to pick a coherent `OS` + `OS_VERSION` combination (e.g. `redhat` + `9`, not `alpine` + `jammy`). 2. **Repairs `main`** — the crow fix from PR #96 (`bc2f6f1`) was lost when that PR was squashed (only the first commit was captured). As a result `main` currently carries the `OS: ${OS}` env vars that break Crow parsing (`unable to parse variable name`) and the unfixed `build-all.R`. This PR re-applies that fix: drop the env additions and derive `platform`/`arch` inside `build-all.R` from the container (bincraft codename → platform mapping + `Sys.info()` arch). ## Notes - `OS_VERSION` options are quoted strings so tags like `8`/`9`/`10` aren't parsed as integers. - Validated: both YAMLs parse, `build-all.R` parses. Reviewed-on: #97	2026-06-18 11:39:11 +00:00
pat-s	0f4330c05c	fix(ci): make build-all self-sufficient on agents without install-deps (#96) ... ## Summary Fixes the recurring `build-all-*` failure on agents where `build-all-versions-install-deps` did **not** run. Both symptoms in the logs — `Parameter 3 does not have length 1` (repeated, in the metadata DB queries) and `argument is of length zero` (system-dependency install) — were the same bug: inside bincraft, `platform` was zero-length. `local/build-all.R` calls `build_binary_package()` without passing `platform`, so bincraft resolves it from the container codename. The primary build step never (re)installed/pinned bincraft and relied on whatever sat in the **per-agent** cache volume; `depends_on` only orders steps, it does not co-locate them on the same agent, so a job landing where install-deps never ran got a stale bincraft that left `platform` empty. ## Changes - **`build-all-versions.yaml`**: pin bincraft `@v4.2.3` in the primary build step (mirroring the R-minor pass and `process-updates.yaml`), so every agent uses a known-good bincraft regardless of where install-deps ran. - Align the R-minor pass `v4.2.2 → v4.2.3`. - Export `OS`/`OS_VERSION`/`ARCH` as runtime env vars — previously only available for `${...}` interpolation, so `build-all.R`'s already-built dedup query matched platform `"-"` and skipped nothing. - Fix the unarchive call: pass the codename via `bincraft::set_codename(NULL)` instead of the malformed `paste(OS, OS_VERSION)` (`"alpine 3.24"`), matching `archive-missed-packages.yaml`. - **`build-all-versions-install-deps.yaml`**: pin install-deps to `@v4.2.3` (was installing HEAD), so the precomputed snapshot and per-agent library stay consistent pipeline-wide. Reviewed-on: #96	2026-06-18 11:27:02 +00:00
pat-s	f9bc72d5ab	chore: bump bincraft to 4.2.3	2026-06-18 09:43:05 +02:00
pat-s	4c67703f52	refactor(ci): route workflows by agent group label instead of named agent (#94) ... ## Summary - Switch crow workflow placement from a per-agent label (`agent: ${AGENT}`) to a group label (`group: rpkgs-${ARCH}`), so jobs target the `rpkgs-amd64` / `rpkgs-arm64` agent pools instead of a single named agent (`artemis`/`gaia`). - Remove the now-unused `AGENT` matrix variable from all build/process/weekly workflows. - Update header comments to reflect group-based placement. Agents opt into a pool via `CROW_AGENT_LABELS="group=rpkgs-<arch>"`. The group name derives from each matrix row's `ARCH`, so adding or replacing agents no longer requires touching the workflow files. Reviewed-on: #94	2026-06-17 16:26:04 +00:00
pat-s	3875612237	fix(ci): restore xwfb-run/xvfb-run detection for rhel-10 process-updates ... The matrix consolidation hardcoded `xvfb-run`, which is absent on the rhel-10 image (it ships Xwayland's xwfb-run), causing `xvfb-run: command not found`. Re-add the per-platform launcher detection the old rhel-10 files used: prefer xwfb-run + weston, fall back to xvfb-run.	2026-06-17 12:47:04 +02:00
pat-s	76f677ba5b	fix(ci): quote matrix PROCESS_NEW so R receives uppercase FALSE/TRUE ... Unquoted `PROCESS_NEW: FALSE` is parsed as a YAML boolean and interpolated into the R command as lowercase `false`, causing `process_new = false` and `Error: object 'false' not found`. Quoting keeps the literal FALSE/TRUE that R expects.	2026-06-17 11:35:44 +02:00
pat-s	dc92bd6a2c	fix(ci): restore cron triggers via per-row cron name filter ... The matrix consolidation routed cron events through `evaluate: 'CI_PIPELINE_CRON == "..."'`, but CI_PIPELINE_CRON is not exposed to the when.evaluate constraint context, so no matrix row ever matched and the scheduled process-updates / weekly runs stopped firing. Switch back to the first-class `cron:` name filter (still parameterized per matrix row) which the server matches directly against the fired cron.	2026-06-17 11:03:12 +02:00
pat-s	b297f0fbc0	refactor(ci): consolidate build-all-versions workflows into 2 files (#93) ... ## Summary Consolidate the four `build-all-versions*` files into two, mirroring the matrix approach used for the process-updates / weekly families: | Before | After | |---|---| | `build-all-versions-amd64.yaml` + `build-all-versions-arm64.yaml` | `build-all-versions.yaml` | | `build-all-versions-install-deps-amd64.yaml` + `-arm64.yaml` | `build-all-versions-install-deps.yaml` | `ARCH` becomes a matrix axis carrying the per-arch differences: `agent` (artemis/gaia), `BACKEND` (kubernetes/docker), and `R_PKG_CACHE_DIR`. Build-all keeps its `SPLIT_INDEX` parallelism, so its matrix is ARCH x SPLIT = 8 rows; only the rows matching the selected arch run. **Invocation unchanged.** Routing uses the existing task selector `task == "build-all-${ARCH}"`, so `--var task=build-all-amd64 ...` still runs exactly the amd64 rows. `OS` / `OS_VERSION` / `R_VERSION` / `K8S_INSTANCE_TYPE` are still passed via `--var`. (You can drop `--var ARCH=...` now — arch comes from the matrix/task.) **No cron.** These pipelines are manual-only, so there are no scheduled runs to affect. ## Validation before merge Same matrix-interpolation-in-`when`/`labels` mechanism validated for #92, plus it now relies on per-row `agent`/`BACKEND` selecting the right execution backend (kubernetes for amd64, docker for arm64). Before merge, on this branch trigger a small `task=build-all-arm64` run and confirm it lands on the gaia/docker backend (and amd64 on artemis/kubernetes). Originals are in git history for rollback. Note: `archive-missed-packages.yaml` was already a single consolidated matrix file (cron + manual) and is left untouched. Reviewed-on: #93	2026-06-16 09:33:13 +00:00
pat-s	4406a8735b	refactor(ci): consolidate per-platform crow workflows into 3 matrix files (#92) ... ## Summary Collapse **44** per-platform crow pipeline files into **3** matrix-driven files (one per family), using `matrix.include` + the crow #1165 declarative manual `variables:` block: | Family | Before | After | |---|---|---| | process-updates | 14 | `process-updates.yaml` | | weekly-audit-missing | 16 | `weekly-audit-missing.yaml` | | weekly-rebuild-missing | 14 | `weekly-rebuild-missing.yaml` | - **One file per family** (not split by arch). Arch placement is via the agent label as a matrix var (`artemis`=amd64, `gaia`=arm64) for process-updates and weekly-rebuild; weekly-audit keeps its arch-`nodeSelector` placement (no agent label). - **Matrix axis is named `OS`** (e.g. `redhat-9`) per request, with per-row `R_VERSION`, image tag, codename, and `process_new` (FALSE for alpine). - **Runs preserved 1:1:** - *Cron*: each existing `<family>-<os>-<arch>` cron still fires only its matching matrix row via `CI_PIPELINE_CRON`. Server-side cron entries unchanged. - *Manual*: a #1165 dropdown variable (`process_cran_updates` / `weekly_audit_missing` / `weekly_rebuild_missing`) selects a single `<os>-<arch>` or `all`. weekly-rebuild defaults to `all`, matching its former bare `event: manual` trigger. ## Behavior deltas (intentional, flagged) To fit one file per family the backend was unified: - The amd64-only `node.kubernetes.io/instance-type: AX42` nodeSelector pin is **dropped**; placement now relies on the `artemis`/`gaia` agents. - **process-updates arm64** jobs now share the same `backend_options` as amd64 (resource requests/limits + tolerations) — they previously had none, so they gain an 18Gi memory limit. Tell me if arm64 should stay uncapped. ## Validation required before merge Relies on crow interpolating matrix variables inside `when.evaluate`, `labels.agent`, and `commands`. Interpolation in image/env/commands is standard crow; the novel bit is `when.evaluate`. The manual dropdown routing exercises the same interpolation as the cron routing, so on this branch: 1. Trigger manually with the dropdown = `redhat-9-amd64` -> confirm only that one job runs (image `build-env-redhat:9`, R 4.4.3, agent artemis). 2. Trigger with `all` -> confirm all os/arch jobs schedule. Crons only fire on the default branch, so no collision while unmerged. Fallback if `${...}` doesn't interpolate in `when`: a runtime `case "$CI_PIPELINE_CRON" in ...` guard, same files otherwise. Reviewed-on: #92	2026-06-16 09:33:06 +00:00
pat-s	1cd86b65e1	feat(build): skip already-built package versions on workflow restart (#91) ... ## Summary When a `build-all-*` workflow is restarted, the build job re-reads the static `pkgs_to_build.rds` that the install-deps step produced once, so it cycles over every package an interrupted run already built. This adds a DB-based skip filter so a restart only processes what is genuinely left. - At job start, `build-all.R` queries the `single_builds` metadata table for `(name, tag)` already built successfully (`error_occurred = FALSE`) on this `platform`/`arch`, and drops those pairs from the chunk before the build loop. It logs how many it skipped. - One indexed query, one round trip, run before the pak forks — no extra S3 listing and no new Python/s3fs memory pressure (`RPostgres`/`DBI` are already used in the container). - Errored versions are intentionally **not** skipped, so transient failures still get retried on restart. ## Dependency Correctness depends on a `error_occurred = FALSE` row meaning the binary is actually published. That guarantee is added in rpkgs/bincraft#56 (success row written only after a confirmed S3 upload). This PR should land together with / after a bincraft release including that fix. Reviewed-on: #91	2026-06-16 07:27:20 +00:00
pat-s	8cf8e33bf5	fix(ci): build per-minor binaries with a per-minor R library ... The sensitive-only multi-R build passes ran under each minor's R but loaded bincraft from the shared R_LIBS_USER=/mnt/cache/R-pkgs cache, which is compiled under the primary R and fails to load under a newer minor (e.g. 4.6: "undefined symbol: SETLENGTH") — so those minor builds silently no-op'd. Give each non-primary minor its own R_LIBS_USER=/mnt/cache/R-pkgs-<minor> and install bincraft 4.2.1 there (built for that minor); the rest of bincraft's deps load from the per-version /opt/R/<minor>/lib. Applied to the build-all-versions and process-updates sensitive-only passes.	2026-06-14 22:22:56 +02:00
pat-s	61a9ac5609	fix(ci): run per-minor index under the primary R in process-updates ... The per-minor upload_package_index() loop invoked each minor's own R, which loads bincraft from the shared /mnt/cache/R-pkgs cache (compiled under the primary R). Under a newer minor (e.g. 4.6) that fails to load ("undefined symbol: SETLENGTH") and the index step dies. Indexing is pure S3 work and needs no specific R, so run it under /opt/R/$R_VERSION like the generic index and build-all-versions already do.	2026-06-14 22:18:19 +02:00
pat-s	95bcf6b377	chore: build alpine 3.24	2026-06-14 14:12:05 +02:00
pat-s	f51617018d	ci: bump bincraft to 4.2.1 and scope multi-R loops to versioned R dirs ... - Pin bincraft v4.2.0 -> v4.2.1 (per-minor archive.rds/index fix) across the process-updates, weekly-rebuild and archive workflows. - Restrict the per-R-minor build and index loops to /opt/R/[0-9]* so the `current` symlink dir is not treated as a distinct minor (avoids double- building a minor and writing a bogus contrib/current/ index slot).	2026-06-14 13:02:47 +02:00
pat-s	d558e27c11	feat: R-minor-sensitive binary builds (full + iterative) (#84) ... ## Summary Builds R-minor-sensitive CRAN packages once per installed R minor version (into per-minor S3 slots `…/contrib/<x.y>/`) and everything else once into the generic slot, driven by bincraft 4.2.0's ABI classifier. Both the full and iterative pipelines are covered. ## What's in here **Detection / precompute** - `local/r-minor-helpers.R` — pure `classify_from_metadata()` (NeedsCompilation / risky `LinkingTo`) + `parse_build_args()`, with unit tests. - `local/packages-to-build.R` — adds a per-package `r_minor_sensitive` flag: cheap CRAN-metadata rules first, source download + `bincraft::needs_per_minor_recompile()` only for the ambiguous compiled subset (fail-safe to sensitive). Classified once per package, applied to all versions. **Full build** - `local/build-all.R` — passes the per-row `is_r_minor_sensitive` flag; new `--sensitive-only` mode builds just the risky subset. - `.crow/build-all-versions-{amd64,arm64}.yaml` — install-deps persists the sensitive subset; build step runs a sensitive-only pass under each non-primary `/opt/R/*` minor; new step uploads the generic index plus a per-minor index for each minor. **Iterative build** - All 14 `.crow/process-updates-*.yaml` — primary pass uses `r_minor_detection = 'classifier'`; a sensitive-only multi-R pass builds risky updates under each other minor; per-minor index upload added. **Tooling / housekeeping** - Pins bincraft `v4.1.1` → `v4.2.0` across all workflows; removes the superseded standalone `build-r-minor-sensitive-packages.yaml`. - Adds prek/pre-commit hooks (prettier, markdownlint, editorconfig-checker, yamllint, air) and applies them repo-wide; excludes the verbatim GPL `LICENSE.md` and auxiliary shell scripts. - Design + implementation docs under `docs/superpowers/`. ## Requires before merge - A `v4.2.0` git tag must be pushed on the bincraft repo (codefloe.com/rpkgs/bincraft) — the workflow install steps pin `@v4.2.0`. The full-build install-deps clones `main`, so it is unaffected. Reviewed-on: #84	2026-06-14 08:04:40 +00:00