Commit graph build-cran-binaries/.crow/build-all-versions.yaml
Author SHA1 Message Date
6958a55028
fix(ci): derive build-all platform/arch in-container, not from CI env
Crow cannot interpolate the workflow-level OS/OS_VERSION form variables inside
an environment: block (only matrix variables work there), so exporting them
broke pipeline parsing with "unable to parse variable name".

Instead, derive platform + arch inside build-all.R from the running container,
mirroring bincraft's own codename -> platform mapping and machine -> arch
detection. This keeps the already-built dedup pre-filter effective without
depending on CI-injected env vars, and degrades to a length-1 NA (querying
nothing) for an unmapped codename rather than crashing.
2026-06-18 13:38:19 +02:00
1b63f7fb0c
feat(ci): make OS and OS_VERSION manual-run dropdowns
Give the OS and OS_VERSION form variables explicit option lists like
target_arch and R_VERSION already have, so the manual-run form presents
dropdowns instead of free-text. Crow form variables are independent, so the
operator must still pick a coherent OS + OS_VERSION combination.
2026-06-18 13:38:18 +02:00
0f4330c05c fix(ci): make build-all self-sufficient on agents without install-deps (#96)
## Summary

Fixes the recurring `build-all-*` failure on agents where `build-all-versions-install-deps` did **not** run.

Both symptoms in the logs — `Parameter 3 does not have length 1` (repeated, in the metadata DB queries) and `argument is of length zero` (system-dependency install) — were the same bug: inside bincraft, `platform` was zero-length. `local/build-all.R` calls `build_binary_package()` without passing `platform`, so bincraft resolves it from the container codename. The primary build step never (re)installed/pinned bincraft and relied on whatever sat in the **per-agent** cache volume; `depends_on` only orders steps, it does not co-locate them on the same agent, so a job landing where install-deps never ran got a stale bincraft that left `platform` empty.

## Changes

- **`build-all-versions.yaml`**: pin bincraft `@v4.2.3` in the primary build step (mirroring the R-minor pass and `process-updates.yaml`), so every agent uses a known-good bincraft regardless of where install-deps ran.
- Align the R-minor pass `v4.2.2 → v4.2.3`.
- Export `OS`/`OS_VERSION`/`ARCH` as runtime env vars — previously only available for `${...}` interpolation, so `build-all.R`'s already-built dedup query matched platform `"-"` and skipped nothing.
- Fix the unarchive call: pass the codename via `bincraft::set_codename(NULL)` instead of the malformed `paste(OS, OS_VERSION)` (`"alpine 3.24"`), matching `archive-missed-packages.yaml`.
- **`build-all-versions-install-deps.yaml`**: pin install-deps to `@v4.2.3` (was installing HEAD), so the precomputed snapshot and per-agent library stay consistent pipeline-wide.

Reviewed-on: #96
2026-06-18 11:27:02 +00:00
f9bc72d5ab
chore: bump bincraft to 4.2.3
Some checks failed
ci/crow/manual/build-all-versions-install-deps/1 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/manual/build-all-versions/3 Pipeline failed
ci/crow/manual/build-all-versions/4 Pipeline failed
ci/crow/cron/process-updates/6 Pipeline failed
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/manual/build-all-versions/2 Pipeline failed
ci/crow/manual/build-all-versions/1 Pipeline failed
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
2026-06-18 09:43:05 +02:00
4c67703f52 refactor(ci): route workflows by agent group label instead of named agent (#94)
Some checks failed
ci/crow/manual/build-all-versions-install-deps/1 Pipeline was successful
ci/crow/manual/build-all-versions/3 Pipeline failed
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/manual/build-all-versions/2 Pipeline failed
ci/crow/manual/build-all-versions/4 Pipeline failed
ci/crow/manual/build-all-versions/1 Pipeline failed
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline failed
## Summary

- Switch crow workflow placement from a per-agent label (`agent: ${AGENT}`) to a group label (`group: rpkgs-${ARCH}`), so jobs target the `rpkgs-amd64` / `rpkgs-arm64` agent pools instead of a single named agent (`artemis`/`gaia`).
- Remove the now-unused `AGENT` matrix variable from all build/process/weekly workflows.
- Update header comments to reflect group-based placement.

Agents opt into a pool via `CROW_AGENT_LABELS="group=rpkgs-<arch>"`. The group name derives from each matrix row's `ARCH`, so adding or replacing agents no longer requires touching the workflow files.

Reviewed-on: #94
2026-06-17 16:26:04 +00:00
b297f0fbc0 refactor(ci): consolidate build-all-versions workflows into 2 files (#93)
Some checks failed
ci/crow/manual/build-all-versions-install-deps/1 Pipeline was successful
ci/crow/manual/build-all-versions/1 Pipeline failed
ci/crow/manual/build-all-versions/2 Pipeline failed
ci/crow/manual/build-all-versions/3 Pipeline failed
ci/crow/manual/build-all-versions/4 Pipeline failed
## Summary

Consolidate the four `build-all-versions*` files into two, mirroring the matrix approach used for the process-updates / weekly families:

| Before | After |
|---|---|
| `build-all-versions-amd64.yaml` + `build-all-versions-arm64.yaml` | `build-all-versions.yaml` |
| `build-all-versions-install-deps-amd64.yaml` + `-arm64.yaml` | `build-all-versions-install-deps.yaml` |

`ARCH` becomes a matrix axis carrying the per-arch differences: `agent` (artemis/gaia), `BACKEND` (kubernetes/docker), and `R_PKG_CACHE_DIR`. Build-all keeps its `SPLIT_INDEX` parallelism, so its matrix is ARCH x SPLIT = 8 rows; only the rows matching the selected arch run.

**Invocation unchanged.** Routing uses the existing task selector `task == "build-all-${ARCH}"`, so `--var task=build-all-amd64 ...` still runs exactly the amd64 rows. `OS` / `OS_VERSION` / `R_VERSION` / `K8S_INSTANCE_TYPE` are still passed via `--var`. (You can drop `--var ARCH=...` now — arch comes from the matrix/task.)

**No cron.** These pipelines are manual-only, so there are no scheduled runs to affect.

## Validation before merge

Same matrix-interpolation-in-`when`/`labels` mechanism validated for #92, plus it now relies on per-row `agent`/`BACKEND` selecting the right execution backend (kubernetes for amd64, docker for arm64). Before merge, on this branch trigger a small `task=build-all-arm64` run and confirm it lands on the gaia/docker backend (and amd64 on artemis/kubernetes). Originals are in git history for rollback.

Note: `archive-missed-packages.yaml` was already a single consolidated matrix file (cron + manual) and is left untouched.
Reviewed-on: #93
2026-06-16 09:33:13 +00:00
6c499afe1c
fix: use separate yamls for arm and amd
Some checks failed
ci/crow/manual/build-all-versions-install-deps Pipeline was successful
ci/crow/cron/update-package-index-alpine-322-arm64 Pipeline failed
ci/crow/cron/update-package-index-alpine-321-amd64 Pipeline failed
ci/crow/cron/update-package-index-redhat-9-amd64 Pipeline failed
ci/crow/cron/update-package-index-alpine-320-amd64 Pipeline failed
ci/crow/cron/update-package-index-ubuntu-2204-amd64 Pipeline failed
ci/crow/cron/update-package-index-redhat-8-amd64 Pipeline was successful
ci/crow/cron/update-package-index-ubuntu-2404-amd64 Pipeline was successful
ci/crow/cron/update-package-index-ubuntu-2204-arm64 Pipeline was successful
ci/crow/cron/update-package-index-alpine-321-arm64 Pipeline was successful
ci/crow/cron/update-package-index-alpine-320-arm64 Pipeline was successful
ci/crow/cron/update-package-index-redhat-9-arm64 Pipeline was successful
ci/crow/cron/update-package-index-alpine-322-amd64 Pipeline was successful
ci/crow/cron/update-package-index-redhat-8-arm64 Pipeline was successful
ci/crow/cron/update-package-index-ubuntu-2404-arm64 Pipeline was successful
ci/crow/manual/build-all-versions-arm64/4 Pipeline failed
ci/crow/manual/build-all-versions-arm64/9 Pipeline failed
ci/crow/manual/build-all-versions-arm64/2 Pipeline failed
ci/crow/manual/build-all-versions-arm64/7 Pipeline failed
ci/crow/manual/build-all-versions-arm64/1 Pipeline failed
ci/crow/manual/build-all-versions-arm64/8 Pipeline failed
ci/crow/manual/build-all-versions-arm64/3 Pipeline failed
ci/crow/manual/build-all-versions-arm64/5 Pipeline failed
ci/crow/manual/build-all-versions-arm64/6 Pipeline failed
ci/crow/manual/build-all-versions-arm64/10 Pipeline failed
2025-06-18 12:52:41 +02:00
a671dafbd7
fix: set instance-type filer for k8s backend and resource limits for docker
Some checks failed
ci/crow/manual/build-all-versions/6 Pipeline is pending
ci/crow/manual/build-all-versions/7 Pipeline is pending
ci/crow/manual/build-all-versions/8 Pipeline is pending
ci/crow/manual/build-all-versions/9 Pipeline is pending
ci/crow/manual/build-all-versions/10 Pipeline is pending
ci/crow/manual/build-all-versions-install-deps Pipeline was successful
ci/crow/manual/build-all-versions/2 Pipeline failed
ci/crow/manual/build-all-versions/4 Pipeline failed
ci/crow/manual/build-all-versions/3 Pipeline failed
ci/crow/manual/build-all-versions/5 Pipeline failed
ci/crow/manual/build-all-versions/1 Pipeline failed
2025-06-18 12:41:19 +02:00
1e5042548c
wipe pkgcache dir
Some checks failed
ci/crow/manual/build-all-versions-install-deps Pipeline was successful
ci/crow/manual/build-all-versions/6 Pipeline failed
ci/crow/manual/build-all-versions/7 Pipeline failed
ci/crow/manual/build-all-versions/5 Pipeline failed
ci/crow/manual/build-all-versions/1 Pipeline failed
ci/crow/manual/build-all-versions/3 Pipeline failed
ci/crow/manual/build-all-versions/4 Pipeline failed
ci/crow/manual/build-all-versions/2 Pipeline failed
ci/crow/manual/build-all-versions/8 Pipeline failed
ci/crow/manual/build-all-versions/10 Pipeline failed
ci/crow/manual/build-all-versions/9 Pipeline failed
2025-06-18 11:25:56 +02:00
e2493de1e8
five splits
Some checks failed
ci/crow/cron/process-updates-ubuntu-2404-arm64 Pipeline was successful
ci/crow/manual/build-all-versions-install-deps Pipeline was successful
ci/crow/manual/build-all-versions/1 Pipeline failed
ci/crow/manual/build-all-versions/3 Pipeline failed
ci/crow/manual/build-all-versions/2 Pipeline failed
ci/crow/manual/build-all-versions/4 Pipeline failed
ci/crow/manual/build-all-versions/5 Pipeline failed
2025-06-18 09:10:56 +02:00
6b1ae4e555
chore: cleanup workflows
Some checks failed
ci/crow/cron/process-updates-ubuntu-2404-arm64 Pipeline failed
ci/crow/manual/build-all-versions-install-deps Pipeline was successful
ci/crow/manual/build-all-versions/7 Pipeline is running
ci/crow/manual/build-all-versions/3 Pipeline is running
ci/crow/manual/build-all-versions/4 Pipeline is running
ci/crow/manual/build-all-versions/9 Pipeline is running
ci/crow/manual/build-all-versions/1 Pipeline is running
ci/crow/manual/build-all-versions/8 Pipeline is running
ci/crow/manual/build-all-versions/2 Pipeline is running
ci/crow/manual/build-all-versions/5 Pipeline is running
ci/crow/manual/build-all-versions/10 Pipeline is running
ci/crow/manual/build-all-versions/6 Pipeline is running
2025-06-17 21:03:49 +02:00
cdd823da82
ci: new generic build-all workflow
Some checks failed
ci/crow/manual/build-all-versions-install-deps Pipeline was successful
ci/crow/manual/build-all-versions/8 Pipeline failed
ci/crow/manual/build-all-versions/2 Pipeline failed
ci/crow/manual/build-all-versions/7 Pipeline failed
ci/crow/manual/build-all-versions/1 Pipeline failed
ci/crow/manual/build-all-versions/4 Pipeline failed
ci/crow/manual/build-all-versions/5 Pipeline failed
ci/crow/manual/build-all-versions/6 Pipeline failed
ci/crow/manual/build-all-versions/3 Pipeline failed
ci/crow/manual/build-all-versions/9 Pipeline failed
ci/crow/manual/build-all-versions/10 Pipeline failed
2025-06-17 09:58:44 +02:00