Author	SHA1	Message	Date
pat-s	4406a8735b	refactor(ci): consolidate per-platform crow workflows into 3 matrix files (#92) ... ## Summary Collapse **44** per-platform crow pipeline files into **3** matrix-driven files (one per family), using `matrix.include` + the crow #1165 declarative manual `variables:` block: | Family | Before | After | |---|---|---| | process-updates | 14 | `process-updates.yaml` | | weekly-audit-missing | 16 | `weekly-audit-missing.yaml` | | weekly-rebuild-missing | 14 | `weekly-rebuild-missing.yaml` | - **One file per family** (not split by arch). Arch placement is via the agent label as a matrix var (`artemis`=amd64, `gaia`=arm64) for process-updates and weekly-rebuild; weekly-audit keeps its arch-`nodeSelector` placement (no agent label). - **Matrix axis is named `OS`** (e.g. `redhat-9`) per request, with per-row `R_VERSION`, image tag, codename, and `process_new` (FALSE for alpine). - **Runs preserved 1:1:** - *Cron*: each existing `<family>-<os>-<arch>` cron still fires only its matching matrix row via `CI_PIPELINE_CRON`. Server-side cron entries unchanged. - *Manual*: a #1165 dropdown variable (`process_cran_updates` / `weekly_audit_missing` / `weekly_rebuild_missing`) selects a single `<os>-<arch>` or `all`. weekly-rebuild defaults to `all`, matching its former bare `event: manual` trigger. ## Behavior deltas (intentional, flagged) To fit one file per family the backend was unified: - The amd64-only `node.kubernetes.io/instance-type: AX42` nodeSelector pin is **dropped**; placement now relies on the `artemis`/`gaia` agents. - **process-updates arm64** jobs now share the same `backend_options` as amd64 (resource requests/limits + tolerations) — they previously had none, so they gain an 18Gi memory limit. Tell me if arm64 should stay uncapped. ## Validation required before merge Relies on crow interpolating matrix variables inside `when.evaluate`, `labels.agent`, and `commands`. Interpolation in image/env/commands is standard crow; the novel bit is `when.evaluate`. The manual dropdown routing exercises the same interpolation as the cron routing, so on this branch: 1. Trigger manually with the dropdown = `redhat-9-amd64` -> confirm only that one job runs (image `build-env-redhat:9`, R 4.4.3, agent artemis). 2. Trigger with `all` -> confirm all os/arch jobs schedule. Crons only fire on the default branch, so no collision while unmerged. Fallback if `${...}` doesn't interpolate in `when`: a runtime `case "$CI_PIPELINE_CRON" in ...` guard, same files otherwise. Reviewed-on: #92	2026-06-16 09:33:06 +00:00
pat-s	1cd86b65e1	feat(build): skip already-built package versions on workflow restart (#91) ... ## Summary When a `build-all-*` workflow is restarted, the build job re-reads the static `pkgs_to_build.rds` that the install-deps step produced once, so it cycles over every package an interrupted run already built. This adds a DB-based skip filter so a restart only processes what is genuinely left. - At job start, `build-all.R` queries the `single_builds` metadata table for `(name, tag)` already built successfully (`error_occurred = FALSE`) on this `platform`/`arch`, and drops those pairs from the chunk before the build loop. It logs how many it skipped. - One indexed query, one round trip, run before the pak forks — no extra S3 listing and no new Python/s3fs memory pressure (`RPostgres`/`DBI` are already used in the container). - Errored versions are intentionally **not** skipped, so transient failures still get retried on restart. ## Dependency Correctness depends on a `error_occurred = FALSE` row meaning the binary is actually published. That guarantee is added in rpkgs/bincraft#56 (success row written only after a confirmed S3 upload). This PR should land together with / after a bincraft release including that fix. Reviewed-on: #91	2026-06-16 07:27:20 +00:00
pat-s	f51617018d	ci: bump bincraft to 4.2.1 and scope multi-R loops to versioned R dirs ... - Pin bincraft v4.2.0 -> v4.2.1 (per-minor archive.rds/index fix) across the process-updates, weekly-rebuild and archive workflows. - Restrict the per-R-minor build and index loops to /opt/R/[0-9]* so the `current` symlink dir is not treated as a distinct minor (avoids double- building a minor and writing a bogus contrib/current/ index slot).	2026-06-14 13:02:47 +02:00
automation-bot	f28ccaf008	chore(deps): update reg.devxy.io/rpkgs/build-env-alpine docker tag to v3.24	2026-06-14 00:33:22 +00:00
automation-bot	d018f31376	chore(deps): update dependency bincraft to v4.2.0	2026-06-14 00:32:46 +00:00
pat-s	2ba6ab28f7	chore: bump to bincraft 4.1.1	2026-05-31 09:30:02 +02:00
pat-s	b1bed68898	refactor(ci): skip bincraft reinstall when image already has it (#72) ... ## Summary bincraft is baked into the build-env images, but every workflow still unconditionally calls `pak::pak("git::.../bincraft.git")` (~30–60s per job). Pin bincraft to a specific tag (currently `v4.0.2`) and guard the install on a version check. ``` - /opt/R/$R_VERSION/bin/R -q -e 'if (!requireNamespace("bincraft", quietly = TRUE) || packageVersion("bincraft") != "4.0.2") pak::pak("git::https://codefloe.com/rpkgs/bincraft.git@v4.0.2")' ``` - Image matches the pin → skip install entirely. - Image is missing bincraft or has a different version → install the pinned tag (covers both broken bakes and intentional version bumps). - Renovate customManager keeps the pin and the version-check string in sync whenever bincraft cuts a new tag (uses `git-tags` datasource against the bincraft repo). Touches 43 `.crow` YAMLs + `renovate.json`. Reviewed-on: #72	2026-05-28 12:37:58 +00:00
pat-s	8c5362b7ab	refactor: R_VERSION syntax	2026-05-27 22:36:09 +02:00
pat-s	6faa3772d1	refactor(ci): use multi-R-version images across workflows (#70) ... ## Summary - Switches all 64 `.crow/*.yaml` workflows, the `Justfile`, and the commented-out `build-all-versions-install-deps.yaml` template to the new multi-R-version container images. - Each image tag drops `-${R_VERSION}` (e.g. `build-env-alpine:3.22-4.5` → `build-env-alpine:3.22`); workflows pick an R version by calling `/opt/R/${R_VERSION}/bin/R` explicitly. - Pattern-1 workflows (hard-coded image tag) gain a single `R_VERSION:` env var per file. - Pattern-2 workflows (`build-all-versions-*`, matrix-driven) continue to receive `R_VERSION` via `crow pipeline create --var`. ## Platform → R_VERSION mapping | Platform | Image | R_VERSION | | --- | --- | --- | | alpine 3.22 / 3.23 | `build-env-alpine:3.{22,23}` | 4.5.3 | | ubuntu jammy / noble | `build-env-ubuntu:{jammy,noble}` | 4.4.3 | | redhat 8 / 9 | `build-env-redhat:{8,9}` | 4.4.3 | | redhat 10 | `build-env-redhat:10` | 4.5.3 | The `alpine-321` audit-only workflows fall back to `build-env-alpine:3.23` since no 3.21 image exists in the new scheme; the audits only query S3/CRAN, so the container OS does not affect correctness. ## Bug fixes folded in 1. All six `weekly-audit-missing-alpine-*.yaml` files were pinned to `alpine:3.23-4.5` regardless of platform. Now each one points at its own alpine image. 2. All fourteen `update-package-index-*.yaml` files were pinned to `build-env-ubuntu:noble-4.4` (or `noble-4.5` for `redhat-10-amd64`) regardless of platform. Now each one uses its own platform's image. 3. `process-updates-ubuntu-2404-{amd64,arm64}.yaml` had `R_VERSION` of `4.4` (no patch) baked into the tag. Aligned to `4.4.3`, matching the audit and rebuild counterparts. ## Smoke targets (post-merge) One workflow per shape, ready to dry-run: - `process-updates-alpine-322-amd64.yaml` - `weekly-rebuild-missing-redhat-9-amd64.yaml` - `weekly-audit-missing-ubuntu-2204-amd64.yaml` - `update-package-index-redhat-10-amd64.yaml` - `archive-missed-packages.yaml` - `build-all-versions-amd64.yaml` (with its install-deps predecessor) - `build-r-minor-sensitive-packages.yaml` ## Follow-ups (out of scope, surfaced during review) - `process-updates-redhat-10-{amd64,arm64}.yaml` set `codename = "rhel9"` in their `upload_package_index(...)` calls. Pre-existing bug — should be `"rhel10"`. The `update-package-index-redhat-10-*` counterparts already use `"rhel10"` correctly. - `update-package-index-redhat-8-amd64.yaml` is missing the `packageVersion("bincraft")` diagnostic line that every sibling file has. Pre-existing asymmetry between amd64 and arm64. ## Spec & plan - Spec: `docs/superpowers/specs/2026-05-25-multi-r-version-images-design.md` - Plan: `docs/superpowers/plans/2026-05-25-multi-r-version-images.md` Reviewed-on: #70	2026-05-26 08:51:27 +00:00
pat-s	84c59d1984	fix: audit workflows	2026-04-21 21:54:48 +02:00
automation-bot	477d21ce54	chore(deps): update reg.devxy.io/rpkgs/build-env-alpine docker tag to v3.23	2026-04-13 00:01:26 +00:00
pat-s	360aa45e90	fix: replace colon in error message to avoid YAML parsing issue in rebuild workflows	2026-04-11 21:53:22 +02:00
pat-s	5d9bdbb3a9	feat: add 14 weekly rebuild workflows for all platform/arch combos	2026-04-11 21:46:38 +02:00