build-cran-binaries/.crow/build-all-versions-amd64.yaml
pat-s 6faa3772d1
Some checks failed
ci/crow/cron/update-package-index-alpine-322-arm64 Pipeline is pending
ci/crow/cron/update-package-index-alpine-323-amd64 Pipeline is pending
ci/crow/cron/update-package-index-alpine-323-arm64 Pipeline is pending
ci/crow/cron/update-package-index-redhat-10-amd64 Pipeline is pending
ci/crow/cron/update-package-index-redhat-10-arm64 Pipeline is pending
ci/crow/cron/update-package-index-redhat-8-amd64 Pipeline is pending
ci/crow/cron/update-package-index-redhat-8-arm64 Pipeline is pending
ci/crow/cron/update-package-index-redhat-9-amd64 Pipeline is pending
ci/crow/cron/update-package-index-redhat-9-arm64 Pipeline is pending
ci/crow/cron/update-package-index-ubuntu-2204-amd64 Pipeline is pending
ci/crow/cron/update-package-index-ubuntu-2204-arm64 Pipeline is pending
ci/crow/cron/update-package-index-ubuntu-2404-amd64 Pipeline is pending
ci/crow/cron/update-package-index-ubuntu-2404-arm64 Pipeline is pending
ci/crow/cron/update-package-index-alpine-322-amd64 Pipeline failed
ci/crow/cron/process-updates-alpine-322-amd64 Pipeline failed
ci/crow/cron/process-updates-alpine-322-arm64 Pipeline failed
ci/crow/cron/process-updates-redhat-8-amd64 Pipeline failed
ci/crow/cron/process-updates-redhat-8-arm64 Pipeline failed
ci/crow/cron/process-updates-redhat-9-amd64 Pipeline failed
ci/crow/cron/process-updates-alpine-323-amd64 Pipeline failed
ci/crow/cron/process-updates-alpine-323-arm64 Pipeline failed
ci/crow/cron/process-updates-redhat-9-arm64 Pipeline failed
ci/crow/cron/process-updates-ubuntu-2204-amd64 Pipeline failed
ci/crow/cron/process-updates-ubuntu-2204-arm64 Pipeline failed
ci/crow/cron/process-updates-ubuntu-2404-amd64 Pipeline failed
ci/crow/cron/process-updates-ubuntu-2404-arm64 Pipeline failed
ci/crow/cron/process-updates-redhat-10-amd64 Pipeline failed
ci/crow/cron/process-updates-redhat-10-arm64 Pipeline failed
refactor(ci): use multi-R-version images across workflows (#70)
## Summary

- Switches all 64 `.crow/*.yaml` workflows, the `Justfile`, and the commented-out `build-all-versions-install-deps.yaml` template to the new multi-R-version container images.
- Each image tag drops `-${R_VERSION}` (e.g. `build-env-alpine:3.22-4.5` → `build-env-alpine:3.22`); workflows pick an R version by calling `/opt/R/${R_VERSION}/bin/R` explicitly.
- Pattern-1 workflows (hard-coded image tag) gain a single `R_VERSION:` env var per file.
- Pattern-2 workflows (`build-all-versions-*`, matrix-driven) continue to receive `R_VERSION` via `crow pipeline create --var`.

## Platform → R_VERSION mapping

| Platform | Image | R_VERSION |
| --- | --- | --- |
| alpine 3.22 / 3.23 | `build-env-alpine:3.{22,23}` | 4.5.3 |
| ubuntu jammy / noble | `build-env-ubuntu:{jammy,noble}` | 4.4.3 |
| redhat 8 / 9 | `build-env-redhat:{8,9}` | 4.4.3 |
| redhat 10 | `build-env-redhat:10` | 4.5.3 |

The `alpine-321` audit-only workflows fall back to `build-env-alpine:3.23` since no 3.21 image exists in the new scheme; the audits only query S3/CRAN, so the container OS does not affect correctness.

## Bug fixes folded in

1. All six `weekly-audit-missing-alpine-*.yaml` files were pinned to `alpine:3.23-4.5` regardless of platform.
   Now each one points at its own alpine image.
2. All fourteen `update-package-index-*.yaml` files were pinned to `build-env-ubuntu:noble-4.4` (or `noble-4.5` for `redhat-10-amd64`) regardless of platform.
   Now each one uses its own platform's image.
3. `process-updates-ubuntu-2404-{amd64,arm64}.yaml` had `R_VERSION` of `4.4` (no patch) baked into the tag.
   Aligned to `4.4.3`, matching the audit and rebuild counterparts.

## Smoke targets (post-merge)

One workflow per shape, ready to dry-run:

- `process-updates-alpine-322-amd64.yaml`
- `weekly-rebuild-missing-redhat-9-amd64.yaml`
- `weekly-audit-missing-ubuntu-2204-amd64.yaml`
- `update-package-index-redhat-10-amd64.yaml`
- `archive-missed-packages.yaml`
- `build-all-versions-amd64.yaml` (with its install-deps predecessor)
- `build-r-minor-sensitive-packages.yaml`

## Follow-ups (out of scope, surfaced during review)

- `process-updates-redhat-10-{amd64,arm64}.yaml` set `codename = "rhel9"` in their `upload_package_index(...)` calls.
  Pre-existing bug — should be `"rhel10"`. The `update-package-index-redhat-10-*` counterparts already use `"rhel10"` correctly.
- `update-package-index-redhat-8-amd64.yaml` is missing the `packageVersion("bincraft")` diagnostic line that every sibling file has.
  Pre-existing asymmetry between amd64 and arm64.

## Spec & plan

- Spec: `docs/superpowers/specs/2026-05-25-multi-r-version-images-design.md`
- Plan: `docs/superpowers/plans/2026-05-25-multi-r-version-images.md`

Reviewed-on: #70
2026-05-26 08:51:27 +00:00

99 lines
3.5 KiB
YAML

### Required extra vars
# ARCH (amd64)
# OS (alpine)
# OS_VERSION (3.22)
# R_VERSION (4.5.3)
# crow pipeline create --var ARCH=arm64 --var OS=alpine --var OS_VERSION=3.23 --var R_VERSION=4.5.3 --var task=build-all-arm64 --branch=main --log-level=info 5
# crow pipeline create --var ARCH=arm64 --var OS=redhat --var OS_VERSION=10 --var R_VERSION=4.5.3 --var task=build-all-arm64 --branch=main --log-level=info 5
# crow pipeline create --var ARCH=amd64 --var OS=redhat --var OS_VERSION=10 --var R_VERSION=4.5.3 --var task=build-all-amd64 --branch=main --log-level=info 5
when:
- event: manual
evaluate: 'task == "build-all-amd64"'
skip_clone: true
labels:
platform: linux/amd64
agent: artemis
matrix:
include:
- SPLIT_INTO: 4
SPLIT_INDEX: 1
BACKEND: kubernetes
- SPLIT_INTO: 4
SPLIT_INDEX: 2
BACKEND: kubernetes
- SPLIT_INTO: 4
SPLIT_INDEX: 3
BACKEND: kubernetes
- SPLIT_INTO: 4
SPLIT_INDEX: 4
BACKEND: kubernetes
# - SPLIT_INTO: 4
# SPLIT_INDEX: 5
# BACKEND: kubernetes
depends_on:
- build-all-versions-install-deps-amd64
steps:
- name: 'Build binaries'
image: "reg.devxy.io/rpkgs/build-env-${OS}:${OS_VERSION}"
pull: true
environment:
RED_HAT_DEV_PW:
from_secret: RED_HAT_DEV_PW
B2_S3_ACCESS_KEY:
from_secret: B2_S3_ACCESS_KEY
B2_S3_SECRET_KEY:
from_secret: B2_S3_SECRET_KEY
PGPASS:
from_secret: PGPASS
REPO_RO_TOKEN:
from_secret: REPO_RO_TOKEN
GITHUB_PAT:
from_secret: GITHUB_PAT
# normal env vars
GIT_USER: pat-s
# set the location of the 'pkgcache' cache dir which persists the R package dependencies needed to install the packages themselves
R_PKG_CACHE_DIR: ""
R_LIBS_USER: /mnt/cache/R-pkgs
CCACHE_DIR: /mnt/cache/ccache
NCPUS: 2
volumes:
- ${ARCH}-binaries-r-dep-cache-${OS}-${OS_VERSION//./}:/mnt/cache
commands:
- git clone -q https://pat-s:$$REPO_RO_TOKEN@git.devxy.io/devxy/build-cran-binaries.git .
- mkdir -p /mnt/cache/pkgcache /mnt/cache/R-pkgs /mnt/cache/ccache /mnt/cache/packages
# Windows-only: 'RInno', 'KeyboardSimulator', 'R2PPT', 'RWinEdt', 'blatr', 'excel.link', 'spectrino', 'taskscheduleR', 'MDSGUI', 'BiplotGUI', 'R2wd', 'rFUSION', 'MediaNews', 'rFUSION', 'MediaNews'
# pkgs = readRDS('/mnt/cache/pkgs_amd64.rds');
- XVFB=$(command -v xwfb-run 2>/dev/null || command -v xvfb-run); XVFB_ARGS=""; if command -v xwfb-run >/dev/null 2>&1; then dnf install -y -q weston 2>/dev/null; XVFB_ARGS="-c weston"; fi
- $XVFB $XVFB_ARGS -n $SPLIT_INDEX -- /opt/R/${R_VERSION}/bin/Rscript local/build-all.R $SPLIT_INTO $SPLIT_INDEX $NCPUS 2>&1
# archive missed packages
- /opt/R/${R_VERSION}/bin/R -q -e "bincraft::process_unarchived_pkgs(paste(Sys.getenv('OS'), Sys.getenv('OS_VERSION')), Sys.getenv('ARCH'), workers = $NCPUS)"
backend_options:
docker:
resources:
requests:
memory: 5Gi
cpu: 1000m
limits:
memory: 20Gi
cpu: 2000m
kubernetes:
resources:
requests:
memory: 5Gi
cpu: 1000m
limits:
memory: 20Gi
cpu: 2000m
nodeSelector:
kubernetes.io/arch: ${ARCH}
node.kubernetes.io/instance-type: ${K8S_INSTANCE_TYPE}
tolerations:
- key: 'CI'
operator: 'Equal'
value: 'true'
effect: 'NoSchedule'