ci: build last four minor R versions instead of only the latest (#6)
All checks were successful
ci/crow/cron/build/1 Pipeline was successful
ci/crow/cron/build/2 Pipeline was successful
ci/crow/cron/build/10 Pipeline was successful
ci/crow/cron/build/5 Pipeline was successful
ci/crow/cron/build/17 Pipeline was successful
ci/crow/cron/build/18 Pipeline was successful
ci/crow/cron/build/15 Pipeline was successful
ci/crow/cron/build/8 Pipeline was successful
ci/crow/cron/build/6 Pipeline was successful
ci/crow/cron/build/12 Pipeline was successful
ci/crow/cron/build/4 Pipeline was successful
ci/crow/cron/build/14 Pipeline was successful
ci/crow/cron/build/16 Pipeline was successful
ci/crow/cron/build/9 Pipeline was successful
ci/crow/cron/build/7 Pipeline was successful
ci/crow/cron/build/13 Pipeline was successful
ci/crow/cron/build/11 Pipeline was successful
ci/crow/cron/build/3 Pipeline was successful

## Summary

The crow build pipeline previously built only the single latest stable R version (from the brew formula).
It now builds the latest patch release of each of the **last four minor R versions** and uploads any that are missing from S3, so older supported minors stay available rather than only the newest stable.

- **Get R versions to build**: derive the four versions from posit's `versions.json` (the same source upstream's `test/get_r_versions.py` uses) via jq — keep only `x.y.z` releases, take the latest patch per minor, then the top four minors (e.g. `4.6.0, 4.5.3, 4.4.3, 4.3.3`).
- **Check which R versions already exist in s3**: iterate the four versions against the S3 listing for the platform/arch.
- **Build**: build only the versions that are missing. In steady state nothing builds; when a new minor lands, only that one is built.

## awk fix

The existence check used an awk regex (`$0 ~ ver`), so the unescaped dots in a version made `4.4.3` spuriously match `4.3.3` (the substring `4/4.3` satisfies the pattern).
Harmless with a single version, but across four adjacent versions it would wrongly skip builds.
Switched to literal substring matching via awk `index($0, ver)`.

Reviewed-on: #6
This commit is contained in:
Patrick Schratz 2026-06-12 17:16:43 +00:00 committed by Patrick Schratz
commit 56c75e2df0

View file

@ -122,15 +122,26 @@ steps:
kubernetes.io/arch: "${ARCH}"
- name: Get latest R version
- name: Get R versions to build
image: reg.devxy.io/docker.io/library/alpine:3.23
privileged: true
commands:
- ip link set dev eth0 mtu 1280 2>/dev/null || true
- for i in 1 2 3 4 5; do apk add -q --no-cache curl jq && break; sleep 5; done
- curl -sf https://formulae.brew.sh/api/formula/r.json | jq -er '.versions.stable' > r-version-to-build.txt
# - echo "4.1.3" > r-version-to-build.txt
- cat r-version-to-build.txt
# Latest patch release of each of the last 4 minor R versions (e.g. 4.6.0, 4.5.3, 4.4.3, 4.3.3).
- |
curl -sf https://cdn.posit.co/r/versions.json | jq -r '
.r_versions
| map(select(test("^[0-9]+\\.[0-9]+\\.[0-9]+$")))
| group_by(split(".")[0:2] | join("."))
| map(max_by(split(".") | map(tonumber)))
| sort_by(split(".") | map(tonumber))
| reverse
| .[0:4]
| .[]' > r-versions-to-build.txt
# To pin specific versions for testing, overwrite the file, e.g.:
# - printf '4.1.3\n' > r-versions-to-build.txt
- cat r-versions-to-build.txt
backend_options:
kubernetes:
resources:
@ -144,19 +155,23 @@ steps:
kubernetes.io/arch: "${ARCH}"
- name: Check if files for R version already exist in s3
- name: Check which R versions already exist in s3
image: reg.devxy.io/docker.io/library/alpine:3.23
privileged: true
commands:
- ip link set dev eth0 mtu 1280 2>/dev/null || true
- for i in 1 2 3 4 5; do apk add -q --no-cache curl && break; sleep 5; done
- |
VERSION=$(cat r-version-to-build.txt)
curl -s "https://s3.eu-central-003.backblazeb2.com/devxy-r-builds?prefix=${PLATFORM_ID}/" | \
grep -oE '<Key>[^<]+</Key>' | sed 's/<[^>]*>//g' | \
awk -v arch="${ARCH_ID}" -v ver="${VERSION}" \
'$0 ~ arch && $0 ~ ver {print substr($NF, 3, 5)}' > r-version-s3.txt
- cat r-version-s3.txt
LISTING=$(curl -s "https://s3.eu-central-003.backblazeb2.com/devxy-r-builds?prefix=${PLATFORM_ID}/" | \
grep -oE '<Key>[^<]+</Key>' | sed 's/<[^>]*>//g')
: > r-versions-existing.txt
for VERSION in $(cat r-versions-to-build.txt); do
if printf '%s\n' "$LISTING" | \
awk -v arch="${ARCH_ID}" -v ver="$VERSION" 'index($0, arch) && index($0, ver) { found=1 } END { exit !found }'; then
echo "$VERSION" >> r-versions-existing.txt
fi
done
- echo "Already present in s3:"; cat r-versions-existing.txt
backend_options:
kubernetes:
nodeSelector:
@ -172,9 +187,20 @@ steps:
commands: |
ip link set dev eth0 mtu 1280 2>/dev/null || true
# docker info
if ! grep -qF "$(cat r-version-to-build.txt)" r-version-s3.txt; then
TO_BUILD=""
for VERSION in $(cat r-versions-to-build.txt); do
if grep -qxF "$VERSION" r-versions-existing.txt; then
echo "R $VERSION already exists for ${PLATFORM} (${ARCH_ID}), skipping"
else
TO_BUILD="$TO_BUILD $VERSION"
fi
done
if [ -n "$TO_BUILD" ]; then
for i in 1 2 3 4 5; do apk add -q --no-cache make just docker-compose && break; sleep 5; done
just build-r-${PLATFORM} $(cat r-version-to-build.txt)
for VERSION in $TO_BUILD; do
echo "Building R $VERSION for ${PLATFORM}"
just build-r-${PLATFORM} "$VERSION"
done
fi
backend_options:
kubernetes: