Compare commits

...
Sign in to create a new pull request.
Author SHA1 Message Date
fd9b8375b7
fix(ci): declare FORCE_REBUILD as a variable and make it actually take effect
Three separate reasons the escape hatch added in #9 would not have worked.

Declare FORCE_REBUILD as a top-level variable so the UI offers it as a
dropdown on a manual run, rather than requiring a hand-typed key.

Accept the value from either trigger path. `crow pipeline start -p` injects
into the step environment, while the UI and `crow pipeline create --var`
pass a substitution variable, and a fresh run of main is a create. Reading
only the bare shell variable would have ignored the flag in exactly the case
it exists for.

Test the value rather than emptiness. A declared variable substitutes its
default, so the previous `[ -n ... ]` check would have seen the literal
string "false" and rebuilt everything on every cron run.

Set overwrite: true on the upload. plugin-s3 skips an object that already
exists when overwrite is false, logging "File already exists, skipping
upload" and still exiting green, so a forced rebuild would have recompiled
every package and published none of them. Normal runs are unaffected, since
dedup means only not-yet-published versions are ever built.

Also drop the Net debug step, the commented-out Pull upstream step and a
stray `# docker info`. The MTU workaround the debug step was diagnosing
stays, as does the commented-out alpine-320 matrix entry, which records a
deliberately disabled platform rather than dead scaffolding.
2026-09-18 09:17:02 +02:00
37f2315f9b
fix(ci): honour FORCE_REBUILD from both manual trigger forms
The escape hatch added in #9 only read a bare $FORCE_REBUILD, which covers
`crow pipeline start -p FORCE_REBUILD=1`, where the parameter is injected
into the step environment. It does not cover
`crow pipeline create --branch main --var FORCE_REBUILD=1`, which passes a
crow substitution variable instead.

Triggering a fresh run of main is a `create`, so the flag would have been
silently ignored in exactly the case it is needed, and the run would skip
every Alpine version it was meant to rebuild.

Read the substituted ${FORCE_REBUILD} into FORCE_REBUILD_VAR and accept
either source. Crow blanks an unsupplied substitution variable rather than
failing, as recorded in 8eacc0a, so a plain cron run still takes the dedup
path.
2026-09-18 09:08:54 +02:00

View file

@ -1,6 +1,18 @@
when: when:
event: [cron, manual] event: [cron, manual]
variables:
FORCE_REBUILD:
description: >-
Rebuild every targeted R version even when the artifact is already in the
bucket. Needed after a builder-image change that alters the resulting R
binary, because artifact filenames carry only the R version and would
otherwise dedup to the stale build forever.
options:
- "false"
- "true"
default: "false"
matrix: matrix:
include: include:
- PLATFORM: rhel-9 - PLATFORM: rhel-9
@ -82,46 +94,6 @@ matrix:
ARCH_ID: x86_64 ARCH_ID: x86_64
steps: steps:
# Ensures that we're always using the latest available upstream build chain
# - name: Pull upstream
# image: reg.devxy.io/docker.io/library/alpine:3.20
# commands:
# - apk add -q --no-cache git
# - git remote add upstream https://github.com/cynkra/r-builds.git
# - git config --global user.email "you@example.com"
# - git config --global user.name "Your Name"
# - git config pull.ff only
# - git fetch upstream
# - git merge upstream/main --allow-unrelated-histories
#.
- name: Net debug
image: reg.devxy.io/docker.io/library/alpine:3.23
privileged: true
commands:
- ip link set dev eth0 mtu 1280 2>/dev/null || true
- echo "=== pod interface ==="
- cat /sys/class/net/eth0/mtu
- ip -o link show eth0
- echo "=== route ==="
- ip route
- echo "=== resolv.conf ==="
- cat /etc/resolv.conf
- for i in 1 2 3; do apk add -q --no-cache iputils && break; sleep 5; done || echo "apk failed; falling back to busybox ping (no -M do)"
- |
echo "=== PMTUD probes (DF=1) to dl-cdn.alpinelinux.org ==="
for sz in 1472 1392 1352 1252 1200; do
echo "--- payload=$sz (wire ~$((sz+28))) ---"
ping -c 2 -W 3 -M do -s $sz dl-cdn.alpinelinux.org || echo "(size=$sz failed)"
done
- ping -c 3 -W 5 dl-cdn.alpinelinux.org || true
- wget --tries=1 --timeout=10 -O /dev/null -nv https://dl-cdn.alpinelinux.org/ 2>&1 || true
backend_options:
kubernetes:
nodeSelector:
kubernetes.io/arch: "${ARCH}"
- name: Get R versions to build - name: Get R versions to build
image: reg.devxy.io/docker.io/library/alpine:3.23 image: reg.devxy.io/docker.io/library/alpine:3.23
privileged: true privileged: true
@ -178,14 +150,23 @@ steps:
esac esac
BASE="https://s3.eu-central-003.backblazeb2.com/devxy-r-builds/${PLATFORM_ID}" BASE="https://s3.eu-central-003.backblazeb2.com/devxy-r-builds/${PLATFORM_ID}"
: > r-versions-existing.txt : > r-versions-existing.txt
# Escape hatch for when the builder image itself changes in a way that # FORCE_REBUILD is declared as a top-level variable, so the UI offers it
# alters the resulting R binary (a newly enabled graphics device, a # as a dropdown on a manual run. The two trigger paths deliver it
# different BLAS). The artifact filename carries only the R version, so # differently, so accept either:
# dedup would otherwise keep the stale binaries forever. Set FORCE_REBUILD # UI, or crow pipeline create --var FORCE_REBUILD=true
# to a non-empty value as a pipeline variable on a manual run, or hardcode # -> a substitution variable, expanded into ${FORCE_REBUILD} below
# it here temporarily; uploads overwrite in place. # crow pipeline start -p FORCE_REBUILD=true
if [ -n "$FORCE_REBUILD" ]; then # -> injected into the step environment, read as bare $FORCE_REBUILD
echo "FORCE_REBUILD is set, ignoring artifacts already in the bucket" # The declared default is the string "false", so test the value rather
# than emptiness; an empty value means neither path supplied one.
FORCE_VALUE="${FORCE_REBUILD}"
[ -n "$FORCE_VALUE" ] || FORCE_VALUE="$FORCE_REBUILD"
case "$FORCE_VALUE" in
true|TRUE|True|1|yes|YES) FORCE_ON=1 ;;
*) FORCE_ON="" ;;
esac
if [ -n "$FORCE_ON" ]; then
echo "FORCE_REBUILD=$FORCE_VALUE, ignoring artifacts already in the bucket"
else else
for VERSION in $(cat r-versions-to-build.txt); do for VERSION in $(cat r-versions-to-build.txt); do
URL="$BASE/$(printf "$FMT" "$VERSION" "${ARCH_ID}")" URL="$BASE/$(printf "$FMT" "$VERSION" "${ARCH_ID}")"
@ -209,7 +190,6 @@ steps:
DOCKER_HOST: tcp://docker:2375 DOCKER_HOST: tcp://docker:2375
commands: | commands: |
ip link set dev eth0 mtu 1280 2>/dev/null || true ip link set dev eth0 mtu 1280 2>/dev/null || true
# docker info
TO_BUILD="" TO_BUILD=""
for VERSION in $(cat r-versions-to-build.txt); do for VERSION in $(cat r-versions-to-build.txt); do
if grep -qxF "$VERSION" r-versions-existing.txt; then if grep -qxF "$VERSION" r-versions-existing.txt; then
@ -255,7 +235,11 @@ steps:
from_secret: B2_S3_ACCESS_KEY_ID from_secret: B2_S3_ACCESS_KEY_ID
secret_key: secret_key:
from_secret: B2_S3_SECRET_KEY from_secret: B2_S3_SECRET_KEY
overwrite: false # Must stay true for FORCE_REBUILD to mean anything: with overwrite=false
# the plugin logs "File already exists, skipping upload" and still exits
# green, so a rebuild would be discarded. Normal runs are unaffected,
# since dedup means only not-yet-published versions are ever built.
overwrite: true
volumes: volumes:
- ci-tmp-rwx:/tmp # agent on k8s - ci-tmp-rwx:/tmp # agent on k8s
# - /tmp:/tmp # agent on VM # - /tmp:/tmp # agent on VM