Compare commits

..
Author SHA1 Message Date
12aa310b8d
fix(ci): detect existing R versions via public object HEAD probe
The existing-version check listed the bucket with an anonymous curl, but
Backblaze B2 requires authentication for the list-bucket API, so it returned
AccessDenied and r-versions-existing.txt was always empty -> every version
was rebuilt despite already existing in s3. Signing the request is not
workable here: the B2 credentials are scoped to the plugin-s3 image in crow
and are not injected into this plain alpine step.

Individual artifacts are public-read, so probe each expected object URL with
an anonymous HEAD instead. The filename mirrors what nfpm produces (version
and release pinned to 1): r-<ver>_1_<arch>.{apk,deb} for alpine/ubuntu and
R-<ver>-1-1.<arch>.rpm for el*. Verified against the live bucket for all
three packager families.

Shell variables use bare $name rather than ${name}: crow runs its own
${VAR} substitution over the commands before the shell executes, so a
${VAR} naming a shell var (BASE, VERSION) was blanked to empty -- only
matrix vars resolve at that stage. Build the filename with printf to avoid
the ${VERSION}_1 brace requirement entirely.
2026-06-29 14:14:27 +02:00
13 changed files with 58 additions and 189 deletions

View file

@ -1,18 +1,6 @@
when:
event: [cron, manual]
variables:
FORCE_REBUILD:
description: >-
Rebuild every targeted R version even when the artifact is already in the
bucket. Needed after a builder-image change that alters the resulting R
binary, because artifact filenames carry only the R version and would
otherwise dedup to the stale build forever.
options:
- "false"
- "true"
default: "false"
matrix:
include:
- PLATFORM: rhel-9
@ -94,6 +82,46 @@ matrix:
ARCH_ID: x86_64
steps:
# Ensures that we're always using the latest available upstream build chain
# - name: Pull upstream
# image: reg.devxy.io/docker.io/library/alpine:3.20
# commands:
# - apk add -q --no-cache git
# - git remote add upstream https://github.com/cynkra/r-builds.git
# - git config --global user.email "you@example.com"
# - git config --global user.name "Your Name"
# - git config pull.ff only
# - git fetch upstream
# - git merge upstream/main --allow-unrelated-histories
#.
- name: Net debug
image: reg.devxy.io/docker.io/library/alpine:3.23
privileged: true
commands:
- ip link set dev eth0 mtu 1280 2>/dev/null || true
- echo "=== pod interface ==="
- cat /sys/class/net/eth0/mtu
- ip -o link show eth0
- echo "=== route ==="
- ip route
- echo "=== resolv.conf ==="
- cat /etc/resolv.conf
- for i in 1 2 3; do apk add -q --no-cache iputils && break; sleep 5; done || echo "apk failed; falling back to busybox ping (no -M do)"
- |
echo "=== PMTUD probes (DF=1) to dl-cdn.alpinelinux.org ==="
for sz in 1472 1392 1352 1252 1200; do
echo "--- payload=$sz (wire ~$((sz+28))) ---"
ping -c 2 -W 3 -M do -s $sz dl-cdn.alpinelinux.org || echo "(size=$sz failed)"
done
- ping -c 3 -W 5 dl-cdn.alpinelinux.org || true
- wget --tries=1 --timeout=10 -O /dev/null -nv https://dl-cdn.alpinelinux.org/ 2>&1 || true
backend_options:
kubernetes:
nodeSelector:
kubernetes.io/arch: "${ARCH}"
- name: Get R versions to build
image: reg.devxy.io/docker.io/library/alpine:3.23
privileged: true
@ -150,31 +178,12 @@ steps:
esac
BASE="https://s3.eu-central-003.backblazeb2.com/devxy-r-builds/${PLATFORM_ID}"
: > r-versions-existing.txt
# FORCE_REBUILD is declared as a top-level variable, so the UI offers it
# as a dropdown on a manual run. The two trigger paths deliver it
# differently, so accept either:
# UI, or crow pipeline create --var FORCE_REBUILD=true
# -> a substitution variable, expanded into ${FORCE_REBUILD} below
# crow pipeline start -p FORCE_REBUILD=true
# -> injected into the step environment, read as bare $FORCE_REBUILD
# The declared default is the string "false", so test the value rather
# than emptiness; an empty value means neither path supplied one.
FORCE_VALUE="${FORCE_REBUILD}"
[ -n "$FORCE_VALUE" ] || FORCE_VALUE="$FORCE_REBUILD"
case "$FORCE_VALUE" in
true|TRUE|True|1|yes|YES) FORCE_ON=1 ;;
*) FORCE_ON="" ;;
esac
if [ -n "$FORCE_ON" ]; then
echo "FORCE_REBUILD=$FORCE_VALUE, ignoring artifacts already in the bucket"
else
for VERSION in $(cat r-versions-to-build.txt); do
URL="$BASE/$(printf "$FMT" "$VERSION" "${ARCH_ID}")"
if [ "$(curl -s -o /dev/null -w '%{http_code}' -I "$URL")" = "200" ]; then
echo "$VERSION" >> r-versions-existing.txt
fi
done
fi
for VERSION in $(cat r-versions-to-build.txt); do
URL="$BASE/$(printf "$FMT" "$VERSION" "${ARCH_ID}")"
if [ "$(curl -s -o /dev/null -w '%{http_code}' -I "$URL")" = "200" ]; then
echo "$VERSION" >> r-versions-existing.txt
fi
done
- echo "Already present in s3:"; cat r-versions-existing.txt
backend_options:
kubernetes:
@ -190,6 +199,7 @@ steps:
DOCKER_HOST: tcp://docker:2375
commands: |
ip link set dev eth0 mtu 1280 2>/dev/null || true
# docker info
TO_BUILD=""
for VERSION in $(cat r-versions-to-build.txt); do
if grep -qxF "$VERSION" r-versions-existing.txt; then
@ -235,11 +245,7 @@ steps:
from_secret: B2_S3_ACCESS_KEY_ID
secret_key:
from_secret: B2_S3_SECRET_KEY
# Must stay true for FORCE_REBUILD to mean anything: with overwrite=false
# the plugin logs "File already exists, skipping upload" and still exits
# green, so a rebuild would be discarded. Normal runs are unaffected,
# since dedup means only not-yet-published versions are ever built.
overwrite: true
overwrite: false
volumes:
- ci-tmp-rwx:/tmp # agent on k8s
# - /tmp:/tmp # agent on VM

View file

@ -1,20 +0,0 @@
# CLAUDE.md
CI pipelines (Crow, in `.crow/`) that build R binaries and upload them to Backblaze B2.
## Conventions
- **PRs:** the remote is Forgejo on `git.devxy.io`; use `fj -H git.devxy.io` (not `gh`).
- **Storage:** Backblaze B2 bucket `devxy-r-builds` (endpoint configured in `.crow/build.yaml`).
## Gotchas
- **B2 requires authentication for the list-bucket API.** Anonymous GET only works for individual public-read objects — an empty bucket listing usually means missing credentials, not an empty bucket. This silently broke rebuild dedup once.
- **R's configure silently disables graphics devices when headers are missing.**
It never fails the build, so a missing `-dev` package yields a working R whose `capabilities()` reports `cairo`, `png`, `jpeg` and `tiff` as `FALSE`.
Installing the runtime libraries on the target system afterwards cannot bring compiled-out devices back; the binary has to be rebuilt.
`builder/build.sh` therefore asserts `REQUIRED_CAPABILITIES` (set per Dockerfile) after `make install`.
- **The `--with-x=no` on the Alpine images does not disable cairo.** R's bitmap and pangocairo checks run independently of `R_X11`, so `png()` and `svg()` work headless via cairo.
- **Artifact filenames carry only the R version, not the builder image.**
When a builder change alters the resulting binary, dedup would keep serving the stale artifact forever; set `FORCE_REBUILD` in `.crow/build.yaml` to rebuild and overwrite in place.
- When a pipeline fails, fetch the Crow logs yourself instead of asking the user to paste them, and monitor reruns in the background until green.

View file

@ -5,13 +5,6 @@ ENV OS_IDENTIFIER alpine-320
RUN set -x \
&& apk add -q R-dev curl ca-certificates bash g++ tzdata openjdk17 texmf-dist texlive-full tar sed patch tcl tk tk-dev xvfb libdeflate libdeflate-dev
# Graphics devices. R's configure only enables the cairo/png/jpeg/tiff devices
# when the corresponding headers are present at compile time, and pkgconf must
# exist for the pangocairo check to run at all. Without these, capabilities()
# reports cairo/png/jpeg/tiff as FALSE, and installing the runtime libraries
# afterwards cannot bring the compiled-out devices back.
RUN apk add -q pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev
# Install s5cmd for S3 uploads (much faster than AWS CLI)
RUN ARCH=$(uname -m); if [ "$ARCH" = "x86_64" ]; then S5CMD_ARCH="64bit"; else S5CMD_ARCH="arm64"; fi && \
curl -sL "https://github.com/peak/s5cmd/releases/download/v2.3.0/s5cmd_2.3.0_Linux-${S5CMD_ARCH}.tar.gz" | tar xz -C /usr/local/bin s5cmd
@ -25,12 +18,7 @@ RUN chmod 0777 /opt
# Override the default pager used by R
ENV PAGER /usr/bin/pager
# X11 stays off: these are headless images, and the cairo/bitmap devices are
# detected independently of X11, so png()/svg() still work via cairo.
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-cairo --with-libpng --with-jpeglib --with-libtiff --with-blas --with-lapack"
# Fail the build if a graphics device silently drops out again.
ENV REQUIRED_CAPABILITIES "cairo,png,jpeg,tiff"
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-blas --with-lapack"
COPY package.alpine-320 /package.sh
COPY build.sh .

View file

@ -5,13 +5,6 @@ ENV OS_IDENTIFIER alpine-321
RUN set -x \
&& apk add -q R-dev curl ca-certificates bash g++ tzdata openjdk17 texmf-dist texlive-full tar sed patch tcl tk tk-dev xvfb libdeflate libdeflate-dev
# Graphics devices. R's configure only enables the cairo/png/jpeg/tiff devices
# when the corresponding headers are present at compile time, and pkgconf must
# exist for the pangocairo check to run at all. Without these, capabilities()
# reports cairo/png/jpeg/tiff as FALSE, and installing the runtime libraries
# afterwards cannot bring the compiled-out devices back.
RUN apk add -q pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev
# Install s5cmd for S3 uploads (much faster than AWS CLI)
RUN ARCH=$(uname -m); if [ "$ARCH" = "x86_64" ]; then S5CMD_ARCH="64bit"; else S5CMD_ARCH="arm64"; fi && \
curl -sL "https://github.com/peak/s5cmd/releases/download/v2.3.0/s5cmd_2.3.0_Linux-${S5CMD_ARCH}.tar.gz" | tar xz -C /usr/local/bin s5cmd
@ -25,12 +18,7 @@ RUN chmod 0777 /opt
# Override the default pager used by R
ENV PAGER /usr/bin/pager
# X11 stays off: these are headless images, and the cairo/bitmap devices are
# detected independently of X11, so png()/svg() still work via cairo.
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-cairo --with-libpng --with-jpeglib --with-libtiff --with-blas --with-lapack"
# Fail the build if a graphics device silently drops out again.
ENV REQUIRED_CAPABILITIES "cairo,png,jpeg,tiff"
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-blas --with-lapack"
COPY package.alpine-321 /package.sh
COPY build.sh .

View file

@ -5,13 +5,6 @@ ENV OS_IDENTIFIER alpine-322
RUN set -x \
&& apk add -q R-dev curl ca-certificates bash g++ tzdata openjdk17 texmf-dist texlive-full tar sed patch tcl tk tk-dev xvfb libdeflate libdeflate-dev
# Graphics devices. R's configure only enables the cairo/png/jpeg/tiff devices
# when the corresponding headers are present at compile time, and pkgconf must
# exist for the pangocairo check to run at all. Without these, capabilities()
# reports cairo/png/jpeg/tiff as FALSE, and installing the runtime libraries
# afterwards cannot bring the compiled-out devices back.
RUN apk add -q pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev
# Install s5cmd for S3 uploads (much faster than AWS CLI)
RUN ARCH=$(uname -m); if [ "$ARCH" = "x86_64" ]; then S5CMD_ARCH="64bit"; else S5CMD_ARCH="arm64"; fi && \
curl -sL "https://github.com/peak/s5cmd/releases/download/v2.3.0/s5cmd_2.3.0_Linux-${S5CMD_ARCH}.tar.gz" | tar xz -C /usr/local/bin s5cmd
@ -25,12 +18,7 @@ RUN chmod 0777 /opt
# Override the default pager used by R
ENV PAGER /usr/bin/pager
# X11 stays off: these are headless images, and the cairo/bitmap devices are
# detected independently of X11, so png()/svg() still work via cairo.
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-cairo --with-libpng --with-jpeglib --with-libtiff --with-blas --with-lapack"
# Fail the build if a graphics device silently drops out again.
ENV REQUIRED_CAPABILITIES "cairo,png,jpeg,tiff"
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-blas --with-lapack"
COPY package.alpine-322 /package.sh
COPY build.sh .

View file

@ -5,13 +5,6 @@ ENV OS_IDENTIFIER alpine-323
RUN set -x \
&& apk add -q R-dev curl ca-certificates bash g++ tzdata openjdk17 texmf-dist texlive-full tar sed patch tcl tk tk-dev xvfb libdeflate libdeflate-dev
# Graphics devices. R's configure only enables the cairo/png/jpeg/tiff devices
# when the corresponding headers are present at compile time, and pkgconf must
# exist for the pangocairo check to run at all. Without these, capabilities()
# reports cairo/png/jpeg/tiff as FALSE, and installing the runtime libraries
# afterwards cannot bring the compiled-out devices back.
RUN apk add -q pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev
# Install s5cmd for S3 uploads (much faster than AWS CLI)
RUN ARCH=$(uname -m); if [ "$ARCH" = "x86_64" ]; then S5CMD_ARCH="64bit"; else S5CMD_ARCH="arm64"; fi && \
curl -sL "https://github.com/peak/s5cmd/releases/download/v2.3.0/s5cmd_2.3.0_Linux-${S5CMD_ARCH}.tar.gz" | tar xz -C /usr/local/bin s5cmd
@ -25,12 +18,7 @@ RUN chmod 0777 /opt
# Override the default pager used by R
ENV PAGER /usr/bin/pager
# X11 stays off: these are headless images, and the cairo/bitmap devices are
# detected independently of X11, so png()/svg() still work via cairo.
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-cairo --with-libpng --with-jpeglib --with-libtiff --with-blas --with-lapack"
# Fail the build if a graphics device silently drops out again.
ENV REQUIRED_CAPABILITIES "cairo,png,jpeg,tiff"
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-blas --with-lapack"
COPY package.alpine-322 /package.sh
COPY build.sh .

View file

@ -5,13 +5,6 @@ ENV OS_IDENTIFIER alpine-324
RUN set -x \
&& apk add -q R-dev curl ca-certificates bash g++ tzdata openjdk17 texmf-dist texlive-full tar sed patch tcl tk tk-dev xvfb libdeflate libdeflate-dev
# Graphics devices. R's configure only enables the cairo/png/jpeg/tiff devices
# when the corresponding headers are present at compile time, and pkgconf must
# exist for the pangocairo check to run at all. Without these, capabilities()
# reports cairo/png/jpeg/tiff as FALSE, and installing the runtime libraries
# afterwards cannot bring the compiled-out devices back.
RUN apk add -q pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev
# Install s5cmd for S3 uploads (much faster than AWS CLI)
RUN ARCH=$(uname -m); if [ "$ARCH" = "x86_64" ]; then S5CMD_ARCH="64bit"; else S5CMD_ARCH="arm64"; fi && \
curl -sL "https://github.com/peak/s5cmd/releases/download/v2.3.0/s5cmd_2.3.0_Linux-${S5CMD_ARCH}.tar.gz" | tar xz -C /usr/local/bin s5cmd
@ -25,12 +18,7 @@ RUN chmod 0777 /opt
# Override the default pager used by R
ENV PAGER /usr/bin/pager
# X11 stays off: these are headless images, and the cairo/bitmap devices are
# detected independently of X11, so png()/svg() still work via cairo.
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-cairo --with-libpng --with-jpeglib --with-libtiff --with-blas --with-lapack"
# Fail the build if a graphics device silently drops out again.
ENV REQUIRED_CAPABILITIES "cairo,png,jpeg,tiff"
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-blas --with-lapack"
COPY package.alpine-324 /package.sh
COPY build.sh .

View file

@ -209,47 +209,6 @@ local({
EOF
}
# verify_capabilities()
# Guard against a graphics device being silently compiled out. R's configure
# only enables the cairo/png/jpeg/tiff devices when the matching headers were
# present at compile time, and it does not fail when they are missing, so a
# missing -dev package produces a working R that simply cannot draw. Once that
# tarball is published, installing the runtime libraries on the target system
# cannot bring the devices back; the build has to be redone.
#
# Dockerfiles opt in by setting REQUIRED_CAPABILITIES to a comma-separated list
# of capabilities() names, e.g. "cairo,png,jpeg,tiff".
verify_capabilities() {
if [ -z "${REQUIRED_CAPABILITIES}" ]; then
return 0
fi
echo "Verifying required R capabilities: ${REQUIRED_CAPABILITIES}"
# The full capabilities() table goes to stderr so it shows up in the build
# log; stdout carries only the space-separated list of missing entries.
local missing
missing=$("${R_INSTALL_PATH}/bin/Rscript" --vanilla -e '
required <- trimws(strsplit(Sys.getenv("REQUIRED_CAPABILITIES"), ",")[[1]])
required <- required[nzchar(required)]
caps <- capabilities()
writeLines(capture.output(print(caps)), con = stderr())
ok <- (required %in% names(caps)) & (unname(caps[required]) %in% TRUE)
cat(paste(required[!ok], collapse = " "), "\n", sep = "")
') || {
echo "ERROR: could not query capabilities() from the freshly built R" >&2
return 1
}
if [ -n "${missing}" ]; then
echo "ERROR: R ${R_VERSION} for ${OS_IDENTIFIER} is missing required capabilities: ${missing}" >&2
echo "ERROR: install the matching -dev packages in builder/Dockerfile.${OS_IDENTIFIER} and rebuild." >&2
return 1
fi
echo "All required capabilities are present."
}
# check for packager script
## If it exists this build is ready for packaging with nFPM, so run the script
## else do nothing
@ -277,7 +236,6 @@ set_up_environment
fetch_r_source $R_VERSION
patch_r $R_VERSION
compile_r $R_VERSION
verify_capabilities
package_r $R_VERSION
archive_r
upload_r $R_VERSION

View file

@ -42,10 +42,7 @@ ${deflate_libs}
# - libgomp1
- icu-dev
- jpeg-dev
# liblzma-dev is the Debian name and does not exist on Alpine; xz-dev is the
# equivalent and pulls xz-libs, which owns liblzma.so.5. Without it libR.so
# cannot relocate its lzma symbols and R fails to start at all.
- xz-dev
# - liblzma-dev
- openblas-dev
- pango
# - libpangocairo-1.0-0

View file

@ -42,10 +42,7 @@ ${deflate_libs}
# - libgomp1
- icu-dev
- jpeg-dev
# liblzma-dev is the Debian name and does not exist on Alpine; xz-dev is the
# equivalent and pulls xz-libs, which owns liblzma.so.5. Without it libR.so
# cannot relocate its lzma symbols and R fails to start at all.
- xz-dev
# - liblzma-dev
- openblas-dev
- pango
# - libpangocairo-1.0-0

View file

@ -42,10 +42,7 @@ ${deflate_libs}
# - libgomp1
- icu-dev
- jpeg-dev
# liblzma-dev is the Debian name and does not exist on Alpine; xz-dev is the
# equivalent and pulls xz-libs, which owns liblzma.so.5. Without it libR.so
# cannot relocate its lzma symbols and R fails to start at all.
- xz-dev
# - liblzma-dev
- openblas-dev
- pango
# - libpangocairo-1.0-0

View file

@ -42,10 +42,7 @@ ${deflate_libs}
# - libgomp1
- icu-dev
- jpeg-dev
# liblzma-dev is the Debian name and does not exist on Alpine; xz-dev is the
# equivalent and pulls xz-libs, which owns liblzma.so.5. Without it libR.so
# cannot relocate its lzma symbols and R fails to start at all.
- xz-dev
# - liblzma-dev
- openblas-dev
- pango
# - libpangocairo-1.0-0

View file

@ -42,10 +42,7 @@ ${deflate_libs}
# - libgomp1
- icu-dev
- jpeg-dev
# liblzma-dev is the Debian name and does not exist on Alpine; xz-dev is the
# equivalent and pulls xz-libs, which owns liblzma.so.5. Without it libR.so
# cannot relocate its lzma symbols and R fails to start at all.
- xz-dev
# - liblzma-dev
- openblas-dev
- pango
# - libpangocairo-1.0-0