fix(alpine): build R with cairo, png, jpeg and tiff graphics devices (#9)
## Motivation
R on the Alpine images was built without graphics support: `capabilities()` reports `cairo`, `png`, `jpeg` and `tiff` as `FALSE`, while Ubuntu reports all four `TRUE`.
The images install `R-dev` but never the graphics headers, and R's `configure` treats those as optional and does not fail when they are absent, so the build succeeds and ships an R that cannot draw.
Reproduced in a clean `alpine:3.22` container with the old package set:
```
Additional capabilities: PNG, NLS, ICU
Capabilities skipped: JPEG, TIFF, cairo
```
One correction to the original diagnosis: `--with-x=no` was not the cause, and libpng was in fact found.
R's bitmap and `pangocairo` checks run independently of `R_X11` (`configure.ac:2604` gates on `want_cairo`, not `use_X11`).
`capabilities("png")` needs a device *backend*, and with X11 off and cairo absent there was none, which is why all four read `FALSE` despite configure listing PNG.
So the fix is purely the missing `-dev` packages.
X11 stays off, which is correct for headless images, and cairo covers `png()` / `jpeg()` / `tiff()` / `svg()`.
## Changes
- `builder/Dockerfile.alpine-{320,321,322,323,324}` — install `pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev`; pass `--with-cairo --with-libpng --with-jpeglib --with-libtiff` explicitly so the intent sits next to `--with-x=no`; set `REQUIRED_CAPABILITIES`.
- `builder/build.sh` — add `verify_capabilities()` between `compile_r` and `package_r`. A build that loses a graphics device now fails instead of being packaged and uploaded. Opt-in via `REQUIRED_CAPABILITIES`, so every non-Alpine platform is untouched.
- `.crow/build.yaml` — add a `FORCE_REBUILD` escape hatch to the dedup step. Artifact filenames carry only the R version, so the HEAD probe would otherwise keep skipping every Alpine version and serving the stale broken binaries.
- `CLAUDE.md` — record the three gotchas.
## Verification
Compiled R 4.5.1 in the patched `alpine-322` image, through the real `build.sh`:
```
jpeg png tiff tcltk X11 aqua
TRUE TRUE TRUE TRUE FALSE FALSE
NLS Rprof profmem cairo ICU long.double
TRUE TRUE TRUE TRUE TRUE TRUE
All required capabilities are present.
```
The devices actually render, not just report `TRUE` (`default bitmapType: cairo`):
```
png -> t.png (2154 bytes)
jpeg -> t.jpeg (4489 bytes)
tiff -> t.tiff (120160 bytes)
svg -> t.svg (9381 bytes)
```
The guard was exercised in both directions against that real build: it passes on the four graphics capabilities, and returns non-zero naming the missing entry when additionally asked for `X11`.
`crow lint .crow/` passes and `bash -n builder/build.sh` is clean.
## Behaviour change
Alpine R binaries gain the cairo, png, jpeg and tiff devices, and link against `libcairo`, `libpangocairo`, `libpng`, `libjpeg` and `libtiff`.
The `.apk` metadata already declared those runtime dependencies, so consumer images need no change.
Any Alpine build that loses a graphics device now fails the pipeline rather than uploading silently.
**Needs a follow-up after merge:** the first run must have `FORCE_REBUILD` set, otherwise dedup skips every existing Alpine version and the broken artifacts stay in the bucket.
Reviewed-on: #9
This commit is contained in:
Patrick Schratz
parent
56d26ef59c
commit
1362e0bd3a
8 changed files with 130 additions and 11 deletions
|
|
@ -178,12 +178,22 @@ steps:
|
|||
esac
|
||||
BASE="https://s3.eu-central-003.backblazeb2.com/devxy-r-builds/${PLATFORM_ID}"
|
||||
: > r-versions-existing.txt
|
||||
for VERSION in $(cat r-versions-to-build.txt); do
|
||||
URL="$BASE/$(printf "$FMT" "$VERSION" "${ARCH_ID}")"
|
||||
if [ "$(curl -s -o /dev/null -w '%{http_code}' -I "$URL")" = "200" ]; then
|
||||
echo "$VERSION" >> r-versions-existing.txt
|
||||
fi
|
||||
done
|
||||
# Escape hatch for when the builder image itself changes in a way that
|
||||
# alters the resulting R binary (a newly enabled graphics device, a
|
||||
# different BLAS). The artifact filename carries only the R version, so
|
||||
# dedup would otherwise keep the stale binaries forever. Set FORCE_REBUILD
|
||||
# to a non-empty value as a pipeline variable on a manual run, or hardcode
|
||||
# it here temporarily; uploads overwrite in place.
|
||||
if [ -n "$FORCE_REBUILD" ]; then
|
||||
echo "FORCE_REBUILD is set, ignoring artifacts already in the bucket"
|
||||
else
|
||||
for VERSION in $(cat r-versions-to-build.txt); do
|
||||
URL="$BASE/$(printf "$FMT" "$VERSION" "${ARCH_ID}")"
|
||||
if [ "$(curl -s -o /dev/null -w '%{http_code}' -I "$URL")" = "200" ]; then
|
||||
echo "$VERSION" >> r-versions-existing.txt
|
||||
fi
|
||||
done
|
||||
fi
|
||||
- echo "Already present in s3:"; cat r-versions-existing.txt
|
||||
backend_options:
|
||||
kubernetes:
|
||||
|
|
|
|||
|
|
@ -10,4 +10,11 @@ CI pipelines (Crow, in `.crow/`) that build R binaries and upload them to Backbl
|
|||
## Gotchas
|
||||
|
||||
- **B2 requires authentication for the list-bucket API.** Anonymous GET only works for individual public-read objects — an empty bucket listing usually means missing credentials, not an empty bucket. This silently broke rebuild dedup once.
|
||||
- **R's configure silently disables graphics devices when headers are missing.**
|
||||
It never fails the build, so a missing `-dev` package yields a working R whose `capabilities()` reports `cairo`, `png`, `jpeg` and `tiff` as `FALSE`.
|
||||
Installing the runtime libraries on the target system afterwards cannot bring compiled-out devices back; the binary has to be rebuilt.
|
||||
`builder/build.sh` therefore asserts `REQUIRED_CAPABILITIES` (set per Dockerfile) after `make install`.
|
||||
- **The `--with-x=no` on the Alpine images does not disable cairo.** R's bitmap and pangocairo checks run independently of `R_X11`, so `png()` and `svg()` work headless via cairo.
|
||||
- **Artifact filenames carry only the R version, not the builder image.**
|
||||
When a builder change alters the resulting binary, dedup would keep serving the stale artifact forever; set `FORCE_REBUILD` in `.crow/build.yaml` to rebuild and overwrite in place.
|
||||
- When a pipeline fails, fetch the Crow logs yourself instead of asking the user to paste them, and monitor reruns in the background until green.
|
||||
|
|
|
|||
|
|
@ -5,6 +5,13 @@ ENV OS_IDENTIFIER alpine-320
|
|||
RUN set -x \
|
||||
&& apk add -q R-dev curl ca-certificates bash g++ tzdata openjdk17 texmf-dist texlive-full tar sed patch tcl tk tk-dev xvfb libdeflate libdeflate-dev
|
||||
|
||||
# Graphics devices. R's configure only enables the cairo/png/jpeg/tiff devices
|
||||
# when the corresponding headers are present at compile time, and pkgconf must
|
||||
# exist for the pangocairo check to run at all. Without these, capabilities()
|
||||
# reports cairo/png/jpeg/tiff as FALSE, and installing the runtime libraries
|
||||
# afterwards cannot bring the compiled-out devices back.
|
||||
RUN apk add -q pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev
|
||||
|
||||
# Install s5cmd for S3 uploads (much faster than AWS CLI)
|
||||
RUN ARCH=$(uname -m); if [ "$ARCH" = "x86_64" ]; then S5CMD_ARCH="64bit"; else S5CMD_ARCH="arm64"; fi && \
|
||||
curl -sL "https://github.com/peak/s5cmd/releases/download/v2.3.0/s5cmd_2.3.0_Linux-${S5CMD_ARCH}.tar.gz" | tar xz -C /usr/local/bin s5cmd
|
||||
|
|
@ -18,7 +25,12 @@ RUN chmod 0777 /opt
|
|||
# Override the default pager used by R
|
||||
ENV PAGER /usr/bin/pager
|
||||
|
||||
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-blas --with-lapack"
|
||||
# X11 stays off: these are headless images, and the cairo/bitmap devices are
|
||||
# detected independently of X11, so png()/svg() still work via cairo.
|
||||
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-cairo --with-libpng --with-jpeglib --with-libtiff --with-blas --with-lapack"
|
||||
|
||||
# Fail the build if a graphics device silently drops out again.
|
||||
ENV REQUIRED_CAPABILITIES "cairo,png,jpeg,tiff"
|
||||
|
||||
COPY package.alpine-320 /package.sh
|
||||
COPY build.sh .
|
||||
|
|
|
|||
|
|
@ -5,6 +5,13 @@ ENV OS_IDENTIFIER alpine-321
|
|||
RUN set -x \
|
||||
&& apk add -q R-dev curl ca-certificates bash g++ tzdata openjdk17 texmf-dist texlive-full tar sed patch tcl tk tk-dev xvfb libdeflate libdeflate-dev
|
||||
|
||||
# Graphics devices. R's configure only enables the cairo/png/jpeg/tiff devices
|
||||
# when the corresponding headers are present at compile time, and pkgconf must
|
||||
# exist for the pangocairo check to run at all. Without these, capabilities()
|
||||
# reports cairo/png/jpeg/tiff as FALSE, and installing the runtime libraries
|
||||
# afterwards cannot bring the compiled-out devices back.
|
||||
RUN apk add -q pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev
|
||||
|
||||
# Install s5cmd for S3 uploads (much faster than AWS CLI)
|
||||
RUN ARCH=$(uname -m); if [ "$ARCH" = "x86_64" ]; then S5CMD_ARCH="64bit"; else S5CMD_ARCH="arm64"; fi && \
|
||||
curl -sL "https://github.com/peak/s5cmd/releases/download/v2.3.0/s5cmd_2.3.0_Linux-${S5CMD_ARCH}.tar.gz" | tar xz -C /usr/local/bin s5cmd
|
||||
|
|
@ -18,7 +25,12 @@ RUN chmod 0777 /opt
|
|||
# Override the default pager used by R
|
||||
ENV PAGER /usr/bin/pager
|
||||
|
||||
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-blas --with-lapack"
|
||||
# X11 stays off: these are headless images, and the cairo/bitmap devices are
|
||||
# detected independently of X11, so png()/svg() still work via cairo.
|
||||
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-cairo --with-libpng --with-jpeglib --with-libtiff --with-blas --with-lapack"
|
||||
|
||||
# Fail the build if a graphics device silently drops out again.
|
||||
ENV REQUIRED_CAPABILITIES "cairo,png,jpeg,tiff"
|
||||
|
||||
COPY package.alpine-321 /package.sh
|
||||
COPY build.sh .
|
||||
|
|
|
|||
|
|
@ -5,6 +5,13 @@ ENV OS_IDENTIFIER alpine-322
|
|||
RUN set -x \
|
||||
&& apk add -q R-dev curl ca-certificates bash g++ tzdata openjdk17 texmf-dist texlive-full tar sed patch tcl tk tk-dev xvfb libdeflate libdeflate-dev
|
||||
|
||||
# Graphics devices. R's configure only enables the cairo/png/jpeg/tiff devices
|
||||
# when the corresponding headers are present at compile time, and pkgconf must
|
||||
# exist for the pangocairo check to run at all. Without these, capabilities()
|
||||
# reports cairo/png/jpeg/tiff as FALSE, and installing the runtime libraries
|
||||
# afterwards cannot bring the compiled-out devices back.
|
||||
RUN apk add -q pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev
|
||||
|
||||
# Install s5cmd for S3 uploads (much faster than AWS CLI)
|
||||
RUN ARCH=$(uname -m); if [ "$ARCH" = "x86_64" ]; then S5CMD_ARCH="64bit"; else S5CMD_ARCH="arm64"; fi && \
|
||||
curl -sL "https://github.com/peak/s5cmd/releases/download/v2.3.0/s5cmd_2.3.0_Linux-${S5CMD_ARCH}.tar.gz" | tar xz -C /usr/local/bin s5cmd
|
||||
|
|
@ -18,7 +25,12 @@ RUN chmod 0777 /opt
|
|||
# Override the default pager used by R
|
||||
ENV PAGER /usr/bin/pager
|
||||
|
||||
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-blas --with-lapack"
|
||||
# X11 stays off: these are headless images, and the cairo/bitmap devices are
|
||||
# detected independently of X11, so png()/svg() still work via cairo.
|
||||
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-cairo --with-libpng --with-jpeglib --with-libtiff --with-blas --with-lapack"
|
||||
|
||||
# Fail the build if a graphics device silently drops out again.
|
||||
ENV REQUIRED_CAPABILITIES "cairo,png,jpeg,tiff"
|
||||
|
||||
COPY package.alpine-322 /package.sh
|
||||
COPY build.sh .
|
||||
|
|
|
|||
|
|
@ -5,6 +5,13 @@ ENV OS_IDENTIFIER alpine-323
|
|||
RUN set -x \
|
||||
&& apk add -q R-dev curl ca-certificates bash g++ tzdata openjdk17 texmf-dist texlive-full tar sed patch tcl tk tk-dev xvfb libdeflate libdeflate-dev
|
||||
|
||||
# Graphics devices. R's configure only enables the cairo/png/jpeg/tiff devices
|
||||
# when the corresponding headers are present at compile time, and pkgconf must
|
||||
# exist for the pangocairo check to run at all. Without these, capabilities()
|
||||
# reports cairo/png/jpeg/tiff as FALSE, and installing the runtime libraries
|
||||
# afterwards cannot bring the compiled-out devices back.
|
||||
RUN apk add -q pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev
|
||||
|
||||
# Install s5cmd for S3 uploads (much faster than AWS CLI)
|
||||
RUN ARCH=$(uname -m); if [ "$ARCH" = "x86_64" ]; then S5CMD_ARCH="64bit"; else S5CMD_ARCH="arm64"; fi && \
|
||||
curl -sL "https://github.com/peak/s5cmd/releases/download/v2.3.0/s5cmd_2.3.0_Linux-${S5CMD_ARCH}.tar.gz" | tar xz -C /usr/local/bin s5cmd
|
||||
|
|
@ -18,7 +25,12 @@ RUN chmod 0777 /opt
|
|||
# Override the default pager used by R
|
||||
ENV PAGER /usr/bin/pager
|
||||
|
||||
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-blas --with-lapack"
|
||||
# X11 stays off: these are headless images, and the cairo/bitmap devices are
|
||||
# detected independently of X11, so png()/svg() still work via cairo.
|
||||
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-cairo --with-libpng --with-jpeglib --with-libtiff --with-blas --with-lapack"
|
||||
|
||||
# Fail the build if a graphics device silently drops out again.
|
||||
ENV REQUIRED_CAPABILITIES "cairo,png,jpeg,tiff"
|
||||
|
||||
COPY package.alpine-322 /package.sh
|
||||
COPY build.sh .
|
||||
|
|
|
|||
|
|
@ -5,6 +5,13 @@ ENV OS_IDENTIFIER alpine-324
|
|||
RUN set -x \
|
||||
&& apk add -q R-dev curl ca-certificates bash g++ tzdata openjdk17 texmf-dist texlive-full tar sed patch tcl tk tk-dev xvfb libdeflate libdeflate-dev
|
||||
|
||||
# Graphics devices. R's configure only enables the cairo/png/jpeg/tiff devices
|
||||
# when the corresponding headers are present at compile time, and pkgconf must
|
||||
# exist for the pangocairo check to run at all. Without these, capabilities()
|
||||
# reports cairo/png/jpeg/tiff as FALSE, and installing the runtime libraries
|
||||
# afterwards cannot bring the compiled-out devices back.
|
||||
RUN apk add -q pkgconf cairo-dev pango-dev fontconfig-dev freetype-dev libpng-dev jpeg-dev tiff-dev zlib-dev
|
||||
|
||||
# Install s5cmd for S3 uploads (much faster than AWS CLI)
|
||||
RUN ARCH=$(uname -m); if [ "$ARCH" = "x86_64" ]; then S5CMD_ARCH="64bit"; else S5CMD_ARCH="arm64"; fi && \
|
||||
curl -sL "https://github.com/peak/s5cmd/releases/download/v2.3.0/s5cmd_2.3.0_Linux-${S5CMD_ARCH}.tar.gz" | tar xz -C /usr/local/bin s5cmd
|
||||
|
|
@ -18,7 +25,12 @@ RUN chmod 0777 /opt
|
|||
# Override the default pager used by R
|
||||
ENV PAGER /usr/bin/pager
|
||||
|
||||
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-blas --with-lapack"
|
||||
# X11 stays off: these are headless images, and the cairo/bitmap devices are
|
||||
# detected independently of X11, so png()/svg() still work via cairo.
|
||||
ENV CONFIGURE_OPTIONS "--enable-R-shlib --with-tcltk --enable-memory-profiling --with-x=no --with-cairo --with-libpng --with-jpeglib --with-libtiff --with-blas --with-lapack"
|
||||
|
||||
# Fail the build if a graphics device silently drops out again.
|
||||
ENV REQUIRED_CAPABILITIES "cairo,png,jpeg,tiff"
|
||||
|
||||
COPY package.alpine-324 /package.sh
|
||||
COPY build.sh .
|
||||
|
|
|
|||
|
|
@ -209,6 +209,47 @@ local({
|
|||
EOF
|
||||
}
|
||||
|
||||
# verify_capabilities()
|
||||
# Guard against a graphics device being silently compiled out. R's configure
|
||||
# only enables the cairo/png/jpeg/tiff devices when the matching headers were
|
||||
# present at compile time, and it does not fail when they are missing, so a
|
||||
# missing -dev package produces a working R that simply cannot draw. Once that
|
||||
# tarball is published, installing the runtime libraries on the target system
|
||||
# cannot bring the devices back; the build has to be redone.
|
||||
#
|
||||
# Dockerfiles opt in by setting REQUIRED_CAPABILITIES to a comma-separated list
|
||||
# of capabilities() names, e.g. "cairo,png,jpeg,tiff".
|
||||
verify_capabilities() {
|
||||
if [ -z "${REQUIRED_CAPABILITIES}" ]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "Verifying required R capabilities: ${REQUIRED_CAPABILITIES}"
|
||||
|
||||
# The full capabilities() table goes to stderr so it shows up in the build
|
||||
# log; stdout carries only the space-separated list of missing entries.
|
||||
local missing
|
||||
missing=$("${R_INSTALL_PATH}/bin/Rscript" --vanilla -e '
|
||||
required <- trimws(strsplit(Sys.getenv("REQUIRED_CAPABILITIES"), ",")[[1]])
|
||||
required <- required[nzchar(required)]
|
||||
caps <- capabilities()
|
||||
writeLines(capture.output(print(caps)), con = stderr())
|
||||
ok <- (required %in% names(caps)) & (unname(caps[required]) %in% TRUE)
|
||||
cat(paste(required[!ok], collapse = " "), "\n", sep = "")
|
||||
') || {
|
||||
echo "ERROR: could not query capabilities() from the freshly built R" >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
if [ -n "${missing}" ]; then
|
||||
echo "ERROR: R ${R_VERSION} for ${OS_IDENTIFIER} is missing required capabilities: ${missing}" >&2
|
||||
echo "ERROR: install the matching -dev packages in builder/Dockerfile.${OS_IDENTIFIER} and rebuild." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo "All required capabilities are present."
|
||||
}
|
||||
|
||||
# check for packager script
|
||||
## If it exists this build is ready for packaging with nFPM, so run the script
|
||||
## else do nothing
|
||||
|
|
@ -236,6 +277,7 @@ set_up_environment
|
|||
fetch_r_source $R_VERSION
|
||||
patch_r $R_VERSION
|
||||
compile_r $R_VERSION
|
||||
verify_capabilities
|
||||
package_r $R_VERSION
|
||||
archive_r
|
||||
upload_r $R_VERSION
|
||||
|
|
|
|||
Loading…
Reference in a new issue