Commit graph build-cran-binaries/docker/build-one.Dockerfile
Author SHA1 Message Date
f3c18b6805
refactor: migrate package installation from pak to uvr
bincraft dropped pak in favour of uvr, so the pipelines, helper scripts
and images in this repo move with it.

- add local/uvr-install.sh as the single replacement for pak::pak(); it
  bootstraps a pinned uvr, mints a throwaway project under TMPDIR and
  runs `uvr add --no-install` + `uvr sync --library`, because `uvr add`
  refuses to run outside a project and only `sync` honours --library
- install bincraft via its Forgejo spec (forgejo::codefloe.com/rpkgs/
  bincraft@<tag>) instead of a git:: URL, keeping the git ls-remote tag
  resolution
- pass UVR_R_BIN/UVR_TARGET_LIB from install-bincraft.R so the
  per-R-minor passes target their own R and library
- replace R_PKG_CACHE_DIR with UVR_CACHE_DIR/UVR_PACKAGES_DIR on the
  persistent volume, preserving the amd64-off/arm64-on split
- drop trim_pkgcache_metadata() and its test; uvr's cache does not grow
  the way pkgcache's _metadata dir did
- let uvr install system requirements from its vendored
  r-system-requirements rules, replacing pak::sysreqs_db_update()
- migrate the shiny app image, the alpine reprex and the CRAN loading
  test; ship uvr-install.sh in the build-one image
- track the uvr pin with renovate
2026-07-31 12:16:45 +00:00
1ad7a6bfe9 chore: resolve latest bincraft release dynamically (no hardcoded pins) (#107)
Some checks failed
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline failed
ci/crow/cron/process-updates/9 Pipeline failed
ci/crow/cron/process-updates/7 Pipeline failed
ci/crow/cron/process-updates/8 Pipeline failed
ci/crow/manual/build-all-versions-install-deps/2 Pipeline was successful
ci/crow/manual/build-all-versions/5 Pipeline failed
ci/crow/manual/build-all-versions/7 Pipeline failed
ci/crow/manual/build-all-versions/6 Pipeline failed
ci/crow/manual/build-all-versions/8 Pipeline was canceled
## Summary

Stop hardcoding the bincraft version. Every `.crow` workflow and the build-one image pinned `@vX.Y.Z` (and a `packageVersion() != "X.Y.Z"` guard), so each bincraft release meant editing the version in ~8 places — and it was easy to miss one (the Dockerfile lagged at v4.2.1; v4.4.1 shipped without the empty-env fix because of exactly this churn).

## Change

New `local/install-bincraft.R` resolves the **latest release tag dynamically**:

- `git ls-remote --tags` on the public repo (no token),
- keep `vX.Y.Z` tags, pick the highest version (filtered/sorted in R for portability, not via git `--sort`/refspec which behaved inconsistently under `system2()`),
- `pak::pak("git::…@<latest>")` — idempotent on the git ref, so re-runs keep the package unless a newer tag exists.

All call sites now invoke the helper instead of a pinned version:

- `.crow/build-all-versions.yaml` (primary + per-minor pass)
- `.crow/build-all-versions-install-deps.yaml`
- `.crow/process-updates.yaml` (primary + per-minor pass)
- `.crow/weekly-rebuild-missing.yaml`
- `.crow/archive-missed-packages.yaml`
- `docker/build-one.Dockerfile` (ships the helper into the image; `ensure_bincraft` sources it)

## Effect

Tag a new bincraft release → the next CI run / `just rebuild` picks it up automatically. No more pin edits, and no more "forgot to bump the Dockerfile" drift.

## Verified

- Resolver returns the current latest tag (`v4.4.2`) via `git ls-remote` + R-side version sort.
- All five workflow YAMLs parse; helper R parses; air/editorconfig clean.

Note: this tracks the latest **tag**, so cutting a release is still the deliberate gate — CI won't pick up un-tagged main.
Reviewed-on: #107
2026-07-01 08:10:02 +00:00
b4ec6291a9
chore: bump bincraft to 4.4.2
All checks were successful
ci/crow/cron/weekly-audit-missing/15 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/13 Pipeline was successful
ci/crow/cron/weekly-audit-missing/5 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/3 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
2026-06-30 16:09:44 +02:00
d496029a79
chore: bump bincraft to 4.4.1
All checks were successful
ci/crow/cron/process-updates/7 Pipeline was successful
2026-06-30 15:51:07 +02:00
b93d1ef6bd chore: silence otelsdk warnings across build paths (#105)
All checks were successful
ci/crow/cron/process-updates/1 Pipeline was successful
## Summary

Silences the recurring `OpenTelemetry error: there is no package called 'otelsdk'` warnings during builds.

## Root cause

The `build-env-*` images configure an OTel exporter (traces/logs/metrics), but `otelsdk` (the R OTel SDK backend) isn't installed. pak's `otel` instrumentation therefore tries to load `otelsdk` on every run and logs the error, falling back to a no-op.

`OTEL_SDK_DISABLED=true` (already set in `build-one.Dockerfile`) does **not** help — the R `otel` package ignores it and gates purely on `OTEL_R_<SIGNAL>_EXPORTER` (then the standard `OTEL_<SIGNAL>_EXPORTER`). When that resolves to a real exporter (`otlp`/`http`/…) with no SDK present, you get the error.

## Fix

Set `OTEL_R_TRACES_EXPORTER`, `OTEL_R_LOGS_EXPORTER`, and `OTEL_R_METRICS_EXPORTER` to `none` so the R otel providers are clean no-ops:

- `docker/build-one.Dockerfile` — exported alongside the existing OTel var.
- `.crow/process-updates.yaml`, `weekly-rebuild-missing.yaml`, `build-all-versions.yaml`, `build-all-versions-install-deps.yaml` — added to each step's `environment` block.

Using the R-specific variables (not the standard `OTEL_*_EXPORTER`) keeps OTel intact for any non-R tooling in the images.

Reviewed-on: #105
2026-06-30 12:41:32 +00:00
b744ae1bb1 fix: build-one image bincraft v4.4.0 + ship patch registry (#104)
## Summary

Fixes the **targeted rebuild** path (`just rebuild` → `docker/build-one.Dockerfile`) so it actually applies patches.

Two problems, both of which would make `build-one.R`'s `patches = "local/patches"` a silent no-op or a hard error:

1. **Stale bincraft pin.** The Dockerfile pinned bincraft `v4.2.1`, which predates the `patches` argument — so `build_binary_package(patches = ...)` would fail with `unused argument`. Bumped to **v4.4.0** to match the `.crow` workflows.
2. **Registry not shipped into the image.** Only `build-one.R` was copied in; `local/patches/registry.json` was absent, so `patches = "local/patches"` resolved to a nonexistent `/work/local/patches` and silently applied nothing. Added `COPY patches /work/local/patches` (build context is `local/`, CWD is `/work`).

## Why

This is the path used to verify the patching end-to-end, e.g.:

```bash
just rebuild alpine 3.23 amd64 rts2 1.0.3
```

rts2 depends on RcppParallel; with this fix the container installs bincraft v4.4.0, ships the registry, and the patched RcppParallel binary (`RCPP_PARALLEL_USE_TBB=0`) is served to pak during dependency install. Expect `Applying patch to RcppParallel …` in the log, RcppParallel installed as a binary (no recompile), then rts2 building and uploading.

Reviewed-on: #104
2026-06-30 12:15:39 +00:00
7e01f481c0
perf(local): skip per-minor passes for non-r-minor-sensitive packages
A package's ABI sensitivity is the same across R minors, so for a non-sensitive
package the per-minor loop only ran `ensure_bincraft` (a ~14s bincraft/cranlike
install per minor) before build-one.R classified and skipped it. The primary
pass now writes a .r_minor_sensitive sentinel when it builds a sensitive
package, and the wrapper gates the whole per-minor loop (installs included) on
that file — non-sensitive rebuilds no longer touch other minors' libraries.
2026-06-14 14:50:40 +02:00
3ae4672d66
feat(local): just rebuild builds across all installed R minors
- Dockerfile: install bincraft into each R minor, run a primary pass plus a
  sensitive-only pass under every other /opt/R/[0-9]* minor (deduped by minor),
  probe/skip xvfb, set GIT_TERMINAL_PROMPT=0; extra-minor failures are non-fatal.
- build-one.R: add --sensitive-only mode, log + shallow-clone the ABI classify
  step, and clear cranlike's stale ./PACKAGES.db before each index refresh
  (workaround for the "table packages already exists" bug; pending cranlike fix).
2026-06-14 13:02:48 +02:00
1224bd74fe feat(local): just rebuild recipe for targeted builds via remote buildx (#87)
## Summary

Adds a local `just rebuild` recipe to (re)build specific versions of a single package on a given OS/arch, dispatching to a remote buildx builder (the build runs there, not locally).

- `just rebuild <os> <tag> <arch> <package> <version>...` → `docker buildx build --builder <artemis|gaia> --platform linux/<arch> …` (amd64→artemis, arm64→gaia; names + `R_VERSION` env-overridable).
- `docker/build-one.Dockerfile` runs `build-one.R` as a secret-mounted `RUN`, built `--no-cache --output type=cacheonly` (pure side-effect: the S3 upload; no image kept).
- `local/build-one.R` auto-classifies each version via the ABI classifier (risky → per-minor slot `contrib/<x.y>/`, else generic), force-rebuilds + uploads + stores metadata, then refreshes the touched slot's `PACKAGES` index.

## Prerequisites

- buildx builders named `artemis` (amd64) and `gaia` (arm64) registered (`docker buildx create --name artemis ssh://…`).
- Exported secrets: `B2_S3_ACCESS_KEY`, `B2_S3_SECRET_KEY`, `PGPASS` (`GITHUB_PAT` optional).
- bincraft `v4.2.0` tag must exist (the build installs `@v4.2.0` and uses its classifier + per-minor index API).

Reviewed-on: #87
2026-06-14 09:03:45 +00:00