Author	SHA1	Message	Date
pat-s	ee15c50f53	refactor: migrate package installation from pak to uvr (#147) ... `bincraft` dropped pak in favour of uvr (5.0.x, "Dependencies and their system requirements are now installed with `uvr` instead of pak during `build_binary_package()`"), so the pipelines, helper scripts and images in this repo move with it. ## Approach uvr is project-scoped in a way pak is not: `uvr add` refuses to run outside a project and always installs into `.uvr/library/`, and only `uvr sync` honours `--library`. So there is no one-line `pak::pak(...)` equivalent. `local/uvr-install.sh` encapsulates the dance — bootstrap a pinned uvr, mint a throwaway project under `TMPDIR`, `uvr add --no-install`, then `uvr sync --library <target>`. Keeping the project outside the checkout also keeps `uvr init`'s `.Rprofile` from hijacking `.libPaths()` for every other R call in the pipeline. This matches what bincraft itself does (`uvr sync --install-system-deps --library <lib>`), and bincraft requires `uvr` on `PATH`, which the bootstrap provides: every pipeline that calls `bincraft::` runs `install-bincraft.R` (and therefore the bootstrap) first. ## Changes | File | Change | | --- | --- | | `local/uvr-install.sh` | **New.** The single replacement for `pak::pak(...)`. Bootstraps uvr `v0.4.4`, resolves the R interpreter from `UVR_R_BIN`/`R_VERSION`/`PATH`, pins the manifest to that R's exact version, and syncs into `UVR_TARGET_LIB`/`R_LIBS_USER`. | | `local/install-bincraft.R` | Installs `forgejo::codefloe.com/rpkgs/bincraft@<tag>` instead of a `git::` URL; keeps the `git ls-remote` tag resolution. Exports `UVR_R_BIN`/`UVR_TARGET_LIB` from `R.home()`/`.libPaths()[1]` so the per-R-minor passes target their own R and library. | | `.crow/auto-apply-patches.yaml`, `.crow/weekly-patch-proposals.yaml`, `.crow/weekly-audit-missing.yaml`, `.crow/weekly-rebuild-missing.yaml`, `.crow/build-all-versions-install-deps.yaml` | `pak::pak(...)` → `UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh ...`. The explicit `UVR_R_BIN` matters in `build-all-versions-install-deps.yaml`, which has no `R_VERSION` in its step environment. | | `.crow/build-all-versions.yaml`, `.crow/process-updates.yaml`, `.crow/weekly-rebuild-missing.yaml` | `R_PKG_CACHE_DIR` → `UVR_CACHE_DIR` + `UVR_PACKAGES_DIR` on the same `/mnt/cache` volume, preserving the amd64-off/arm64-on split. Drops the `rm -rf .../pkgcache/_metadata/...` cleanup. | | `local/r-minor-helpers.R`, `local/build-all.R`, `local/tests/test-trim-pkgcache.R` | Removes `trim_pkgcache_metadata()`, its every-25-packages call and its tests. uvr's cache does not mint a fresh ~70 MB snapshot per `PACKAGES` change. | | `.crow/build-all-versions-install-deps.yaml` | Drops `pak::sysreqs_db_update()`; uvr resolves sysreqs from its vendored `r-system-requirements` rules via `--install-system-deps`. | | `docker/Containerfile-shiny-app` | Bootstraps uvr and drives both dependency installs through one uvr project with `UVR_LIBRARY` pointed at the image's R library. | | `docker/build-one.Dockerfile` | Ships `uvr-install.sh` at `/work/local/` so `install-bincraft.R` finds it. | | `docker/reprex/alpine.sh` | Replaces `pak::local_install_deps()` with DESCRIPTION parsing + `uvr add`. | | `local/test-package-loading.R` | Installs via the helper instead of `pak::pkg_install()`. | | `README.md` | Documents uvr for sysreq inference, archived-version installs and cache clearing. | | `renovate.json` | Tracks the `UVR_PIN` in `uvr-install.sh` via `github-releases`. | ## Behaviour notes - **`weekly-audit-missing` still takes bincraft from the default branch**, not the latest release tag, matching what the `git::` pak call did. Called out in a comment rather than silently changed. - **The uvr pin is repo-wide.** bincraft resolves `uvr` from `PATH` and pins no version of its own, so `UVR_PIN` in `uvr-install.sh` governs the whole pipeline. - **Persistent caches now also benefit bincraft**, which reads `UVR_CACHE_DIR`/`UVR_PACKAGES_DIR` from the inherited pipeline environment. - **`uvr sync` will not prune the shared library.** Pruning is disabled whenever `--library` is passed (`do_prune = prune && library_override.is_none()`), so `/mnt/cache/R-pkgs` keeps bincraft and its dependencies. The wipe-on-ABI-mismatch path is *not* similarly guarded, which is why the helper pins the manifest to the active R's exact version. - **`plans/` and `specs/` are untouched** — they are dated records of decisions made in June 2026 and describe bincraft's then-pak-based internals; rewriting them would misstate history. ## Verification `shellcheck`, all pre-commit hooks (on this commit's file range) and the `local/tests/` suite (100 assertions) pass. Not yet exercised in CI: the build-env images do not ship `uvr`, so the per-step `curl install.sh` bootstrap is untested against a real image. Worth a manual `build-all-versions-install-deps` run before merging. Reviewed-on: #147	2026-07-31 12:19:26 +00:00
pat-s	1ad7a6bfe9	chore: resolve latest bincraft release dynamically (no hardcoded pins) (#107) ... ## Summary Stop hardcoding the bincraft version. Every `.crow` workflow and the build-one image pinned `@vX.Y.Z` (and a `packageVersion() != "X.Y.Z"` guard), so each bincraft release meant editing the version in ~8 places — and it was easy to miss one (the Dockerfile lagged at v4.2.1; v4.4.1 shipped without the empty-env fix because of exactly this churn). ## Change New `local/install-bincraft.R` resolves the **latest release tag dynamically**: - `git ls-remote --tags` on the public repo (no token), - keep `vX.Y.Z` tags, pick the highest version (filtered/sorted in R for portability, not via git `--sort`/refspec which behaved inconsistently under `system2()`), - `pak::pak("git::…@<latest>")` — idempotent on the git ref, so re-runs keep the package unless a newer tag exists. All call sites now invoke the helper instead of a pinned version: - `.crow/build-all-versions.yaml` (primary + per-minor pass) - `.crow/build-all-versions-install-deps.yaml` - `.crow/process-updates.yaml` (primary + per-minor pass) - `.crow/weekly-rebuild-missing.yaml` - `.crow/archive-missed-packages.yaml` - `docker/build-one.Dockerfile` (ships the helper into the image; `ensure_bincraft` sources it) ## Effect Tag a new bincraft release → the next CI run / `just rebuild` picks it up automatically. No more pin edits, and no more "forgot to bump the Dockerfile" drift. ## Verified - Resolver returns the current latest tag (`v4.4.2`) via `git ls-remote` + R-side version sort. - All five workflow YAMLs parse; helper R parses; air/editorconfig clean. Note: this tracks the latest **tag**, so cutting a release is still the deliberate gate — CI won't pick up un-tagged main. Reviewed-on: #107	2026-07-01 08:10:02 +00:00
pat-s	b4ec6291a9	chore: bump bincraft to 4.4.2	2026-06-30 16:09:44 +02:00
pat-s	d496029a79	chore: bump bincraft to 4.4.1	2026-06-30 15:51:07 +02:00
pat-s	b93d1ef6bd	chore: silence otelsdk warnings across build paths (#105) ... ## Summary Silences the recurring `OpenTelemetry error: there is no package called 'otelsdk'` warnings during builds. ## Root cause The `build-env-*` images configure an OTel exporter (traces/logs/metrics), but `otelsdk` (the R OTel SDK backend) isn't installed. pak's `otel` instrumentation therefore tries to load `otelsdk` on every run and logs the error, falling back to a no-op. `OTEL_SDK_DISABLED=true` (already set in `build-one.Dockerfile`) does **not** help — the R `otel` package ignores it and gates purely on `OTEL_R_<SIGNAL>_EXPORTER` (then the standard `OTEL_<SIGNAL>_EXPORTER`). When that resolves to a real exporter (`otlp`/`http`/…) with no SDK present, you get the error. ## Fix Set `OTEL_R_TRACES_EXPORTER`, `OTEL_R_LOGS_EXPORTER`, and `OTEL_R_METRICS_EXPORTER` to `none` so the R otel providers are clean no-ops: - `docker/build-one.Dockerfile` — exported alongside the existing OTel var. - `.crow/process-updates.yaml`, `weekly-rebuild-missing.yaml`, `build-all-versions.yaml`, `build-all-versions-install-deps.yaml` — added to each step's `environment` block. Using the R-specific variables (not the standard `OTEL_*_EXPORTER`) keeps OTel intact for any non-R tooling in the images. Reviewed-on: #105	2026-06-30 12:41:32 +00:00
pat-s	b744ae1bb1	fix: build-one image bincraft v4.4.0 + ship patch registry (#104) ... ## Summary Fixes the **targeted rebuild** path (`just rebuild` → `docker/build-one.Dockerfile`) so it actually applies patches. Two problems, both of which would make `build-one.R`'s `patches = "local/patches"` a silent no-op or a hard error: 1. **Stale bincraft pin.** The Dockerfile pinned bincraft `v4.2.1`, which predates the `patches` argument — so `build_binary_package(patches = ...)` would fail with `unused argument`. Bumped to **v4.4.0** to match the `.crow` workflows. 2. **Registry not shipped into the image.** Only `build-one.R` was copied in; `local/patches/registry.json` was absent, so `patches = "local/patches"` resolved to a nonexistent `/work/local/patches` and silently applied nothing. Added `COPY patches /work/local/patches` (build context is `local/`, CWD is `/work`). ## Why This is the path used to verify the patching end-to-end, e.g.: ```bash just rebuild alpine 3.23 amd64 rts2 1.0.3 ``` rts2 depends on RcppParallel; with this fix the container installs bincraft v4.4.0, ships the registry, and the patched RcppParallel binary (`RCPP_PARALLEL_USE_TBB=0`) is served to pak during dependency install. Expect `Applying patch to RcppParallel …` in the log, RcppParallel installed as a binary (no recompile), then rts2 building and uploading. Reviewed-on: #104	2026-06-30 12:15:39 +00:00
pat-s	7e01f481c0	perf(local): skip per-minor passes for non-r-minor-sensitive packages ... A package's ABI sensitivity is the same across R minors, so for a non-sensitive package the per-minor loop only ran `ensure_bincraft` (a ~14s bincraft/cranlike install per minor) before build-one.R classified and skipped it. The primary pass now writes a .r_minor_sensitive sentinel when it builds a sensitive package, and the wrapper gates the whole per-minor loop (installs included) on that file — non-sensitive rebuilds no longer touch other minors' libraries.	2026-06-14 14:50:40 +02:00
pat-s	3ae4672d66	feat(local): just rebuild builds across all installed R minors ... - Dockerfile: install bincraft into each R minor, run a primary pass plus a sensitive-only pass under every other /opt/R/[0-9]* minor (deduped by minor), probe/skip xvfb, set GIT_TERMINAL_PROMPT=0; extra-minor failures are non-fatal. - build-one.R: add --sensitive-only mode, log + shallow-clone the ABI classify step, and clear cranlike's stale ./PACKAGES.db before each index refresh (workaround for the "table packages already exists" bug; pending cranlike fix).	2026-06-14 13:02:48 +02:00
pat-s	1224bd74fe	feat(local): just rebuild recipe for targeted builds via remote buildx (#87) ... ## Summary Adds a local `just rebuild` recipe to (re)build specific versions of a single package on a given OS/arch, dispatching to a remote buildx builder (the build runs there, not locally). - `just rebuild <os> <tag> <arch> <package> <version>...` → `docker buildx build --builder <artemis|gaia> --platform linux/<arch> …` (amd64→artemis, arm64→gaia; names + `R_VERSION` env-overridable). - `docker/build-one.Dockerfile` runs `build-one.R` as a secret-mounted `RUN`, built `--no-cache --output type=cacheonly` (pure side-effect: the S3 upload; no image kept). - `local/build-one.R` auto-classifies each version via the ABI classifier (risky → per-minor slot `contrib/<x.y>/`, else generic), force-rebuilds + uploads + stores metadata, then refreshes the touched slot's `PACKAGES` index. ## Prerequisites - buildx builders named `artemis` (amd64) and `gaia` (arm64) registered (`docker buildx create --name artemis ssh://…`). - Exported secrets: `B2_S3_ACCESS_KEY`, `B2_S3_SECRET_KEY`, `PGPASS` (`GITHUB_PAT` optional). - bincraft `v4.2.0` tag must exist (the build installs `@v4.2.0` and uses its classifier + per-minor index API). Reviewed-on: #87	2026-06-14 09:03:45 +00:00