Commit graph build-cran-binaries/justfile
Author SHA1 Message Date
09536cddd6 feat(build): raise BuildKit cache-mount budget for remote builders (#112)
Some checks failed
ci/crow/cron/weekly-rebuild-missing/5 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/10 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/11 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/12 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/13 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/9 Pipeline was canceled
ci/crow/cron/weekly-rebuild-missing/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline failed
ci/crow/cron/process-updates/11 Pipeline was canceled
ci/crow/cron/process-updates/16 Pipeline was canceled
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was canceled
ci/crow/cron/process-updates/1 Pipeline was canceled
ci/crow/cron/process-updates/2 Pipeline was canceled
ci/crow/cron/process-updates/8 Pipeline was canceled
ci/crow/manual/build-all-versions-install-deps/2 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was canceled
ci/crow/cron/process-updates/3 Pipeline was canceled
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was canceled
ci/crow/manual/build-all-versions/7 Pipeline was canceled
ci/crow/manual/build-all-versions/6 Pipeline failed
ci/crow/manual/build-all-versions/8 Pipeline was canceled
ci/crow/manual/build-all-versions/5 Pipeline was canceled
ci/crow/cron/process-updates/4 Pipeline was canceled
ci/crow/cron/process-updates/14 Pipeline was canceled
ci/crow/cron/process-updates/13 Pipeline was successful
BuildKit's default GC caps the ephemeral cache tier — `RUN --mount=type=cache` mounts, local build context, git checkouts — at a hardcoded **512 MB** (shown as `488.3 MiB` in `buildx inspect`).
Across our 7-distro build matrix that fills instantly and forces re-downloads of system + R packages on every rebuild.

## Changes

- **`docker/buildkitd.toml`** (new) — GC config passed via `--config` when creating the `docker-container` builders `artemis` (amd64) and `gaia` (arm64):
  - cache-mount tier: **512 MB → 8 GB**, retained 7 days
  - total cache bounded at **40 GB** with **20 GB min-free**
- **`justfile`** — document the `--config docker/buildkitd.toml` flag on the builder-create commands so a recreate does not silently revert to the 512 MB default.

## Notes

- Limits are absolute (not `%`) because the two hosts differ ~6× in free space (Hetzner ~42 GiB free vs Mac mini ~279 GiB). The 20 GB min-free is the safety valve on the disk-tight Hetzner host; the old default wanted 64 GiB free, which does not exist there.
- Both live builders were already recreated with this config and verified running.

Reviewed-on: #112
2026-07-08 08:22:03 +00:00
e5398af5f5
feat(local): just rebuild — TF_VAR secrets, live progress, builder docs
- Accept TF_VAR_-prefixed B2 keys (from direnv); make GITHUB_PAT optional.
- Stream the remote build with --progress=plain.
- Document creating the artemis/gaia buildx builders with the docker-container
  driver (the default remote driver does not work with an ssh:// docker host).
2026-06-14 13:02:47 +02:00
7f5674ffb3 fix(local): accept TF_VAR_-prefixed B2 keys, optional GITHUB_PAT (#88)
Some checks failed
ci/crow/cron/process-updates-redhat-10-amd64 Pipeline was successful
ci/crow/cron/process-updates-redhat-10-arm64 Pipeline failed
ci/crow/cron/process-updates-ubuntu-2404-arm64 Pipeline failed
ci/crow/cron/process-updates-ubuntu-2404-amd64 Pipeline was successful
## Summary

Follow-up fix to the `just rebuild` recipe (#87): the local `.envrc` exports the B2 credentials as `TF_VAR_B2_S3_ACCESS_KEY` / `TF_VAR_B2_S3_SECRET_KEY` (and has no `GITHUB_PAT`), but the recipe required the plain `B2_S3_ACCESS_KEY` / `B2_S3_SECRET_KEY` names and always passed a `github_pat` secret.

- Accept the `TF_VAR_`-prefixed B2 names (falling back to the plain names if set directly).
- Pass the `github_pat` buildx secret only when `GITHUB_PAT` is set, so the build works without it.

Reviewed-on: #88
2026-06-14 09:08:48 +00:00
1224bd74fe feat(local): just rebuild recipe for targeted builds via remote buildx (#87)
## Summary

Adds a local `just rebuild` recipe to (re)build specific versions of a single package on a given OS/arch, dispatching to a remote buildx builder (the build runs there, not locally).

- `just rebuild <os> <tag> <arch> <package> <version>...` → `docker buildx build --builder <artemis|gaia> --platform linux/<arch> …` (amd64→artemis, arm64→gaia; names + `R_VERSION` env-overridable).
- `docker/build-one.Dockerfile` runs `build-one.R` as a secret-mounted `RUN`, built `--no-cache --output type=cacheonly` (pure side-effect: the S3 upload; no image kept).
- `local/build-one.R` auto-classifies each version via the ABI classifier (risky → per-minor slot `contrib/<x.y>/`, else generic), force-rebuilds + uploads + stores metadata, then refreshes the touched slot's `PACKAGES` index.

## Prerequisites

- buildx builders named `artemis` (amd64) and `gaia` (arm64) registered (`docker buildx create --name artemis ssh://…`).
- Exported secrets: `B2_S3_ACCESS_KEY`, `B2_S3_SECRET_KEY`, `PGPASS` (`GITHUB_PAT` optional).
- bincraft `v4.2.0` tag must exist (the build installs `@v4.2.0` and uses its classifier + per-minor index API).

Reviewed-on: #87
2026-06-14 09:03:45 +00:00