feat(local): just rebuild recipe for targeted builds via remote buildx (#87)
## Summary Adds a local `just rebuild` recipe to (re)build specific versions of a single package on a given OS/arch, dispatching to a remote buildx builder (the build runs there, not locally). - `just rebuild <os> <tag> <arch> <package> <version>...` → `docker buildx build --builder <artemis|gaia> --platform linux/<arch> …` (amd64→artemis, arm64→gaia; names + `R_VERSION` env-overridable). - `docker/build-one.Dockerfile` runs `build-one.R` as a secret-mounted `RUN`, built `--no-cache --output type=cacheonly` (pure side-effect: the S3 upload; no image kept). - `local/build-one.R` auto-classifies each version via the ABI classifier (risky → per-minor slot `contrib/<x.y>/`, else generic), force-rebuilds + uploads + stores metadata, then refreshes the touched slot's `PACKAGES` index. ## Prerequisites - buildx builders named `artemis` (amd64) and `gaia` (arm64) registered (`docker buildx create --name artemis ssh://…`). - Exported secrets: `B2_S3_ACCESS_KEY`, `B2_S3_SECRET_KEY`, `PGPASS` (`GITHUB_PAT` optional). - bincraft `v4.2.0` tag must exist (the build installs `@v4.2.0` and uses its classifier + per-minor index API). Reviewed-on: #87
This commit is contained in:
parent
d558e27c11
commit
1224bd74fe
1 changed files with 220 additions and 0 deletions
62
justfile
Normal file
62
justfile
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
# Local helpers for build-cran-binaries.
|
||||
#
|
||||
# `rebuild` (re)builds specific versions of a single package on a given OS/arch
|
||||
# by dispatching to a remote buildx builder (the build runs there, not locally).
|
||||
# Sensitivity is auto-detected: risky packages land in the per-minor slot
|
||||
# (contrib/<x.y>/), everything else in the generic slot; the touched index is
|
||||
# refreshed so the result is immediately servable.
|
||||
#
|
||||
# Prerequisites:
|
||||
# - buildx builders named `artemis` (amd64) and `gaia` (arm64), e.g.
|
||||
# docker buildx create --name artemis --node artemis ssh://<host-amd64>
|
||||
# docker buildx create --name gaia --node gaia ssh://<host-arm64>
|
||||
# - exported secrets: B2_S3_ACCESS_KEY, B2_S3_SECRET_KEY, PGPASS (GITHUB_PAT optional)
|
||||
#
|
||||
# Overridable (env or `just VAR=… rebuild …`):
|
||||
# R_VERSION (default 4.5.3) — selects the R minor → the per-minor slot
|
||||
# AMD64_BUILDER / ARM64_BUILDER — buildx builder names
|
||||
#
|
||||
# Examples:
|
||||
# just rebuild alpine 3.23 amd64 rlang 1.1.4 1.1.3
|
||||
# just rebuild redhat 10 arm64 data.table 1.15.4
|
||||
# R_VERSION=4.4.3 just rebuild ubuntu noble amd64 Rcpp 1.0.12
|
||||
|
||||
r_version := env_var_or_default("R_VERSION", "4.5.3")
|
||||
amd64_builder := env_var_or_default("AMD64_BUILDER", "artemis")
|
||||
arm64_builder := env_var_or_default("ARM64_BUILDER", "gaia")
|
||||
|
||||
# (re)build PACKAGE at one or more VERSIONS on OS/TAG/ARCH via a remote builder
|
||||
rebuild os tag arch package *versions:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
if [ -z "{{ versions }}" ]; then
|
||||
echo "error: provide at least one version, e.g. just rebuild alpine 3.23 amd64 rlang 1.1.4" >&2
|
||||
exit 1
|
||||
fi
|
||||
case "{{ arch }}" in
|
||||
amd64) builder="{{ amd64_builder }}" ;;
|
||||
arm64) builder="{{ arm64_builder }}" ;;
|
||||
*) echo "error: arch must be 'amd64' or 'arm64'" >&2; exit 1 ;;
|
||||
esac
|
||||
: "${B2_S3_ACCESS_KEY:?set B2_S3_ACCESS_KEY in your environment}"
|
||||
: "${B2_S3_SECRET_KEY:?set B2_S3_SECRET_KEY in your environment}"
|
||||
: "${PGPASS:?set PGPASS in your environment}"
|
||||
: "${GITHUB_PAT:=}"
|
||||
echo "Dispatching build of {{ package }} ({{ versions }}) on {{ os }}:{{ tag }}/{{ arch }} (R {{ r_version }}) to builder '$builder'"
|
||||
docker buildx build \
|
||||
--builder "$builder" \
|
||||
--platform "linux/{{ arch }}" \
|
||||
--no-cache \
|
||||
--output type=cacheonly \
|
||||
--secret id=b2_access,env=B2_S3_ACCESS_KEY \
|
||||
--secret id=b2_secret,env=B2_S3_SECRET_KEY \
|
||||
--secret id=pgpass,env=PGPASS \
|
||||
--secret id=github_pat,env=GITHUB_PAT \
|
||||
--build-arg OS="{{ os }}" \
|
||||
--build-arg OS_VERSION="{{ tag }}" \
|
||||
--build-arg R_VERSION="{{ r_version }}" \
|
||||
--build-arg PACKAGE="{{ package }}" \
|
||||
--build-arg VERSIONS="{{ versions }}" \
|
||||
--build-arg CACHEBUST="$(date +%s)" \
|
||||
-f docker/build-one.Dockerfile \
|
||||
local
|
||||
Loading…
Reference in a new issue