Author	SHA1	Message	Date
pat-s	d558e27c11	feat: R-minor-sensitive binary builds (full + iterative) (#84) ... ## Summary Builds R-minor-sensitive CRAN packages once per installed R minor version (into per-minor S3 slots `…/contrib/<x.y>/`) and everything else once into the generic slot, driven by bincraft 4.2.0's ABI classifier. Both the full and iterative pipelines are covered. ## What's in here **Detection / precompute** - `local/r-minor-helpers.R` — pure `classify_from_metadata()` (NeedsCompilation / risky `LinkingTo`) + `parse_build_args()`, with unit tests. - `local/packages-to-build.R` — adds a per-package `r_minor_sensitive` flag: cheap CRAN-metadata rules first, source download + `bincraft::needs_per_minor_recompile()` only for the ambiguous compiled subset (fail-safe to sensitive). Classified once per package, applied to all versions. **Full build** - `local/build-all.R` — passes the per-row `is_r_minor_sensitive` flag; new `--sensitive-only` mode builds just the risky subset. - `.crow/build-all-versions-{amd64,arm64}.yaml` — install-deps persists the sensitive subset; build step runs a sensitive-only pass under each non-primary `/opt/R/*` minor; new step uploads the generic index plus a per-minor index for each minor. **Iterative build** - All 14 `.crow/process-updates-*.yaml` — primary pass uses `r_minor_detection = 'classifier'`; a sensitive-only multi-R pass builds risky updates under each other minor; per-minor index upload added. **Tooling / housekeeping** - Pins bincraft `v4.1.1` → `v4.2.0` across all workflows; removes the superseded standalone `build-r-minor-sensitive-packages.yaml`. - Adds prek/pre-commit hooks (prettier, markdownlint, editorconfig-checker, yamllint, air) and applies them repo-wide; excludes the verbatim GPL `LICENSE.md` and auxiliary shell scripts. - Design + implementation docs under `docs/superpowers/`. ## Requires before merge - A `v4.2.0` git tag must be pushed on the bincraft repo (codefloe.com/rpkgs/bincraft) — the workflow install steps pin `@v4.2.0`. The full-build install-deps clones `main`, so it is unaffected. Reviewed-on: #84	2026-06-14 08:04:40 +00:00
pat-s	8c5362b7ab	refactor: R_VERSION syntax	2026-05-27 22:36:09 +02:00
pat-s	6faa3772d1	refactor(ci): use multi-R-version images across workflows (#70) ... ## Summary - Switches all 64 `.crow/*.yaml` workflows, the `Justfile`, and the commented-out `build-all-versions-install-deps.yaml` template to the new multi-R-version container images. - Each image tag drops `-${R_VERSION}` (e.g. `build-env-alpine:3.22-4.5` → `build-env-alpine:3.22`); workflows pick an R version by calling `/opt/R/${R_VERSION}/bin/R` explicitly. - Pattern-1 workflows (hard-coded image tag) gain a single `R_VERSION:` env var per file. - Pattern-2 workflows (`build-all-versions-*`, matrix-driven) continue to receive `R_VERSION` via `crow pipeline create --var`. ## Platform → R_VERSION mapping | Platform | Image | R_VERSION | | --- | --- | --- | | alpine 3.22 / 3.23 | `build-env-alpine:3.{22,23}` | 4.5.3 | | ubuntu jammy / noble | `build-env-ubuntu:{jammy,noble}` | 4.4.3 | | redhat 8 / 9 | `build-env-redhat:{8,9}` | 4.4.3 | | redhat 10 | `build-env-redhat:10` | 4.5.3 | The `alpine-321` audit-only workflows fall back to `build-env-alpine:3.23` since no 3.21 image exists in the new scheme; the audits only query S3/CRAN, so the container OS does not affect correctness. ## Bug fixes folded in 1. All six `weekly-audit-missing-alpine-*.yaml` files were pinned to `alpine:3.23-4.5` regardless of platform. Now each one points at its own alpine image. 2. All fourteen `update-package-index-*.yaml` files were pinned to `build-env-ubuntu:noble-4.4` (or `noble-4.5` for `redhat-10-amd64`) regardless of platform. Now each one uses its own platform's image. 3. `process-updates-ubuntu-2404-{amd64,arm64}.yaml` had `R_VERSION` of `4.4` (no patch) baked into the tag. Aligned to `4.4.3`, matching the audit and rebuild counterparts. ## Smoke targets (post-merge) One workflow per shape, ready to dry-run: - `process-updates-alpine-322-amd64.yaml` - `weekly-rebuild-missing-redhat-9-amd64.yaml` - `weekly-audit-missing-ubuntu-2204-amd64.yaml` - `update-package-index-redhat-10-amd64.yaml` - `archive-missed-packages.yaml` - `build-all-versions-amd64.yaml` (with its install-deps predecessor) - `build-r-minor-sensitive-packages.yaml` ## Follow-ups (out of scope, surfaced during review) - `process-updates-redhat-10-{amd64,arm64}.yaml` set `codename = "rhel9"` in their `upload_package_index(...)` calls. Pre-existing bug — should be `"rhel10"`. The `update-package-index-redhat-10-*` counterparts already use `"rhel10"` correctly. - `update-package-index-redhat-8-amd64.yaml` is missing the `packageVersion("bincraft")` diagnostic line that every sibling file has. Pre-existing asymmetry between amd64 and arm64. ## Spec & plan - Spec: `docs/superpowers/specs/2026-05-25-multi-r-version-images-design.md` - Plan: `docs/superpowers/plans/2026-05-25-multi-r-version-images.md` Reviewed-on: #70	2026-05-26 08:51:27 +00:00
pat-s	9b27b3e650	refactor: extract build-all R one-liner into local/build-all.R ... - Load bincraft eagerly via library() to avoid lazy-load memory spike - Do bulk S3 listing upfront and pass as s3_package_cache to avoid per-package S3 calls that accumulate memory and trigger fork failures - Consolidate amd64/arm64 exclude lists into single script - Much easier to read and maintain than a YAML-embedded one-liner	2026-04-17 19:01:08 +02:00
pat-s	e47949f546	fix: remove force=TRUE to preserve error metadata tracking ... force=TRUE bypasses the error-DB filter and overwrites build metadata, causing failed packages to be retried indefinitely. Since the precompute step already filters to missing versions, force is unnecessary.	2026-04-17 18:35:59 +02:00
pat-s	a732993d2b	feat: iterate over (package, version) pairs in build-all pipeline ... Build step now reads precomputed package+version pairs and passes specific tags with force=TRUE, eliminating all per-package precondition I/O.	2026-04-17 18:22:36 +02:00
pat-s	9330bef97c	chore: adjust matrix split counts for amd64 and arm64 builds	2026-04-15 10:13:16 +02:00
pat-s	a27011d049	chore: 4-way split for amd	2026-04-12 19:07:32 +02:00
pat-s	a64dea6b23	refactor: separate install-deps workflows for arch	2026-04-11 09:49:44 +02:00
pat-s	1c1bfb5f6d	chore: align arm64 workflow	2026-04-11 09:47:19 +02:00
pat-s	4b27430270	fix: use weston instead of mutter as Wayland compositor to reduce memory usage ... Mutter (default xwfb-run compositor) consumes ~200-500MB per instance, causing fork failures with 5 parallel matrix jobs. Weston is much lighter.	2026-04-10 20:15:42 +02:00
pat-s	4892497c76	fix: redirect R message stream to stdout to bypass xwfb-run stderr swallowing	2026-04-10 17:48:29 +02:00
pat-s	112bc60ec2	debug: add per-package progress output to stdout to diagnose hang vs buffering	2026-04-10 17:30:20 +02:00
pat-s	270843eaf9	refactor: precompute package list in install-deps step to reduce matrix job memory usage ... Move packages-to-build.R execution to the single install-deps step and save results as RDS. Matrix jobs read the precomputed list instead of loading the massive CRAN archive/release databases, freeing memory for fork/system calls.	2026-04-10 16:21:33 +02:00
pat-s	6da70e5074	fix: use unique X server number per matrix job to prevent Xwayland collisions	2026-04-10 12:43:11 +02:00
pat-s	3f985b3bc7	fix: use plan('sequential') instead of removing future, bincraft needs it internally for future_mapply	2026-04-10 12:41:09 +02:00
pat-s	f831db042c	fix: use plan('multisession') instead of plan('sequential') for future_mapply compatibility	2026-04-10 12:37:44 +02:00
pat-s	1ff6a0c32a	fix: use plan('sequential') instead of removing future, bincraft needs it internally for future_mapply	2026-04-10 08:18:53 +02:00
pat-s	7ca1b6e5c6	fix: remove pak::sysreqs_db_update() from matrix step, already cached by install-deps	2026-04-10 00:28:34 +02:00
pat-s	e062c58185	fix: restore pak::sysreqs_db_update() in matrix step and pre-cache it in install-deps	2026-04-10 00:15:23 +02:00
pat-s	0c4b56262c	fix: remove redundant pak::sysreqs_db_update() from matrix step to reduce memory pressure	2026-04-10 00:06:25 +02:00
pat-s	11c4071c0d	fix: remove future multisession to avoid fork memory failures with parallel matrix jobs	2026-04-09 23:58:51 +02:00
pat-s	272275e868	refactor: move bincraft install to separate pre-matrix pipeline step ... Adds build-all-versions-install-deps pipeline that runs once before the matrix builds, cleaning stale locks and installing bincraft to the shared cache without contention.	2026-04-09 23:55:30 +02:00
pat-s	08070ce62f	test: use sequential lapply instead of future multisession to rule out fork memory issue	2026-04-09 23:49:47 +02:00
pat-s	0aba224c11	fix: install bincraft to per-job /tmp/R-libs to avoid shared cache conflicts between parallel matrix builds	2026-04-09 23:41:41 +02:00
pat-s	0efe1d3485	fix: remove cached bincraft dir before install to avoid pak metadata conflict	2026-04-09 23:37:40 +02:00
pat-s	978eb42e2a	fix: skip dependency upgrades when installing bincraft with pak::local_install()	2026-04-09 23:33:29 +02:00
pat-s	7d58aa7e59	fix: use pak::local_install() from cloned path to avoid git URL resolution bug	2026-04-09 23:27:50 +02:00
pat-s	fee3e4ca94	fix: install bincraft via git clone + R CMD INSTALL --no-deps to avoid dependency rebuild and lock conflicts	2026-04-09 23:25:49 +02:00
pat-s	9ce6b0b4e1	fix: remove old bincraft before reinstall to avoid RemoteType metadata conflict	2026-04-09 23:24:30 +02:00
pat-s	3fbf4ca1f8	fix: force reinstall bincraft to bypass RemoteType mismatch with pak	2026-04-09 23:22:47 +02:00
pat-s	2f81bd6be5	fix: clean stale R package lock dirs before installing bincraft ... Parallel matrix jobs sharing /mnt/cache/R-pkgs leave stale 00LOCK-* dirs that block subsequent installs.	2026-04-09 23:21:12 +02:00
pat-s	1b24fa731a	fix: install remotes package if missing before installing bincraft	2026-04-09 23:19:53 +02:00
pat-s	c9040ee50a	fix: use remotes::install_git() instead of pak::pak() for bincraft installation ... pak::pak() with git:: URLs fails with "subscript out of bounds" in the build environment.	2026-04-09 23:18:59 +02:00
pat-s	36c69e983f	chore: always update bincraft at build start instead of using cached version	2026-04-09 23:16:35 +02:00
pat-s	688c299138	chore: redirect stderr to stdout in build commands to surface R errors	2026-04-09 23:01:40 +02:00
pat-s	dae04820f8	fix: add -- separator to xwfb-run invocations to prevent flag interception ... xwfb-run uses argparse and was consuming R's -q flag as its own argument, causing build failures.	2026-04-09 22:51:37 +02:00
pat-s	0f8d36cb51	chore: use xwfb-run conditionally for RHEL 10 support	2026-03-24 11:31:23 +01:00
pat-s	3adf48b1c0	chore: build on artemis on docker backend	2026-03-24 09:50:44 +01:00
pat-s	e959d9743f	refactor: migrate to backblaze	2026-03-22 00:32:03 +01:00
pat-s	75893a8eda	build rhel10	2026-01-05 17:32:13 +01:00
pat-s	50c4138aa2	chore: ignore gwlasso	2025-12-21 00:22:16 +01:00
pat-s	1ca51368f9	fix: quotes in R call	2025-12-15 18:07:04 +01:00
pat-s	2ebe7ba567	chore: only build 5 most recent releases	2025-12-13 00:10:16 +01:00
pat-s	b153d70b92	build alpine 3.23	2025-12-07 00:07:39 +01:00
pat-s	510bff292a	ci: GIT_RO_TOKEN -> REPO_RO_TOKEN	2025-10-17 14:45:07 +02:00
pat-s	5dbfe1127b	reprocess packages without history	2025-07-03 22:45:19 +02:00
pat-s	15eaa91bf7	remove install-deps step for arm64 builds	2025-06-26 23:34:24 +02:00
pat-s	7454d942e0	build missing arm64 alpine	2025-06-26 23:20:18 +02:00
pat-s	ec99df2fa8	ignore pkgs	2025-06-25 23:22:01 +02:00