Author	SHA1	Message	Date
pat-s	e23bd3e539	perf: seed pure-R binaries into a new slot, interleave shards, and stop builds before the timeout (#242) ... Speeds up the full rebuild for a new slot, e.g. the upcoming Alpine version. ## Changes - **Seed pure-R binaries** (`local/seed-pure-r.R`, `local/seed-helpers.R`), opt-in via the new `SEED_FROM` variable on `build-all-versions-install-deps`. About 74% of the ~84k candidate versions have no compiled code and cost ~20s each to build; their R objects are identical across slots. Before the build list is computed, each donor candidate is downloaded and verified, then copied server-side: - donor must be the same OS family and arch; - DESCRIPTION has `Built` (not a CRAN source fallback) with an empty platform and no `libs/`; - built by the target's primary R minor; - bincraft's `tarball_abi_verdict()` calls it portable across every installed minor (unknown means no seed). Each copy gets the `single_builds` and `abi_flat_safety` rows an upload writes, so build-all skips it and the per-minor indexes keep it. Re-runs skip existing objects and rewrite rows idempotently. - **Interleaved shards** in `local/build-all.R` via `shard_slice()`; contiguous alphabetical quarters ranged from 0.1h to 28h. - **Shared build deadline**: `build-all-versions` sets `BUILD_DEADLINE` from `BUILD_BUDGET_HOURS` (44h, repo timeout 50h); `build-all.R` and `build-newer-r.R` stop cleanly when it passes, so archiving still runs. - `codename_platform()` replaces the hard-coded codename-to-platform list. ## Testing - New unit tests for the seed verdict, candidate filter, platform names and deadline; all `local/tests` pass. - Verdicts checked on real amd64/alpine324 tarballs: R6 2.6.1/2.5.1 and jsonstat seeded, cli refused as compiled. - `crow lint .crow/` passes. - The seed has not run against B2/Postgres yet; the first bootstrap run is its end-to-end test. ## Usage Trigger `build-all-versions` with `target_arch`, `OS`, `OS_VERSION` and `SEED_FROM="alpine324 alpine323"`. The new Alpine version still needs adding to the `OS_VERSION` options once its image exists. Reviewed-on: #242	2026-10-24 02:04:06 +00:00
pat-s	63149e246a	fix(build-newer-r): pass the system CA bundle to the ABI-verdict query (#241) ... Follow-up to #240. The bulk skip in `build-newer-r.R` never took effect: in all 16 `process-updates` runs on `773add0` (14300–14315) the ABI-verdict query failed with `root certificate file "/root/.postgresql/root.crt" does not exist`. The fallback did its job: every run logged `0 of 1323 already carry a binary … 1323 left for bincraft` and built exactly as before. So nothing broke, but the run saved no time either. ## Changes - `local/build-newer-r.R`: pass `sslrootcert = bincraft::postgres_sslrootcert("verify-full")` to the connection, as every other DB script in `local/` already does. With `verify-full` and no `sslrootcert`, libpq looks for `~/.postgresql/root.crt`. ## Verification - In the `build-env-ubuntu:jammy`, `build-env-redhat:9` and `build-env-alpine:3.24` images, connecting with the distro's CA bundle gets past TLS verification to password authentication (tested with a deliberately wrong password). - The S3 listings in the same runs logged no errors. ## Behaviour change - The next runs should log a non-zero "N of M already carry a binary at their routed path" count and skip those candidates without calling bincraft. Reviewed-on: #241	2026-10-24 00:09:26 +00:00
pat-s	773add042c	perf: use ccache and parallel make, and skip already-built newer-R candidates in bulk (#240)	2026-10-23 13:17:59 +00:00
pat-s	3618168572	fix(ci): pass the metadata DB to per-minor reindexes and fail on errors (#234) ... Step 1 of rpkgs/bincraft#211: pass the metadata database explicitly so bincraft can drop its implicit production defaults. ## Changes - Pass `metadata_db_*` explicitly to every per-minor `upload_package_index()` call in `reindex.yaml`, `process-updates.yaml`, `weekly-rebuild-reindex.yaml`, `build-all-versions.yaml` and `prune-stale-flat-binaries.yaml`. - Replace `|| true` in those per-minor loops with a list of failed minors: every minor is still attempted, then the step exits 1 naming the ones that failed. - Give the "Upload package indexes" step of `build-all-versions.yaml` the `PGPASS` secret. It never had it, so its per-minor reindexes always aborted and `|| true` hid that. - Run the CDN purge in `process-updates.yaml` and `prune-stale-flat-binaries.yaml` even when a reindex failed, as `reindex.yaml` and `weekly-rebuild-reindex.yaml` already do, because the generic index was still replaced. - `local/repair-built-stamp.R` passes the database too, and fails once all minors are tried instead of only printing the error. The generic (non-per-minor) calls are unchanged: they never read the database. ## Behaviour change A per-minor reindex that fails now turns the pipeline red instead of passing silently. Expect build-all-versions runs to start reindexing per-minor slots for real, since they now have the password. ## Ordering Merge this before rpkgs/bincraft's companion PR is released: `local/install-bincraft.R` installs the latest bincraft tag, and the explicit arguments work with both the current and the next release. Reviewed-on: #234	2026-10-17 07:46:09 +00:00
pat-s	6b764d2014	fix(build): build packages requiring newer R on compatible minors (#216)	2026-09-13 09:35:50 +00:00