devxy/build-cran-binaries
Watch
1
0
Fork
You've already forked build-cran-binaries
 0
Code Issues 11 Pull requests Releases Activity

feat(build): skip already-built package versions on workflow restart (#91)
All checks were successful
ci/crow/cron/process-updates-alpine-323-amd64 Pipeline was successful
Details
ci/crow/cron/process-updates-alpine-323-arm64 Pipeline was successful
Details
ci/crow/cron/process-updates-redhat-9-arm64 Pipeline was successful
Details
ci/crow/cron/process-updates-ubuntu-2204-amd64 Pipeline was successful
Details
ci/crow/cron/process-updates-ubuntu-2204-arm64 Pipeline was successful
Details
ci/crow/cron/process-updates-ubuntu-2404-amd64 Pipeline was successful
Details

Browse source 
## Summary

When a `build-all-*` workflow is restarted, the build job re-reads the static `pkgs_to_build.rds` that the install-deps step produced once, so it cycles over every package an interrupted run already built. This adds a DB-based skip filter so a restart only processes what is genuinely left.

- At job start, `build-all.R` queries the `single_builds` metadata table for `(name, tag)` already built successfully (`error_occurred = FALSE`) on this `platform`/`arch`, and drops those pairs from the chunk before the build loop. It logs how many it skipped.
- One indexed query, one round trip, run before the pak forks — no extra S3 listing and no new Python/s3fs memory pressure (`RPostgres`/`DBI` are already used in the container).
- Errored versions are intentionally **not** skipped, so transient failures still get retried on restart.

## Dependency

Correctness depends on a `error_occurred = FALSE` row meaning the binary is actually published. That guarantee is added in rpkgs/bincraft#56 (success row written only after a confirmed S3 upload). This PR should land together with / after a bincraft release including that fix.

Reviewed-on: #91
This commit is contained in:
Patrick Schratz 2026-06-16 07:27:20 +00:00 • committed by Patrick Schratz
commit 1cd86b65e1
1 changed files with 70 additions and 45 deletions
Download patch file Download diff file Expand all files Collapse all files

2
.crow/weekly-rebuild-missing-ubuntu-2404-amd64.yaml
View file

@ -39,7 +39,7 @@ steps:
- git clone -q https://pat-s:$$REPO_RO_TOKEN@git.devxy.io/devxy/build-cran-binaries.git .
- mkdir -p /mnt/cache/pkgcache /mnt/cache/R-pkgs /mnt/cache/ccache /mnt/cache/packages
- rm -rf /mnt/cache/R-pkgs/00LOCK-*
- /opt/R/$R_VERSION/bin/R -q -e 'if (!requireNamespace("bincraft", quietly = TRUE) || packageVersion("bincraft") != "4.2.1") pak::pak("git::https://codefloe.com/rpkgs/bincraft.git@v4.2.1")'
- /opt/R/$R_VERSION/bin/R -q -e 'if (!requireNamespace("bincraft", quietly = TRUE) || packageVersion("bincraft") != "4.2.2") pak::pak("git::https://codefloe.com/rpkgs/bincraft.git@v4.2.2")'
- /opt/R/$R_VERSION/bin/R -q -e 'packageVersion("bincraft")'
- XVFB=$(command -v xwfb-run 2>/dev/null || command -v xvfb-run); XVFB_ARGS=""; if command -v xwfb-run >/dev/null 2>&1; then dnf install -y -q weston 2>/dev/null; XVFB_ARGS="-c weston"; fi
- /opt/R/$R_VERSION/bin/R -q -e 'pak::pak("httr2")'