feat(build): skip already-built package versions on workflow restart (#91)
All checks were successful
ci/crow/cron/process-updates-alpine-323-amd64 Pipeline was successful
ci/crow/cron/process-updates-alpine-323-arm64 Pipeline was successful
ci/crow/cron/process-updates-redhat-9-arm64 Pipeline was successful
ci/crow/cron/process-updates-ubuntu-2204-amd64 Pipeline was successful
ci/crow/cron/process-updates-ubuntu-2204-arm64 Pipeline was successful
ci/crow/cron/process-updates-ubuntu-2404-amd64 Pipeline was successful

## Summary

When a `build-all-*` workflow is restarted, the build job re-reads the static `pkgs_to_build.rds` that the install-deps step produced once, so it cycles over every package an interrupted run already built. This adds a DB-based skip filter so a restart only processes what is genuinely left.

- At job start, `build-all.R` queries the `single_builds` metadata table for `(name, tag)` already built successfully (`error_occurred = FALSE`) on this `platform`/`arch`, and drops those pairs from the chunk before the build loop. It logs how many it skipped.
- One indexed query, one round trip, run before the pak forks — no extra S3 listing and no new Python/s3fs memory pressure (`RPostgres`/`DBI` are already used in the container).
- Errored versions are intentionally **not** skipped, so transient failures still get retried on restart.

## Dependency

Correctness depends on a `error_occurred = FALSE` row meaning the binary is actually published. That guarantee is added in rpkgs/bincraft#56 (success row written only after a confirmed S3 upload). This PR should land together with / after a bincraft release including that fix.

Reviewed-on: #91
This commit is contained in:
Patrick Schratz 2026-06-16 07:27:20 +00:00 committed by Patrick Schratz
commit 1cd86b65e1

View file

@ -62,7 +62,7 @@ steps:
GIT_USER: pat-s
R_VERSION: 4.5.3
commands:
- /opt/R/$R_VERSION/bin/R -q -e 'if (!requireNamespace("bincraft", quietly = TRUE) || packageVersion("bincraft") != "4.2.1") pak::pak("git::https://codefloe.com/rpkgs/bincraft.git@v4.2.1", dependencies = TRUE)'
- /opt/R/$R_VERSION/bin/R -q -e 'if (!requireNamespace("bincraft", quietly = TRUE) || packageVersion("bincraft") != "4.2.2") pak::pak("git::https://codefloe.com/rpkgs/bincraft.git@v4.2.2", dependencies = TRUE)'
- /opt/R/$R_VERSION/bin/R -q -e 'packageVersion("bincraft")'
- /opt/R/$R_VERSION/bin/R -q -e 'bincraft::process_unarchived_pkgs(Sys.getenv("CODENAME"), Sys.getenv("ARCH"), workers = 2L)'
backend_options: