devxy/build-cran-binaries
Watch
1
0
Fork
You've already forked build-cran-binaries
 0
Code Issues 11 Pull requests Releases Activity

feat(build): skip already-built package versions on workflow restart (#91)
All checks were successful
ci/crow/cron/process-updates-alpine-323-amd64 Pipeline was successful
Details
ci/crow/cron/process-updates-alpine-323-arm64 Pipeline was successful
Details
ci/crow/cron/process-updates-redhat-9-arm64 Pipeline was successful
Details
ci/crow/cron/process-updates-ubuntu-2204-amd64 Pipeline was successful
Details
ci/crow/cron/process-updates-ubuntu-2204-arm64 Pipeline was successful
Details
ci/crow/cron/process-updates-ubuntu-2404-amd64 Pipeline was successful
Details

Browse source 
## Summary

When a `build-all-*` workflow is restarted, the build job re-reads the static `pkgs_to_build.rds` that the install-deps step produced once, so it cycles over every package an interrupted run already built. This adds a DB-based skip filter so a restart only processes what is genuinely left.

- At job start, `build-all.R` queries the `single_builds` metadata table for `(name, tag)` already built successfully (`error_occurred = FALSE`) on this `platform`/`arch`, and drops those pairs from the chunk before the build loop. It logs how many it skipped.
- One indexed query, one round trip, run before the pak forks — no extra S3 listing and no new Python/s3fs memory pressure (`RPostgres`/`DBI` are already used in the container).
- Errored versions are intentionally **not** skipped, so transient failures still get retried on restart.

## Dependency

Correctness depends on a `error_occurred = FALSE` row meaning the binary is actually published. That guarantee is added in rpkgs/bincraft#56 (success row written only after a confirmed S3 upload). This PR should land together with / after a bincraft release including that fix.

Reviewed-on: #91
This commit is contained in:
Patrick Schratz 2026-06-16 07:27:20 +00:00 • committed by Patrick Schratz
commit 1cd86b65e1
1 changed files with 70 additions and 45 deletions
Download patch file Download diff file Expand all files Collapse all files

2
.crow/build-all-versions-amd64.yaml
View file

@ -77,7 +77,7 @@ steps:
echo "=== R-minor-sensitive pass under R $RV ==="
LIB="/mnt/cache/R-pkgs-$RMINOR"
mkdir -p "$LIB"
R_LIBS_USER="$LIB" "$(dirname "$RBIN")/R" -q -e 'if (!requireNamespace("bincraft", quietly = TRUE) || packageVersion("bincraft") != "4.2.1") pak::pak("git::https://codefloe.com/rpkgs/bincraft.git@v4.2.1")' || true
R_LIBS_USER="$LIB" "$(dirname "$RBIN")/R" -q -e 'if (!requireNamespace("bincraft", quietly = TRUE) || packageVersion("bincraft") != "4.2.2") pak::pak("git::https://codefloe.com/rpkgs/bincraft.git@v4.2.2")' || true
R_LIBS_USER="$LIB" $XVFB $XVFB_ARGS -n $SPLIT_INDEX -- "$(dirname "$RBIN")/Rscript" local/build-all.R --sensitive-only $SPLIT_INTO $SPLIT_INDEX $NCPUS 2>&1 || true
done
# archive missed packages