feat(build): skip already-built package versions on workflow restart #91

Merged

pat-s merged 3 commits from t3code/270bb972 into main

2026-06-16 07:27:21 +00:00

pat-s commented

2026-06-16 07:04:35 +00:00

Owner

Summary

When a build-all-* workflow is restarted, the build job re-reads the static pkgs_to_build.rds that the install-deps step produced once, so it cycles over every package an interrupted run already built. This adds a DB-based skip filter so a restart only processes what is genuinely left.

At job start, build-all.R queries the single_builds metadata table for (name, tag) already built successfully (error_occurred = FALSE) on this platform/arch, and drops those pairs from the chunk before the build loop. It logs how many it skipped.
One indexed query, one round trip, run before the pak forks — no extra S3 listing and no new Python/s3fs memory pressure (RPostgres/DBI are already used in the container).
Errored versions are intentionally not skipped, so transient failures still get retried on restart.

Dependency

Correctness depends on a error_occurred = FALSE row meaning the binary is actually published. That guarantee is added in rpkgs/bincraft#56 (success row written only after a confirmed S3 upload). This PR should land together with / after a bincraft release including that fix.

## Summary When a `build-all-*` workflow is restarted, the build job re-reads the static `pkgs_to_build.rds` that the install-deps step produced once, so it cycles over every package an interrupted run already built. This adds a DB-based skip filter so a restart only processes what is genuinely left. - At job start, `build-all.R` queries the `single_builds` metadata table for `(name, tag)` already built successfully (`error_occurred = FALSE`) on this `platform`/`arch`, and drops those pairs from the chunk before the build loop. It logs how many it skipped. - One indexed query, one round trip, run before the pak forks — no extra S3 listing and no new Python/s3fs memory pressure (`RPostgres`/`DBI` are already used in the container). - Errored versions are intentionally **not** skipped, so transient failures still get retried on restart. ## Dependency Correctness depends on a `error_occurred = FALSE` row meaning the binary is actually published. That guarantee is added in rpkgs/bincraft#56 (success row written only after a confirmed S3 upload). This PR should land together with / after a bincraft release including that fix.

pat-s added 1 commit

2026-06-16 07:04:35 +00:00

feat(build): skip already-built package versions on workflow restart d668dd96c2

build-all reads a static pkgs_to_build.rds produced once by the install-deps
step, so a restarted build job re-cycles every package an interrupted run
already produced. Query the build-metadata DB at job start and drop the
(Package, Version) pairs already built successfully for this platform/arch,
so a restart only processes what is genuinely left.

Relies on bincraft writing the success row only after a confirmed S3 upload
(rpkgs/bincraft#56), so a DB success guarantees the binary is published.

pat-s added 1 commit

2026-06-16 07:22:09 +00:00

chore(ci): bump pinned bincraft v4.1.1 -> v4.2.2 in update/rebuild workflows ee8ef1833a

Moves the process-updates-*, weekly-rebuild-missing-* and archive-missed
workflows onto bincraft 4.2.2, which records a build's success metadata only
after the S3 upload is confirmed (rpkgs/bincraft#56). Keeps these pipelines
consistent with the metadata-after-upload behavior the restart-skip filter
relies on.

pat-s added 1 commit

2026-06-16 07:25:22 +00:00

Merge origin/main into t3code/270bb972 9eeb6f5b9a

Resolve .crow conflicts by taking main's content (which moved bincraft pins
to 4.2.1 and added per-minor build/install loops) and re-bumping every
bincraft pin to 4.2.2, so all pinned pipelines use the metadata-after-upload
release. build-all.R auto-merged: the restart skip-filter coexists with
main's r-minor-sensitive build path.

pat-s merged commit 1cd86b65e1 into main

2026-06-16 07:27:21 +00:00

pat-s deleted branch t3code/270bb972

2026-06-16 07:27:21 +00:00

pat-s referenced this pull request from a commit

2026-06-16 07:27:21 +00:00

feat(build): skip already-built package versions on workflow restart (#91)

~~pat-s referenced this pull request 2026-06-16 07:53:39 +00:00~~

refactor(ci): consolidate per-platform crow workflows into 3 matrix files #92