fix(build-all): exclude previously-errored versions in prefilter (#113)
Some checks failed
ci/crow/cron/process-updates/15 Pipeline was canceled

## Motivation

Build jobs were cycling through hundreds of packages that were only ever printed as `Skipping … due to previous build error recorded in metadata DB`, wasting wall-clock on per-package preparation before dropping each one.

## Cause

The prefilter query in `local/build-all.R` selected only successfully-built versions (`error_occurred = FALSE`) into `built`, so line 112 removed only those from the chunk.
Every previously-errored version stayed in the work list and was walked one-by-one, each hitting the internal skip in `build_binary_package()`.
This also explains the misleading `Skipped 0 already-built package versions` line for alphabetical chunks whose leading packages only have error records.

## Changes

- `local/build-all.R`: drop the `AND error_occurred = FALSE` clause so `built` holds every version already attempted (built or errored) for this platform/arch; the existing filter then removes all of them up front.
- Rename the log line to `already-attempted` so the reported count reflects successes and errors.
- Update the surrounding comment to explain why errored versions are excluded.

## Behaviour change

Previously-errored versions are now dropped before the build loop instead of being iterated and individually skipped.
No package that would otherwise build is affected, `build_binary_package()` already skipped these internally.
Retrying errored versions is out of scope and would need a separate opt-in flag on both the prefilter and the in-loop skip.

Reviewed-on: #113
This commit is contained in:
Patrick Schratz 2026-07-13 09:28:45 +00:00 committed by Patrick Schratz
commit ffc2319558

View file

@ -64,10 +64,13 @@ sprintf("# of package versions for this job: %s", nrow(chunk))
exclude <- jsonlite::fromJSON("local/excluded-packages.json")[["package"]]
chunk <- chunk[!chunk$Package %in% exclude, ]
# Skip package versions already built in a previous run.
# Skip package versions already attempted in a previous run (built or errored).
# pkgs_to_build.rds is a static snapshot from the install-deps step, so on a
# restart it still lists everything an interrupted run already produced. The
# metadata DB reflects that progress, so we re-derive the remaining set here.
# We exclude *all* attempted versions, not just successful ones: a previously
# errored version is skipped by build_binary_package() anyway, so leaving it in
# the chunk only makes the job cycle through it one-by-one for no benefit.
# Derive platform + arch from the running container, mirroring the codename ->
# platform mapping bincraft uses internally. The OS/OS_VERSION selectors are
# workflow-level CI variables that are not injected into the container
@ -104,13 +107,13 @@ con <- DBI::dbConnect(
)
built <- DBI::dbGetQuery(
con,
"SELECT name, tag FROM single_builds WHERE platform = $1 AND arch = $2 AND error_occurred = FALSE",
"SELECT name, tag FROM single_builds WHERE platform = $1 AND arch = $2",
params = list(platform, arch)
)
DBI::dbDisconnect(con)
before <- nrow(chunk)
chunk <- chunk[!paste(chunk$Package, chunk$Version) %in% paste(built$name, built$tag), ]
sprintf("Skipped %d already-built package versions; %d remaining for this job", before - nrow(chunk), nrow(chunk))
sprintf("Skipped %d already-attempted package versions; %d remaining for this job", before - nrow(chunk), nrow(chunk))
# Read pre-computed S3 listing from install-deps step
# This avoids loading s3fs/reticulate/Python in the build container,