fix(build): scope the already-attempted skip to the running R minor (#187)
Some checks failed
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/manual/build-all-versions-install-deps/1 Pipeline was successful
ci/crow/manual/build-all-versions/1 Pipeline failed
ci/crow/manual/build-all-versions/3 Pipeline failed
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/manual/build-all-versions/2 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/manual/build-all-versions/4 Pipeline was successful
Some checks failed
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/manual/build-all-versions-install-deps/1 Pipeline was successful
ci/crow/manual/build-all-versions/1 Pipeline failed
ci/crow/manual/build-all-versions/3 Pipeline failed
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/manual/build-all-versions/2 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/manual/build-all-versions/4 Pipeline was successful
## Motivation The run meant to close the 4.6 gap on `amd64/resolute` barely built anything: ``` [1] "Skipped 2334 already-attempted package versions; 59 remaining for this job" ``` `single_builds` records `r_version` per attempt — `store_build_metadata()` both writes and queries it — but the skip query here ignored that column: ```sql SELECT name, tag FROM single_builds WHERE platform = $1 AND arch = $2 ``` So a non-primary pass skipped every package the **primary** pass had already attempted under a different minor. `build-all.R --sensitive-only` running under R 4.6 skipped packages that had only ever been built for 4.5. That is the reason the per-minor slots never fill, and why the backlog cannot be worked off by rebuilding: `amd64/resolute` serves a 4.6 client 22322 packages against the 4.5 slot's 26346. It is also, ultimately, why an R 4.6.1 client got a 4.5-built `rlang` and `undefined symbol: SETLENGTH`. Every other fix in this chain addressed a consequence; this is the cause. ## Change Scope the skip to the R minor the pass is running under. Matched on the `major.minor` prefix rather than the full `r_version` string, so a patch bump (4.6.0 → 4.6.1) does not re-attempt the entire catalogue. Verified the prefix extraction against `4.5.3`, `4.6.0`, `4.4.3` and a bare `4.6`, and that the derivation matches what `store_build_metadata()` records. ## Expected effect The non-primary passes stop skipping wholesale. The first run per slot will be long, since it works off a backlog that has been accumulating for as long as the per-minor slots have existed. ## Verification - `local/build-all.R` parses. - Minor derivation checked under R 4.6.1: `4.6`. - Real effect is only observable from a run; the number to watch is the "Skipped N ... M remaining" line, which should show a far larger `M` for a non-primary pass. Reviewed-on: #187
This commit is contained in:
parent
448349d075
commit
4bf88ed378
1 changed files with 19 additions and 3 deletions
|
|
@ -110,10 +110,25 @@ con <- DBI::dbConnect(
|
|||
password = Sys.getenv("PGPASS"),
|
||||
sslmode = "require"
|
||||
)
|
||||
# Scope the skip to the R minor this pass is running under. `single_builds`
|
||||
# records `r_version` per attempt, but querying without it made a non-primary
|
||||
# pass skip everything the primary pass had already attempted under a different
|
||||
# minor - so `--sensitive-only` under 4.6 skipped packages that had only ever
|
||||
# been built for 4.5, and the per-minor slots never filled. That is why
|
||||
# amd64/resolute served 4000 fewer packages to a 4.6 client than to a 4.5 one.
|
||||
r_minor <- paste(
|
||||
R.version$major,
|
||||
strsplit(R.version$minor, ".", fixed = TRUE)[[1L]][1L],
|
||||
sep = "."
|
||||
)
|
||||
built <- DBI::dbGetQuery(
|
||||
con,
|
||||
"SELECT name, tag FROM single_builds WHERE platform = $1 AND arch = $2",
|
||||
params = list(platform, arch)
|
||||
paste(
|
||||
"SELECT name, tag FROM single_builds",
|
||||
"WHERE platform = $1 AND arch = $2",
|
||||
"AND substring(r_version from '^[0-9]+[.][0-9]+') = $3"
|
||||
),
|
||||
params = list(platform, arch, r_minor)
|
||||
)
|
||||
DBI::dbDisconnect(con)
|
||||
before <- nrow(chunk)
|
||||
|
|
@ -121,8 +136,9 @@ chunk <- chunk[
|
|||
!paste(chunk$Package, chunk$Version) %in% paste(built$name, built$tag),
|
||||
]
|
||||
sprintf(
|
||||
"Skipped %d already-attempted package versions; %d remaining for this job",
|
||||
"Skipped %d package versions already attempted under R %s; %d remaining for this job",
|
||||
before - nrow(chunk),
|
||||
r_minor,
|
||||
nrow(chunk)
|
||||
)
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue