devxy/build-cran-binaries
Watch
1
0
Fork
You've already forked build-cran-binaries
 0
Code Issues 11 Pull requests Releases Activity

fix(build): recompute a stale package snapshot, not just a missing one

Browse source 
The guard asked only whether the snapshot files exist. They describe S3
and CRAN state at the moment they were written, and the cache volume is
per-agent, so an agent that ran an earlier pipeline keeps serving that
pipeline's answer indefinitely.

arm64/alpine324 reported '0 remaining' for both 4.4 and 4.6 from a
snapshot listing 43 r-minor-sensitive packages, while the install-deps
step of the same pipeline had just computed 6871 on another agent. Both
numbers appear in one log: line 49 says 6871, line 99 says 43.

That is why the slot looked finished in minutes while its own index step
dropped 2407 and 2436 packages as missing.

Keyed on the pipeline when the CI exposes an identifier, so a new
pipeline recomputes once per agent and its shards share the result;
otherwise an age check with a two hour default. Verified across five
cases: missing, same pipeline, new pipeline, recent without a CI
variable, and aged out.
This commit is contained in:
Patrick Schratz 2026-08-31 21:55:23 +00:00
commit 85d14ffed5
No known key found for this signature in database
GPG key ID: 62050D5BC68AB6DC
1 changed files with 46 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

48
local/build-all.R
View file

@ -26,9 +26,50 @@ package_cache_files <- c(
"/mnt/cache/packages/r_minor_sensitive_pkgs.rds",
"/mnt/cache/packages/s3_cache.rds"
)
if (!all(file.exists(package_cache_files))) {
# Existence is not freshness. The snapshot describes S3 and CRAN state at the
# moment it was written, and the volume is per-agent, so an agent that ran an
# earlier pipeline keeps serving that pipeline's answer forever. arm64/alpine324
# reported "0 remaining" for both 4.4 and 4.6 from a stale snapshot listing 43
# sensitive packages, while the install-deps step in the very same pipeline had
# just computed 6871 on another agent.
#
# Keyed on the pipeline when the CI exposes one, so a new pipeline recomputes
# once per agent and its shards then share the result. Off CI, or when no such
# variable is set, fall back to an age check.
snapshot_id_path <- "/mnt/cache/packages/snapshot.id"
snapshot_ttl_hours <- as.numeric(
Sys.getenv("PACKAGE_SNAPSHOT_TTL_HOURS", unset = "2")
)
current_snapshot_id <- ""
for (v in c("CI_PIPELINE_NUMBER", "CI_BUILD_NUMBER", "CI_PIPELINE_ID")) {
val <- Sys.getenv(v, unset = "")
if (nzchar(val)) {
current_snapshot_id <- paste(v, val, sep = "=")
break
}
}
snapshot_is_stale <- function() {
if (!all(file.exists(package_cache_files))) {
return(TRUE)
}
if (nzchar(current_snapshot_id)) {
cached <- tryCatch(
readLines(snapshot_id_path, warn = FALSE)[1L],
error = function(e) NA_character_,
warning = function(w) NA_character_
)
return(!identical(cached, current_snapshot_id))
}
age_hours <- as.numeric(
difftime(Sys.time(), file.mtime(package_cache_files[1L]), units = "hours")
)
isTRUE(age_hours > snapshot_ttl_hours)
}
if (snapshot_is_stale()) {
message(
"Package snapshot missing from cache; recomputing via packages-to-build.R"
"Package snapshot missing or stale; recomputing via packages-to-build.R"
)
dir.create("/mnt/cache/packages", showWarnings = FALSE, recursive = TRUE)
save_rds_atomic <- function(obj, path) {
@ -42,6 +83,9 @@ if (!all(file.exists(package_cache_files))) {
pkgs[r_minor_sensitive == TRUE],
"/mnt/cache/packages/r_minor_sensitive_pkgs.rds"
)
if (nzchar(current_snapshot_id)) {
writeLines(current_snapshot_id, snapshot_id_path)
}
message("Package snapshot recomputed.")
}