devxy/build-cran-binaries
Watch
1
0
Fork
You've already forked build-cran-binaries
 0
Code Issues 10 Pull requests 1 Releases Activity

feat(build): skip already-built package versions on workflow restart #91

Merged
pat-s merged 3 commits from t3code/270bb972 into main 2026-06-16 07:27:21 +00:00
Conversation 0 Commits 3 Files changed 32 +25
1 changed files with 25 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit d668dd96c2 - Show all commits

feat(build): skip already-built package versions on workflow restart

Prev Next 
build-all reads a static pkgs_to_build.rds produced once by the install-deps
step, so a restarted build job re-cycles every package an interrupted run
already produced. Query the build-metadata DB at job start and drop the
(Package, Version) pairs already built successfully for this platform/arch,
so a restart only processes what is genuinely left.

Relies on bincraft writing the success row only after a confirmed S3 upload
(rpkgs/bincraft#56), so a DB success guarantees the binary is published.
Patrick Schratz 2026-06-16 09:04:17 +02:00
commit d668dd96c2
Signed by: pat-s
GPG key ID: 3C6318841EF78925

25
local/build-all.R
View file

@ -25,6 +25,31 @@ sprintf("# of package versions for this job: %s", nrow(chunk))
exclude <- jsonlite::fromJSON("local/excluded-packages.json")[["package"]]
chunk <- chunk[!chunk$Package %in% exclude, ]
# Skip package versions already built in a previous run.
# pkgs_to_build.rds is a static snapshot from the install-deps step, so on a
# restart it still lists everything an interrupted run already produced. The
# metadata DB reflects that progress, so we re-derive the remaining set here.
platform <- paste(Sys.getenv("OS"), gsub("[.]", "", Sys.getenv("OS_VERSION")), sep = "-")
arch <- Sys.getenv("ARCH")
con <- DBI::dbConnect(
RPostgres::Postgres(),
dbname = "build_metadata",
host = "r-binaries.devxy.io",
port = 15432,
user = "rpkgs",
password = Sys.getenv("PGPASS"),
sslmode = "require"
)
built <- DBI::dbGetQuery(
con,
"SELECT name, tag FROM single_builds WHERE platform = $1 AND arch = $2 AND error_occurred = FALSE",
params = list(platform, arch)
)
DBI::dbDisconnect(con)
before <- nrow(chunk)
chunk <- chunk[!paste(chunk$Package, chunk$Version) %in% paste(built$name, built$tag), ]
sprintf("Skipped %d already-built package versions; %d remaining for this job", before - nrow(chunk), nrow(chunk))
# Read pre-computed S3 listing from install-deps step
# This avoids loading s3fs/reticulate/Python in the build container,
# saving significant memory for pak subprocess forks