devxy/build-cran-binaries
Watch
1
0
Fork
You've already forked build-cran-binaries
 0
Code Issues 11 Pull requests Releases Activity

fix(build): hold back packages the cran mirror has not picked up yet

Browse source 
check_for_binary() reads the published version from the cran GitHub
mirror (GET /repos/cran/<pkg>/commits), and that mirror lags CRAN. A
package that has just appeared has no repository there, so the call 404s.

That 404 is permanent, but it is wrapped in purrr::insistently with
max_times = 10 and pause_cap = 60, so it is retried with a backoff of
1, 2, 4, 8, 16, 32, 60, 60, 60, 60 seconds and then aborts the shard.
AsyPeer 0.0.1, published today at 13:50 UTC, took out an entire
resolute build that way.

Release versions published within CRAN_MIRROR_LAG_DAYS (default 3) are
therefore held back. They cost nothing to defer: the daily update
pipeline builds new and updated packages anyway, and they arrive here on
the next run once the mirror has caught up.

Measured against the live CRAN index: 135 of 24831 packages (0.54%) at
three days, including AsyPeer.
This commit is contained in:
Patrick Schratz 2026-08-31 17:01:06 +00:00
commit 46a55254e9
No known key found for this signature in database
GPG key ID: 62050D5BC68AB6DC
1 changed files with 29 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

31
local/packages-to-build.R
View file

@ -68,9 +68,36 @@ archive_versions <- archive_versions[
]
# Now get release versions (assuming cran_release has Package and Version columns)
#
# Packages published in the last few days are held back. `check_for_binary()`
# reads the published version from the `cran` GitHub mirror
# (`GET /repos/cran/<pkg>/commits`), and that mirror lags CRAN: a package that
# has just appeared has no repository there yet. The call then 404s, which is
# permanent, but it is wrapped in `purrr::insistently` and retried ten times
# with a backoff capped at 60s - so one unmirrored package burns about five
# minutes and then aborts the whole shard.
#
# Holding them back costs nothing: the daily update pipeline builds new and
# updated packages anyway, and they arrive here on the next run once the mirror
# has caught up.
mirror_lag_days <- as.numeric(
Sys.getenv("CRAN_MIRROR_LAG_DAYS", unset = "3")
)
published <- as.POSIXct(cran_release$Published, tz = "UTC")
too_recent <- !is.na(published) &
published > (Sys.time() - mirror_lag_days * 86400)
if (any(too_recent)) {
message(sprintf(
"Holding back %d package(s) published in the last %g day(s); the cran GitHub mirror will not have them yet: %s",
sum(too_recent),
mirror_lag_days,
paste(utils::head(cran_release$Package[too_recent], 10L), collapse = ", ")
))
}
release_versions <- data.table(
Package = cran_release$Package,
Version = as.character(cran_release$Version)
Package = cran_release$Package[!too_recent],
Version = as.character(cran_release$Version[!too_recent])
)
pkgs_to_build <- unique(rbind(archive_versions, release_versions, fill = TRUE))