fix(build): hold back packages the cran mirror has not picked up yet #188

Merged

pat-s merged 1 commit from fix/skip-unmirrored-recent-packages into main

2026-08-31 17:12:57 +00:00

pat-s commented

2026-08-31 17:01:10 +00:00

Owner

Motivation

A resolute build aborted on a single package:

[23/361] AsyPeer_0.0.1 (r_minor_sensitive=TRUE)
Error: GitHub API error (404): Not Found
x URL not found: <https://api.github.com/repos/cran/AsyPeer/commits>
Retrying in 2 seconds. ... Retrying in 60 seconds.
Error in `rate_sleep()`: ! Request failed after 10 attempts.
Execution halted

This is not rate limiting — it is a 404. check_for_binary() reads the published version from the cran GitHub mirror, and that mirror lags CRAN. AsyPeer 0.0.1 was published today at 13:50 UTC and has no repository there yet.

The 404 is permanent, but the call is wrapped in purrr::insistently with max_times = 10 and pause_cap = 60, so it retries on a 1/2/4/8/16/32/60/60/60/60 second backoff — about five minutes — and then aborts the whole shard.

Change

Hold back release versions published within CRAN_MIRROR_LAG_DAYS (default 3).

Deferring them costs nothing: the daily update pipeline builds new and updated packages anyway, and they arrive here on the next run once the mirror has caught up.

Measured against the live CRAN index

lag	held back
1 day	29 of 24831 (0.12%)
3 days	135 (0.54%)
7 days	412 (1.66%)

AsyPeer is among the 135 at three days.

Worth doing separately

Retrying a 404 at all is wrong — it can never succeed, and any other permanent 404 (a package pulled from the mirror, say) will abort a shard the same way. check_for_binary() should distinguish a permanent 404 from a transient failure and, when the mirror simply lacks the package, treat the version as unknown rather than fatal. That is a bincraft change and I have not made it here.

## Motivation A resolute build aborted on a single package: ``` [23/361] AsyPeer_0.0.1 (r_minor_sensitive=TRUE) Error: GitHub API error (404): Not Found x URL not found: <https://api.github.com/repos/cran/AsyPeer/commits> Retrying in 2 seconds. ... Retrying in 60 seconds. Error in `rate_sleep()`: ! Request failed after 10 attempts. Execution halted ``` This is not rate limiting — it is a **404**. `check_for_binary()` reads the published version from the `cran` GitHub mirror, and that mirror lags CRAN. `AsyPeer 0.0.1` was published today at 13:50 UTC and has no repository there yet. The 404 is permanent, but the call is wrapped in `purrr::insistently` with `max_times = 10` and `pause_cap = 60`, so it retries on a 1/2/4/8/16/32/60/60/60/60 second backoff — about five minutes — and then aborts the whole shard. ## Change Hold back release versions published within `CRAN_MIRROR_LAG_DAYS` (default 3). Deferring them costs nothing: the daily update pipeline builds new and updated packages anyway, and they arrive here on the next run once the mirror has caught up. ## Measured against the live CRAN index | lag | held back | |---|---| | 1 day | 29 of 24831 (0.12%) | | **3 days** | **135 (0.54%)** | | 7 days | 412 (1.66%) | `AsyPeer` is among the 135 at three days. ## Worth doing separately Retrying a 404 at all is wrong — it can never succeed, and any other permanent 404 (a package pulled from the mirror, say) will abort a shard the same way. `check_for_binary()` should distinguish a permanent 404 from a transient failure and, when the mirror simply lacks the package, treat the version as unknown rather than fatal. That is a bincraft change and I have not made it here.

pat-s added 1 commit

2026-08-31 17:01:10 +00:00

fix(build): hold back packages the cran mirror has not picked up yet 46a55254e9

check_for_binary() reads the published version from the cran GitHub
mirror (GET /repos/cran/<pkg>/commits), and that mirror lags CRAN. A
package that has just appeared has no repository there, so the call 404s.

That 404 is permanent, but it is wrapped in purrr::insistently with
max_times = 10 and pause_cap = 60, so it is retried with a backoff of
1, 2, 4, 8, 16, 32, 60, 60, 60, 60 seconds and then aborts the shard.
AsyPeer 0.0.1, published today at 13:50 UTC, took out an entire
resolute build that way.

Release versions published within CRAN_MIRROR_LAG_DAYS (default 3) are
therefore held back. They cost nothing to defer: the daily update
pipeline builds new and updated packages anyway, and they arrive here on
the next run once the mirror has caught up.

Measured against the live CRAN index: 135 of 24831 packages (0.54%) at
three days, including AsyPeer.

pat-s merged commit 213d30cea4 into main

2026-08-31 17:12:57 +00:00

pat-s deleted branch fix/skip-unmirrored-recent-packages

2026-08-31 17:12:57 +00:00

pat-s referenced this pull request from a commit

2026-08-31 17:12:57 +00:00

fix(build): hold back packages the cran mirror has not picked up yet (#188)