Compare commits

..
Author SHA1 Message Date
aba2063ea0 feat(edge): gate per-minor routing on published minors and add a staging zone (#175)
Some checks are pending
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline is running
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
## Motivation

`UNION_SLOTS` is empty, so per-minor routing has never been exercised end to end. Before it can be enabled and advertised, two things were missing: a way to test it without pointing production at it, and evidence that the published indexes actually support it.

Verifying the data first turned up a defect that would have broken users the moment the flag was flipped.

## The defect

`contribPath()` redirects to `contrib/<minor>/` whenever the User-Agent carries any R minor, with no existence check and no fallback:

```ts
const rMinor = extractRMinor(userAgent);
return rMinor ? `/${slot}/latest/src/contrib/${rMinor}/${rest}` : flat;
```

Only `4.4`, `4.5` and `4.6` are published. `4.3` and `4.2` return 404 on all 16 slots. With `UNION_SLOTS` set, an R 4.3 client would be redirected to a non-existent index and see **zero** packages: a silent, total failure rather than a degraded one. R 4.3 is still advertised as supported on the website and in `docs/configuration.mdoc`, though `build-env-images` now pins only 4.6.0/4.5.3/4.4.3.

## Changes

- **Gate routing on `KNOWN_MINORS`** (default `4.4,4.5,4.6`), falling back to the flat index for anything else. Unknown minor now behaves exactly as today.
- **Honour `EXTRA_PUBLIC_HOSTS`.** `publicCdnOrigin()` falls back to the hardcoded `PUBLIC_CDN_ORIGIN` for any hostname not in `PUBLIC_CDN_HOSTS`, so a staging zone on a `b-cdn.net` hostname would redirect into *production* and silently measure the wrong system. This lets the identical script run on staging and redirect within itself.
- **Add the `cran-rpkgs-test` pull zone** with `UNION_SLOTS` pre-enabled for all 16 slots, same B2 origin, served on the bunny default hostname so it needs no DNS record and is never advertised.
- **Add `scripts/verify-r-minor-routing.sh`**, covering every `<arch>/<os>` slot: index reachability per minor, the union property against flat, `Path:` target resolution, coverage parity across minors, and with `--live` the real User-Agent routing, the non-R User-Agent case, and that tarballs are never rewritten.
- **Cover the fallback in the edge test suite** for both an unpublished minor (4.3) and a future one (4.7).

## Findings from the full run

112 passed, 16 failed across the 16 slots. Every failure is the same: no R 4.3 index.

All 16 slots carry union indexes that are supersets of flat, every sampled `Path:` target resolves, and all indexes were republished within minutes of each other, so the build side is healthy.

Coverage is **not** yet even, which is why "full coverage for ABI-sensitive packages" is not a claim to make yet:

| slot | flat | 4.4 | 4.5 | 4.6 |
|---|---|---|---|---|
| amd64/resolute | 24305 | 24402 | 24748 | 24395 |
| amd64/alpine324 | 24397 | 24457 | 24744 | 24448 |
| amd64/noble | 24780 | 24805 | 24805 | 24805 |

On the R 4.5-built distros (`resolute`, `alpine324`, and their arm64 twins) a 4.4 or 4.6 client sees ~300 fewer packages than a 4.5 client. On `noble`/`jammy`/`rhel9`/`alpine323` the spread is under 5. The new parity check encodes this with a configurable `PARITY_TOLERANCE`.

## Verification

- `just edge-test`: 18 steps pass. The two new steps were confirmed to fail with the `KNOWN_MINORS` gate removed and pass with it.
- `tofu validate`: passes. **Not applied** - no bunny.net or state credentials were available, so the staging zone still needs a `tofu apply`.
- `scripts/verify-r-minor-routing.sh`: full 16-slot run, results above.
- `shellcheck`: clean.

## Not done here

Applying the staging zone, then running `BASE=https://cran-rpkgs-test.b-cdn.net scripts/verify-r-minor-routing.sh --live` against it. Production `UNION_SLOTS` is deliberately left empty.

Reviewed-on: #175
2026-08-30 21:20:16 +00:00
eeebef8edb
fix(patches): support RcppParallel 6.2.1
All checks were successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
This change will:\n\n- Relax the TBB include hunk context to tolerate the upstream TBB_CXXFLAGS block.\n- Pin the source patch to the verified RcppParallel version.
2026-08-30 14:26:37 +00:00
b0d58f3f7c feat(cdn): derive Ubuntu routes from codenames (#174)
All checks were successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
## Motivation

Ubuntu release codenames cannot be derived from version numbers, so the edge router currently needs a code change for every Ubuntu release.

The generic repository setup can already read `VERSION_CODENAME` from `/etc/os-release` and send it in the user agent.

## Changes

- Prefer a strictly validated `codename=<name>` token for Ubuntu slot routing.
- Retain the existing version-to-codename table for clients using the previous user-agent format.
- Add a future Ubuntu 28.04 regression case whose codename does not exist in the middleware table.

## User-agent format

`R/4.6.1 (Ubuntu 26.04; codename=resolute) (aarch64-unknown-linux-gnu)`

## Verification

- `just edge-test`, 1 test with 16 steps passed.
- `git diff --check`

Reviewed-on: #174
2026-08-30 12:25:22 +00:00
a8820e6e90 fix(cdn): route Ubuntu 26.04 to resolute (#173)
## Motivation

The generic `https://cran.rpkgs.com/` repository falls back to upstream CRAN on Ubuntu 26.04 because the edge router does not recognize its release version.

This causes Resolute users to download and compile source packages even when matching binaries exist.

## Changes

- Map Ubuntu 26.04 user agents to the `resolute` repository slot.
- Cover ARM64 Ubuntu 26.04 routing with an edge middleware regression test.

## Verification

- `just edge-test`, 1 test with 15 steps passed.
- Confirmed the current live generic route redirects the Resolute user agent to upstream CRAN.
- Confirmed the explicit `amd64/resolute/latest` repository installs `rlang` as a binary in `reg.devxy.io/r/r-ubuntu:4.6-resolute`.

Reviewed-on: #173
2026-08-30 12:19:52 +00:00
77a2f1f04a fix(ci): install RPostgres for audit workflows (#172)
All checks were successful
ci/crow/manual/weekly-rebuild-missing/49 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/23 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/24 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/51 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/16 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/28 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/29 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/34 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/35 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/36 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/40 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/30 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/17 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/46 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/18 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/48 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/47 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/42 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/41 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/52 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/53 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/54 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/5 Pipeline was successful
## Summary

- Install RPostgres before running the missing-binaries audit.
- Install RPostgres before running weekly patch proposal and automatic patch workflows.

## Validation

- `prek run --files .crow/auto-apply-patches.yaml .crow/weekly-audit-missing.yaml .crow/weekly-patch-proposals.yaml`
- `crow lint .crow/`
- `git diff --check`

Reviewed-on: #172
2026-08-30 07:31:28 +00:00
e9782bb529
fix(ci): install RPostgres for metadata updates
Some checks failed
ci/crow/cron/weekly-rebuild-missing/44 Pipeline was canceled
ci/crow/cron/weekly-rebuild-missing/40 Pipeline was canceled
ci/crow/cron/weekly-rebuild-missing/43 Pipeline was canceled
ci/crow/cron/weekly-rebuild-missing/45 Pipeline was canceled
ci/crow/cron/process-updates/2 Pipeline was canceled
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/manual/weekly-audit-missing/1 Pipeline was successful
ci/crow/manual/weekly-audit-missing/2 Pipeline was successful
ci/crow/manual/weekly-audit-missing/3 Pipeline was successful
ci/crow/manual/weekly-audit-missing/4 Pipeline was successful
ci/crow/manual/weekly-audit-missing/5 Pipeline was successful
ci/crow/manual/weekly-audit-missing/11 Pipeline was successful
ci/crow/manual/weekly-audit-missing/13 Pipeline was successful
ci/crow/manual/weekly-audit-missing/15 Pipeline was successful
ci/crow/manual/weekly-audit-missing/17 Pipeline was successful
ci/crow/manual/weekly-audit-missing/9 Pipeline was successful
ci/crow/manual/weekly-audit-missing/7 Pipeline failed
ci/crow/manual/weekly-audit-missing/6 Pipeline was successful
ci/crow/manual/weekly-audit-missing/8 Pipeline failed
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/47 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/46 Pipeline was successful
ci/crow/manual/weekly-audit-missing/10 Pipeline was successful
ci/crow/manual/weekly-audit-missing/16 Pipeline was successful
ci/crow/manual/weekly-audit-missing/14 Pipeline was successful
ci/crow/manual/weekly-audit-missing/18 Pipeline was successful
ci/crow/manual/weekly-audit-missing/12 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/48 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/16 Pipeline failed
ci/crow/cron/process-updates/3 Pipeline was successful
2026-08-28 08:48:58 +00:00
automation-bot
d2333c6cbe chore(deps): update terraform bunnynet to v0.18.2
Some checks failed
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline failed
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline failed
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
2026-08-27 00:33:08 +00:00
automation-bot
f4ab6f9dc5 chore(deps): update pre-commit hook editorconfig-checker/editorconfig-checker to v3.11.2
Some checks failed
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline failed
ci/crow/cron/process-updates/8 Pipeline failed
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
2026-08-26 00:32:02 +00:00
automation-bot
d1da0c6cb4 chore(deps): update terraform bunnynet to v0.18.1
Some checks failed
ci/crow/cron/weekly-rebuild-missing/47 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/48 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/16 Pipeline failed
ci/crow/cron/weekly-rebuild-missing/21 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/20 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/7 Pipeline failed
ci/crow/cron/weekly-rebuild-missing/8 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/9 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/3 Pipeline failed
ci/crow/cron/weekly-rebuild-missing/2 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/3 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/1 Pipeline failed
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline is running
ci/crow/cron/process-updates/5 Pipeline failed
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline failed
ci/crow/cron/process-updates/8 Pipeline failed
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline failed
2026-08-22 00:32:09 +00:00
automation-bot
c7b4dca6e2 chore(deps): lock file maintenance
Some checks failed
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline failed
ci/crow/cron/process-updates/12 Pipeline failed
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline failed
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
2026-08-17 00:31:56 +00:00
50495f5c3b
Revert "fix(ci): split oversized weekly rebuild matrix"
Some checks failed
ci/crow/cron/weekly-rebuild-missing/41 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/42 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/40 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/14 Pipeline failed
ci/crow/cron/weekly-rebuild-missing/45 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/43 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/44 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/15 Pipeline failed
ci/crow/cron/weekly-rebuild-missing/48 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/47 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/46 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/16 Pipeline failed
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline failed
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
This reverts commit b75fd2f1c4.
2026-08-14 07:01:56 +00:00
b75fd2f1c4
fix(ci): split oversized weekly rebuild matrix
All checks were successful
ci/crow/cron/process-updates/9 Pipeline was successful
2026-08-14 06:56:41 +00:00
132d1d2d3c
docs(ci): clarify parallel manual matrix runs 2026-08-14 06:44:10 +00:00
automation-bot
706fd10d79 chore(deps): update terraform bunnynet to ~> 0.18
All checks were successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
2026-08-14 00:32:00 +00:00
9bded261ee fix(cdn): restore Alliance pull-zone hostname (#166)
All checks were successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
## Motivation

Applying #165 recreated the Alliance SwissPass pull zone without its custom hostname because the hostname association was not represented in OpenTofu.
The recreated zone also received a new numeric ID, making the weekly purge configuration stale.

## Changes

- Manage `cran.allianceswisspass.devxy.io` as a pull-zone hostname with TLS and forced HTTPS.
- Resolve the Alliance pull-zone ID from its hostname before purging instead of persisting a replaceable numeric ID.
- Install `jq` in the purge step for the Bunny API lookup.

## Verification

- Targeted `prek` hooks pass.
- `tofu validate` passes.
- `crow lint .crow/` passes.
- `just edge-test` passes all 14 routing steps.
- `bash -n scripts/purge_cdn_zone.sh` passes.

## Deployment

Run `tofu apply` to restore the Alliance hostname on the recreated pull zone.

Reviewed-on: #166
2026-08-13 14:13:04 +00:00
a1c1f5e78f fix(cdn): align repository routing across pull zones (#165)
## Motivation

`cran.rpkgs.com` and `cran.allianceswisspass.devxy.io` serve the same B2 repository through separate Bunny pull zones, but only the first zone was managed and purged after weekly reindexing.
This allowed the Alliance endpoint to retain stale repository metadata and left locked `renv` restores unable to retrieve versions whose binary archive object was absent.

## Changes

- Adopt the Alliance SwissPass pull zone `3265648` into OpenTofu and configure it with the shared B2 origin and middleware script.
- Purge both Bunny pull zones after the weekly rebuild reindex.
- Preserve the requested public hostname in middleware redirects.
- Redirect missing archived binaries to the corresponding CRAN source package, checking whether the version is archived or still current.
- Cover the existing archived-binary passthrough behavior in the edge routing matrix.

## Verification

- `prek run -a`
- `just edge-test`
- `crow lint .crow/`
- `tofu validate`
- `bash -n scripts/purge_cdn_zone.sh`

## Deployment

Run `tofu apply` to adopt pull zone `3265648`, publish the middleware release, and align both pull zones.
After the apply, rerun the Alliance SwissPass CI restore that requested `cli 3.6.5` and `AzureStor 3.7.1`.

Reviewed-on: #165
2026-08-13 14:08:10 +00:00
aa4c95457f fix(rebuild): harden split workflow setup (#164)
All checks were successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/54 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/52 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/53 Pipeline was successful
ci/crow/manual/weekly-rebuild-reindex/18 Pipeline was successful
## Motivation

Weekly rebuild shards can all hit a transient CRAN DNS/index outage at once, and the dependent CDN purge always fails because it tries to clone over the checkout preserved from the re-index step.

## Changes

- Retry `uvr add` resolution up to four times with bounded backoff.
- Reuse the existing Crow workspace checkout in the CDN purge step.
- Remove the purge step's unused Git package and repository token.

## Validation

- `crow lint .crow/`
- `shellcheck local/uvr-install.sh scripts/purge_cdn_zone.sh`
- `git diff --check`

Reviewed-on: #164
2026-08-13 13:38:28 +00:00
4b7dc28cc8 feat(rebuild): shard the weekly rebuild and make each shard resumable (#163)
Some checks failed
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/weekly-audit-missing/6 Pipeline was successful
ci/crow/cron/weekly-audit-missing/5 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline failed
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/18 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/16 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/17 Pipeline was successful
ci/crow/manual/weekly-rebuild-reindex/6 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/51 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/13 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/14 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/15 Pipeline was successful
ci/crow/manual/weekly-rebuild-reindex/5 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/49 Pipeline was successful
ci/crow/manual/weekly-rebuild-reindex/17 Pipeline failed
ci/crow/manual/weekly-rebuild-missing/50 Pipeline was successful
## Problem

`weekly-rebuild-missing` runs one job per `<os>-<arch>` and walks that slot's list serially in a single `R -q -e` argument.
That was cheap while every source fallback was skipped as "already built".
Since bincraft #105/#106/#107 and #159 the gate works, and the lists are large: 8 917 source-served records on `amd64/alpine324`, 15 023 on `amd64/resolute`.

Pipeline 10910 (`weekly_rebuild_missing:alpine-324-amd64`) ran for two days, reached `[8692/23885] cholera`, and was killed there.

Two failures follow from that shape:

- **No parallelism.** The work is embarrassingly parallel across packages; one job does all of it.
- **No resumability and no clean stopping point.** The loop ends only by exhausting the list, so the only way to stop it is a kill. A restart re-walks from the first entry, paying a CRAN version resolution and an S3 `HEAD` per package before reaching new work. And a kill matches neither `success` nor `failure`, so the `Purge CDN cache` step never ran: the ~4 600 binaries 10910 did publish stayed hidden behind stale edge copies.

## What this changes

**Three shards per slot.** Each of the 18 `OS`/`ARCH` rows gains `SPLIT_INTO`/`SPLIT_INDEX`, mirroring `build-all-versions.yaml`. Cron and manual routing are unchanged: both filters already match on `${OS}-${ARCH}`, so they now match all three shards of a slot.

**`local/rebuild-missing.R`** replaces the ~1 500-character inline one-liner. The slice is interleaved rather than contiguous, because the list is alphabetical and cost clusters by name (`Rcpp*`, `Bioc*`, `rstan*`).

**Resume by re-deriving state from the bucket.** One `s3_dir_info()` listing gives ETags for the slot; a package is outstanding iff its object's ETag equals CRAN's published `MD5sum`, i.e. it is still byte-identical to CRAN's source. That is `check_s3_root_package()` evaluated in bulk. No progress file, no volume, no DB cursor, and correct when a sibling shard or a `process-updates` run completes something concurrently.

It reads ETags rather than the index's `Built` field the way `packages-to-build.R` does, because the index is no longer rewritten until the dependent pipeline runs and so cannot reflect the current run's progress.

Unknown always means "already a binary", never "rebuild it": a multipart ETag, an unreadable CRAN index or an empty listing can never mass-schedule work.

**A 20 h wall-clock budget** per shard. It exits 0, so the re-index and purge always fire and the remainder is picked up next run with no bookkeeping.

**`.crow/weekly-rebuild-reindex.yaml`** takes over re-indexing and the purge, with `depends_on: [weekly-rebuild-missing]` and `runs_on: [success, failure]`. Three shards writing one slot's `PACKAGES` concurrently would race: `update_PACKAGES()` lists the live bucket, so an early lister that uploads last publishes an index missing its siblings' work.

## Verification

`crow lint .crow/` passes on all 11 pipelines. `prek run` passes.

19 assertions in `local/tests/test-rebuild-missing.R`, 0 failures, covering the partition (disjoint, covering, deterministic, short lists, out-of-range index) and the outstanding filter (source ETag kept, binary ETag dropped, absent object kept, multipart and missing-from-CRAN treated as built).

One of those tests caught a real bug before it shipped: an empty ETag table indexed to zero length rather than to `NA`, which recycled the result away and reported "nothing to build" — the dangerous direction. Fixed with an explicit `lookup()`.

The filter run against the live `amd64/alpine324` index, using its `MD5sum` column as the ETag (established to match the objects):

```
index packages:               24343
outstanding (filter):          8950
no Built stamp:                8917
filter vs no-Built agreement:  8917 of 8917
outstanding but stamped Built:   33 (version drift vs CRAN)
shard sizes: 2984/2983/2983 (sum 8950, unique 8950)
```

It reproduces the source-served set exactly. The extra 33 are packages whose slot version differs from CRAN's current one, so no object exists at the CRAN version key: correctly outstanding.

## Notes for review

- The 20 h budget is a chosen default, exposed as `REBUILD_BUDGET_HOURS` in the pipeline.
- `depends_on` is file-level, not row-level, so on a full cron run no slot is re-indexed until the slowest of all 54 jobs finishes. The budget bounds that at roughly a day.
- An explicit cancel still skips the re-index. Recovery is to trigger `weekly-rebuild-reindex` on its own.
- The purge runs on every re-index row rather than one designated slot: a cron fires only its own slot's row, so gating on a named slot would leave every other slot unpurged.
- Out of scope: `build-all-versions` still cannot rebuild source fallbacks, because `local/build-all.R:113-122` drops every version with any `single_builds` row, which is precisely the source-fallback set.

Design: `specs/2026-08-12-shard-weekly-rebuild-design.md`
Reviewed-on: #163
2026-08-12 08:30:29 +00:00
17 changed files with 782 additions and 135 deletions

View file

@ -58,7 +58,7 @@ steps:
- git clone -q https://pat-s:$$REPO_RO_TOKEN@git.devxy.io/devxy/build-cran-binaries.git . - git clone -q https://pat-s:$$REPO_RO_TOKEN@git.devxy.io/devxy/build-cran-binaries.git .
- mkdir -p /mnt/cache/R-pkgs - mkdir -p /mnt/cache/R-pkgs
- rm -rf /mnt/cache/R-pkgs/00LOCK-* - rm -rf /mnt/cache/R-pkgs/00LOCK-*
- UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh httr2 jsonlite - UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh RPostgres httr2 jsonlite
- /opt/R/$R_VERSION/bin/Rscript local/propose-patches.R --open-pr --limit $PATCH_LIMIT - /opt/R/$R_VERSION/bin/Rscript local/propose-patches.R --open-pr --limit $PATCH_LIMIT
backend_options: backend_options:
kubernetes: kubernetes:

View file

@ -3,15 +3,15 @@
# Routing is preserved 1:1: # Routing is preserved 1:1:
# - cron: each existing `process-cran-updates-<os>-<arch>` cron fires only # - cron: each existing `process-cran-updates-<os>-<arch>` cron fires only
# its matching matrix row (via the per-row `cron:` name filter). # its matching matrix row (via the per-row `cron:` name filter).
# - manual: pick a target from the `process_cran_updates` dropdown # - manual: pick a target from the `process_cran_updates` dropdown;
# ("all" = every os/arch). # "all" fans out every os/arch as parallel matrix workflows.
# Arch placement is handled by the group label (rpkgs-amd64, rpkgs-arm64). # Arch placement is handled by the group label (rpkgs-amd64, rpkgs-arm64).
variables: variables:
# Gates this pipeline. A manual pipeline creation instantiates every file in # Gates this pipeline. A manual pipeline creation instantiates every file in
# .crow/, and a declared default is applied even when the run never passed # .crow/, and a declared default is applied even when the run never passed
# this variable, so the default must be a value that matches no matrix row. # this variable, so the default must be a value that matches no matrix row.
process_cran_updates: process_cran_updates:
description: "Manual run target: a specific <os>-<arch>, 'all' for every os/arch, or 'none' to run nothing." description: "Manual run target: a specific <os>-<arch>, 'all' to run every os/arch in parallel, or 'none' to run nothing."
options: options:
- none - none
- all - all
@ -203,6 +203,7 @@ steps:
- rm -rf /mnt/cache/R-pkgs/00LOCK-* /mnt/cache/R-pkgs/bincraft - rm -rf /mnt/cache/R-pkgs/00LOCK-* /mnt/cache/R-pkgs/bincraft
- mkdir -p /mnt/cache/uvr/cache /mnt/cache/uvr/packages /mnt/cache/R-pkgs /mnt/cache/ccache /mnt/cache/packages - mkdir -p /mnt/cache/uvr/cache /mnt/cache/uvr/packages /mnt/cache/R-pkgs /mnt/cache/ccache /mnt/cache/packages
- /opt/R/$R_VERSION/bin/Rscript local/install-bincraft.R - /opt/R/$R_VERSION/bin/Rscript local/install-bincraft.R
- UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh RPostgres
- /opt/R/$R_VERSION/bin/R -q -e 'packageVersion("bincraft")' - /opt/R/$R_VERSION/bin/R -q -e 'packageVersion("bincraft")'
# rhel-10 ships xwfb-run (Xwayland) instead of xvfb-run; prefer it and start weston, else fall back to xvfb-run # rhel-10 ships xwfb-run (Xwayland) instead of xvfb-run; prefer it and start weston, else fall back to xvfb-run
- XVFB=$(command -v xwfb-run 2>/dev/null || command -v xvfb-run); XVFB_ARGS=""; if command -v xwfb-run >/dev/null 2>&1; then dnf install -y -q weston 2>/dev/null; XVFB_ARGS="-c weston"; fi - XVFB=$(command -v xwfb-run 2>/dev/null || command -v xvfb-run); XVFB_ARGS=""; if command -v xwfb-run >/dev/null 2>&1; then dnf install -y -q weston 2>/dev/null; XVFB_ARGS="-c weston"; fi
@ -218,6 +219,7 @@ steps:
LIB="/mnt/cache/R-pkgs-$RMINOR" LIB="/mnt/cache/R-pkgs-$RMINOR"
mkdir -p "$LIB" mkdir -p "$LIB"
R_LIBS_USER="$LIB" "$(dirname "$RBIN")/Rscript" local/install-bincraft.R || true R_LIBS_USER="$LIB" "$(dirname "$RBIN")/Rscript" local/install-bincraft.R || true
R_LIBS_USER="$LIB" UVR_R_BIN="$RBIN" local/uvr-install.sh RPostgres || true
R_LIBS_USER="$LIB" $XVFB $XVFB_ARGS -- "$(dirname "$RBIN")/R" -q -e "options(crayon.enabled = TRUE, Ncpus = 4, future.globals.onReference = NULL); bincraft::process_cran_updates(interval = $INTERVAL, platform = '${OS}', process_updated = TRUE, process_new = FALSE, process_removed = FALSE, patches = 'local/patches', r_minor_detection = 'classifier',r_minor_sensitive_only = TRUE, s3_endpoint = 'https://s3.eu-central-003.backblazeb2.com', s3_region = 'eu-central-003', s3_bucket = 'devxy-rpkgs-binaries', s3_access_key_id = Sys.getenv('B2_S3_ACCESS_KEY'), s3_secret_access_key = Sys.getenv('B2_S3_SECRET_KEY'), metadata_db_host = 'r-binaries.devxy.io', metadata_db_name = 'build_metadata', metadata_db_table = 'single_builds', metadata_db_user = 'rpkgs', metadata_db_password = Sys.getenv('PGPASS'), metadata_db_sslmode = 'require', metadata_db_port = 15432, archive = TRUE, upload = TRUE, store_build_metadata = TRUE)" || true R_LIBS_USER="$LIB" $XVFB $XVFB_ARGS -- "$(dirname "$RBIN")/R" -q -e "options(crayon.enabled = TRUE, Ncpus = 4, future.globals.onReference = NULL); bincraft::process_cran_updates(interval = $INTERVAL, platform = '${OS}', process_updated = TRUE, process_new = FALSE, process_removed = FALSE, patches = 'local/patches', r_minor_detection = 'classifier',r_minor_sensitive_only = TRUE, s3_endpoint = 'https://s3.eu-central-003.backblazeb2.com', s3_region = 'eu-central-003', s3_bucket = 'devxy-rpkgs-binaries', s3_access_key_id = Sys.getenv('B2_S3_ACCESS_KEY'), s3_secret_access_key = Sys.getenv('B2_S3_SECRET_KEY'), metadata_db_host = 'r-binaries.devxy.io', metadata_db_name = 'build_metadata', metadata_db_table = 'single_builds', metadata_db_user = 'rpkgs', metadata_db_password = Sys.getenv('PGPASS'), metadata_db_sslmode = 'require', metadata_db_port = 15432, archive = TRUE, upload = TRUE, store_build_metadata = TRUE)" || true
done done
- /opt/R/$R_VERSION/bin/R -q -e 'library(bincraft); upload_package_index(codename = "${OS_ID}", s3_endpoint = "https://s3.eu-central-003.backblazeb2.com", s3_region = "eu-central-003", s3_bucket = "devxy-rpkgs-binaries", s3_access_key_id = Sys.getenv("B2_S3_ACCESS_KEY"), s3_secret_access_key = Sys.getenv("B2_S3_SECRET_KEY"))' - /opt/R/$R_VERSION/bin/R -q -e 'library(bincraft); upload_package_index(codename = "${OS_ID}", s3_endpoint = "https://s3.eu-central-003.backblazeb2.com", s3_region = "eu-central-003", s3_bucket = "devxy-rpkgs-binaries", s3_access_key_id = Sys.getenv("B2_S3_ACCESS_KEY"), s3_secret_access_key = Sys.getenv("B2_S3_SECRET_KEY"))'

View file

@ -3,15 +3,15 @@
# Routing is preserved 1:1: # Routing is preserved 1:1:
# - cron: each existing `weekly-audit-missing-<os>-<arch>` cron fires only # - cron: each existing `weekly-audit-missing-<os>-<arch>` cron fires only
# its matching matrix row (via the per-row `cron:` name filter). # its matching matrix row (via the per-row `cron:` name filter).
# - manual: pick a target from the `weekly_audit_missing` dropdown # - manual: pick a target from the `weekly_audit_missing` dropdown;
# ("all" = every os/arch). # "all" fans out every os/arch as parallel matrix workflows.
# Arch placement is via the group label (rpkgs-amd64, rpkgs-arm64). # Arch placement is via the group label (rpkgs-amd64, rpkgs-arm64).
variables: variables:
# Gates this pipeline. A manual pipeline creation instantiates every file in # Gates this pipeline. A manual pipeline creation instantiates every file in
# .crow/, and a declared default is applied even when the run never passed # .crow/, and a declared default is applied even when the run never passed
# this variable, so the default must be a value that matches no matrix row. # this variable, so the default must be a value that matches no matrix row.
weekly_audit_missing: weekly_audit_missing:
description: "Manual run target: a specific <os>-<arch>, 'all' for every os/arch, or 'none' to run nothing." description: "Manual run target: a specific <os>-<arch>, 'all' to run every os/arch in parallel, or 'none' to run nothing."
options: options:
- none - none
- all - all
@ -147,7 +147,7 @@ steps:
- mkdir -p /mnt/cache/packages /mnt/cache/R-pkgs - mkdir -p /mnt/cache/packages /mnt/cache/R-pkgs
- rm -rf /mnt/cache/R-pkgs/00LOCK-* - rm -rf /mnt/cache/R-pkgs/00LOCK-*
- /opt/R/$R_VERSION/bin/Rscript local/install-bincraft.R - /opt/R/$R_VERSION/bin/Rscript local/install-bincraft.R
- UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh httr2 jsonlite - UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh RPostgres httr2 jsonlite
- /opt/R/$R_VERSION/bin/R -q -e 'source("local/weekly-missing-binaries-audit.R")' - /opt/R/$R_VERSION/bin/R -q -e 'source("local/weekly-missing-binaries-audit.R")'
backend_options: backend_options:
docker: docker:

View file

@ -53,7 +53,7 @@ steps:
- git clone -q https://pat-s:$$REPO_RO_TOKEN@git.devxy.io/devxy/build-cran-binaries.git . - git clone -q https://pat-s:$$REPO_RO_TOKEN@git.devxy.io/devxy/build-cran-binaries.git .
- mkdir -p /mnt/cache/R-pkgs - mkdir -p /mnt/cache/R-pkgs
- rm -rf /mnt/cache/R-pkgs/00LOCK-* - rm -rf /mnt/cache/R-pkgs/00LOCK-*
- UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh httr2 jsonlite - UVR_R_BIN=/opt/R/$R_VERSION/bin/R local/uvr-install.sh RPostgres httr2 jsonlite
- /opt/R/$R_VERSION/bin/Rscript local/propose-patches.R --open-issue - /opt/R/$R_VERSION/bin/Rscript local/propose-patches.R --open-issue
- /opt/R/$R_VERSION/bin/Rscript local/proposal-tracking.R --open-issue - /opt/R/$R_VERSION/bin/Rscript local/proposal-tracking.R --open-issue
backend_options: backend_options:

View file

@ -4,17 +4,11 @@
# - cron: each existing `weekly-rebuild-missing-<os>-<arch>` cron fires only # - cron: each existing `weekly-rebuild-missing-<os>-<arch>` cron fires only
# its matching matrix rows (via the per-row `cron:` name filter), # its matching matrix rows (via the per-row `cron:` name filter),
# which is now all three shards of that slot. # which is now all three shards of that slot.
# - manual: `weekly_rebuild_missing` dropdown, default "all" (matches the # - manual: pick a target from the `weekly_rebuild_missing` dropdown;
# previous bare manual trigger that ran every os/arch); pick a # "all" fans out every os/arch and shard as parallel matrix
# single <os>-<arch> to run just one. # workflows, while a single <os>-<arch> runs its three shards.
# Arch placement is handled by the group label (rpkgs-amd64, rpkgs-arm64). # Arch placement is handled by the group label (rpkgs-amd64, rpkgs-arm64).
# #
# 9 OS versions x 2 arches x 3 shards = 54 rows. Crow counts the *declared*
# matrix against CROW_MAX_MATRIX_SIZE before any `when:` gate is applied, so a
# single-slot manual run expands all 54 too. The server default is 50 and was
# raised for this; `crow lint` does not check the limit, so adding an OS
# version here is only caught when a pipeline is triggered.
#
# The shard picks up its own slice and re-derives what is still outstanding # The shard picks up its own slice and re-derives what is still outstanding
# from the bucket, so a restart resumes rather than replaying; see # from the bucket, so a restart resumes rather than replaying; see
# local/rebuild-missing.R. # local/rebuild-missing.R.
@ -27,7 +21,7 @@ variables:
# .crow/, and a declared default is applied even when the run never passed # .crow/, and a declared default is applied even when the run never passed
# this variable, so the default must be a value that matches no matrix row. # this variable, so the default must be a value that matches no matrix row.
weekly_rebuild_missing: weekly_rebuild_missing:
description: "Manual run target: a specific <os>-<arch>, 'all' for every os/arch, or 'none' to run nothing." description: "Manual run target: a specific <os>-<arch>, 'all' to run every os/arch in parallel, or 'none' to run nothing."
options: options:
- none - none
- all - all

View file

@ -16,7 +16,7 @@ variables:
# exactly the slots it rebuilt. A manual pipeline creation instantiates every # exactly the slots it rebuilt. A manual pipeline creation instantiates every
# file in .crow/, so the default must match no matrix row. # file in .crow/, so the default must match no matrix row.
weekly_rebuild_missing: weekly_rebuild_missing:
description: "Manual run target: a specific <os>-<arch>, 'all' for every os/arch, or 'none' to run nothing." description: "Manual run target: a specific <os>-<arch>, 'all' to run every os/arch in parallel, or 'none' to run nothing."
options: options:
- none - none
- all - all
@ -173,15 +173,13 @@ steps:
OTEL_R_METRICS_EXPORTER: none OTEL_R_METRICS_EXPORTER: none
BUNNYNET_API_KEY: BUNNYNET_API_KEY:
from_secret: BUNNYNET_API_KEY from_secret: BUNNYNET_API_KEY
REPO_RO_TOKEN: # cran.rpkgs.com and cran.allianceswisspass.devxy.io are on separate
from_secret: REPO_RO_TOKEN # Bunny pull zones, so both must be purged after the shared origin changes.
# All hostnames on the zone share this id, so one purge covers BUNNY_PULLZONES: '3857050 cran.allianceswisspass.devxy.io'
# cran.devxy.io, cran.allianceswisspass.devxy.io and cran.rpkgs.com.
BUNNY_PULLZONE: '3857050'
commands: commands:
- apk add --no-cache -q bash curl git - apk add --no-cache -q bash curl jq
- git clone -q https://pat-s:$$REPO_RO_TOKEN@git.devxy.io/devxy/build-cran-binaries.git . # Crow carries the checkout from the re-index step into this step.
- bash scripts/purge_cdn_zone.sh "$BUNNYNET_API_KEY" "$BUNNY_PULLZONE" - bash scripts/purge_cdn_zone.sh "$BUNNYNET_API_KEY" $BUNNY_PULLZONES
# Runs on every row rather than on one designated slot: a cron fires only # Runs on every row rather than on one designated slot: a cron fires only
# its own slot's row, so gating on a named slot would leave every other # its own slot's row, so gating on a named slot would leave every other
# slot unpurged. A manual "all" run therefore purges the zone 18 times, # slot unpurged. A manual "all" run therefore purges the zone 18 times,

View file

@ -36,7 +36,7 @@ repos:
hooks: hooks:
- id: air-format - id: air-format
- repo: https://github.com/editorconfig-checker/editorconfig-checker - repo: https://github.com/editorconfig-checker/editorconfig-checker
rev: v3.11.1 rev: v3.11.2
hooks: hooks:
- id: editorconfig-checker - id: editorconfig-checker
exclude: ^local/patches/.*\.patch$ exclude: ^local/patches/.*\.patch$

134
.terraform.lock.hcl generated
View file

@ -2,79 +2,79 @@
# Manual edits may be lost in future updates. # Manual edits may be lost in future updates.
provider "registry.opentofu.org/hashicorp/http" { provider "registry.opentofu.org/hashicorp/http" {
version = "3.6.0" version = "3.6.1"
hashes = [ hashes = [
"h1:0n4RBz9zNw6TTddh5+x7E8L2+qzPXNwKhK4uoZ/DUwE=", "h1:7fra+jbUXbG5wMaz5L6RKMBv6gIuenJcBiIww87GoXo=",
"h1:22Ob7lpzMBSqdrCvoFN5EgmhGPHPBovV/9qo0c/Cd+A=", "h1:BzSV3Ie9XMXF7sZHKAS54CzV95v5GBZNhQ4nrprUgfQ=",
"h1:2IRBvmWOYrq/ooaYYn2i86jZb7iIUvlg0KlmOMfDHoQ=", "h1:CkrbSKS+pNVgvP3bMe2WoYHaFCIWJUkCtlC5vyTAdLI=",
"h1:5mucXikk4OcW3un3u94QnMx4AB4Wfih+sXeMd5QxSNk=", "h1:FboJEwgVIRmqUJkjEoSRpfavVCJotUTe1zzT+pBzcV0=",
"h1:5oU7Zm+2gAVGmxqtJ9E8uTudUkYy/DEn/y3IWphdv4k=", "h1:GlXELDLSZrdV3Svx1jjEBAXiJFkkdF/Hgx1qrmRK5hE=",
"h1:5w0R4b1/VSzpqQF1tXXPr/qmaQLPVRXamOmPKWFcTk4=", "h1:VuXFI2IcnZ6t4sDqtvkuIzbPK1CJQa0CkaM0MBuOlSU=",
"h1:AEVeJr8xGmwad+JUUQ833C3x5d4W+W2szF5DfwxYppw=", "h1:WmL2nFQbSzRiDsDiwUbZbBp/cxGQrXrZnB7A4LGSvJU=",
"h1:CPHJ+0zQbS/cX1m55Y90jIOgf1jV3ocUUnqsXAh+9Eg=", "h1:Zdj26awWJ+m8kMoAMhItsIDcDFg81PWgKKJrvNi3WOI=",
"h1:JPewnGDOJudNer5+ghqwXoaJkfot3QRq9uiEYvo+JHU=", "h1:lHvYYIumeZ+KJgCrmhCLnRGzrvNMjSHBTdV24coyMEc=",
"h1:QzbluV2vQLxsJYxjpziQCmPndIoJ/UGS4/UHH/GpwUM=", "h1:pAOYMwA6Zki3ujAbG20b49u1IYXdBz56pW1JHqKdX5U=",
"h1:TjUNbUdqweRBq/ycQ4ixpNkx5qaYwpXEOn9QCpqNZP8=", "h1:qi9GUp2+g69C8zY6Z68u4fWPwcZlDTa/CtdhvPgWbMA=",
"h1:XNbcODP60ajj21N/OO7af8bBg1ltIsYkq9egn7BYbiY=", "h1:w5A3xJ2mowj2wgiE3oNfOI0lFJf5X9IgxOJ6SErMczA=",
"h1:tgrbgmX7WYQz9G9ncgu7TkpVB+RlLjJA/Rvp9KPlZH8=", "h1:xAO03iJyuNGSOqolIcXcofH8cocgUb6Cnzq6yivbWcI=",
"h1:vLxthX/ZWsOZ+aHKbAMqmNKqD0K5f4nJ8ppy0Ioyup0=", "h1:xXigGPwW8MlrB6Br2ce+Bf35BbdzdPKa97T/q/xrrcA=",
"h1:wZOdGBAZkY8OKEPjKz82j1HloAKOmmvtjWyTxM+I110=", "h1:yDYzQ2ncNE9q1288xAgflIPq98bOOYsAb9tq6vkbFzw=",
"zh:0f719fa5426bc883e9fa6abf7f6498e48025edafbc29015e2f5c028f1cca3b9d", "zh:129d7d5944b31f40916b1ca86b31cef65a6b02fd36008809d13c561894bfedb9",
"zh:1b4d7dafefd6c61764b2f9ed6943ceb9a200dee3590d18747e3a5f6b20ce85e0", "zh:24631608288b0bcd35c1fc63dc5839572254d881c0589ebba036be52b2fc04d6",
"zh:1d23a712984866d29f7b07028a4e99c783c71f1a5dddf08bc3d4e7da9d91a1fa", "zh:5a0f100d7eb256463fe5a2aa1a7128391147b2c5fc895ff1b1ef54fc5b8f15ab",
"zh:257d23d58c3bb024b6bc8eb88736eaf912e934ad47c639d0c3c742bddda849a1", "zh:6a8a1126ab9ca61be3b62ec184f6b2e7cbf01cde810acc548cee27d71277b09b",
"zh:479860e1a5468f5e04013b9364c9496d7ed0804bf9a1acd8e07558d57609993d", "zh:6fffef54fd3aada85c074e34d41386aa09c79a308a4679132da31c7272733c6c",
"zh:4cb5e681bf599b411b27c4a2c4066a5fb2ed79aaa3a1a3cb5a30002fec062ce9", "zh:899c992d2aa290ebe1304da0289c5104a630bca421cc6a88ce55bf0960aab1b4",
"zh:4fb35c3f643dae9f3670d719397a415f815a0b95f8ed7bd8a72f27a94ba78092", "zh:960fd6c2847859a843dd9dbfc95a0037a470aa744094d155a38a057175cf1502",
"zh:59ba40825ab38db5b4a0989a2db0df35cc15d8984f898176011ba352f27d77b7", "zh:9b032b685a644634158ace5529e260dfc4447a280056f02858d205ea26753f69",
"zh:61fc1252eb88088638f4c69ea4e2171cde2e5089fa632ac1e943b13787348f73", "zh:bba5477c97020c28ed12d4f5b36be2c1bf14d946d7e44b3690e5c23cd7ddf5e6",
"zh:7c5d6dd5f7cbc460e95d368be35c29b4e0402069b8912dbd5d1cd7fa9acef216", "zh:c2ff6c33efef52441fa3485137972792031626dcabca2b1d8b6527d45f185279",
"zh:7f76d756240d4284642f359ad470226e5378670239aadc366ef54d9d914d4d2e", "zh:cd492b3dfd150de6bef8ad505293d3d53c6c907706f36d0e497b4fc027d8edb6",
"zh:8133ad0814098177e0d067c816ccf1bf48bbadacd18f6f2c808c90447505723b", "zh:d1f832bc33c42781454dc020c6937e7d0133155a5a9f64335309d64a34b36bb7",
"zh:c93be06269bb728f1968f8c50506de56c887017ac1d6e4be1f925651d8437eb6", "zh:d42e9cbebc77643556853b1ebbec14cefe70c57ee86cd3b8c71fbe7f523f07df",
"zh:ef47b78a10a82e6cf53344a6a85a94041c28286c10a70541c564d762f1cfede0", "zh:d4c0466f578d7f990646bb0847e31ba3797f2100b6380ee1ca736887546c7621",
"zh:f5796a53a74999135bd9087aff50fddda59129d09b2f9b1902ff8c0c1e047e48", "zh:d9d81ecebfe6edabdd4c527f3f4debde3e052ff87c5ef4c67497ab3d7539e424",
] ]
} }
provider "registry.terraform.io/bunnyway/bunnynet" { provider "registry.terraform.io/bunnyway/bunnynet" {
version = "0.17.0" version = "0.18.2"
constraints = "~> 0.17" constraints = "~> 0.18"
hashes = [ hashes = [
"h1:+qDt35lVSK7acw6a1xHuPYrqmZEcHSmtd+6n1TxNuYw=", "h1:3rZl+Co3WMpwj8SciPaCNXoGA31aSoqp6iweLarr5m4=",
"h1:1dCu2l4DhPBjizVAH/WwAjT1Xbo52K4PMvHoD5zUhuU=", "h1:6d9cKLhz8QOZ4R5yVX1G0TsWL+K1Abtfbm3xngndxto=",
"h1:Dvn46Auwuel4jqrqZXs2D7kdujNhs17LEmqhuY0k4/4=", "h1:EBjjkfp5Gx7nXP1DVO+tLhsow6fEUvaIjsCEFRT2fY8=",
"h1:M5eDL3m2uSEr1XATJW0foHzKl8pFhCtgKuOM24bJRwU=", "h1:Nu2DoHGOv2YN7ag4kFGpfnPeRDh6bzWqY5anW+ETGpM=",
"h1:PddaC7nM/gY4x9i3xy6TxOs9MAu2/6g58Xs/gv4DRV8=", "h1:OnvZxg28m4/UJeEhHVLU4kM2MZ704sxRzYfLWlLxnhA=",
"h1:QVIKiZluI+NQAKu8NpFBl3Nvyx+d81vW9btEUdIQREc=", "h1:PiCse2/UcB7nkPxosveHsJN/jKdBC8AH6tKTxcHSYKw=",
"h1:S6TnzXHsRoGYvC1vJBkDiVEc0spceksY4n6x5WN5iYw=", "h1:QAahdtlDBUon7eMwNN0D2V6CxgasOXIi+9/UExik6Sg=",
"h1:VcxZDWqCWMSjcUsC1K4sB6uYEoeoou+BC0ePoJXmf3A=", "h1:Su5z0A7/UaSm/E7FJnFjpDVQaa1Ju5+fZ8Mirf8E+k8=",
"h1:W0y/agBVqls1cJlFGFYMu2VnqoPXFzxVHPIYe3OqfYQ=", "h1:UA3a78FJAPAGqCCvlIg9ekPltpVsrmEhwFLalWCFnew=",
"h1:XmNd5fP9a0O77ve5BMQP2vARExgIa7rYl6KvyUYXPSs=", "h1:XAlCTNHRtgUkNjdUItkiak6ajjT7wFJzJN8frXKD5Ms=",
"h1:e0EFKrWSQwaa/kGhnha4DXk4T68Av8QxP84mRSdWC9M=", "h1:ZgLBOPebYxH059z1cGHmjYO8CTf+tbWPb3VbO97S2YM=",
"h1:eM+/lUiU0pNSgQKoqKPgE3xJrJ0MHIpKG+yhaGB/P0M=", "h1:anR91C2F6NDJoQQQIy6KHChodnTaSKnApSWSGM4jSX0=",
"h1:fPWWA4T0/y7GX+tCGN23l1jODhZ3uCdR/MKgZDXYpAE=", "h1:gVmaNmIu4gEiITM+CAb66e+zncAqzNBYkniTZfvxZ5Y=",
"h1:g+r2GVi4gVC4DuQg3PL70gW9BDskgWUzCBIMXTUq63A=", "h1:pODlGrkPqHV4yhXiO7LLLu11HtcuxOAB2zUx3B8w1vI=",
"h1:gaZ8eALDtVHqykVDHav8004gHiMGaYR/3KwET0FUgao=", "h1:qEYeHEKVRcc78q5xiRGJSY8DGQpLj40KafEXUxFfaQc=",
"h1:kbqW25eaiv4N/N/z+sxLdJZ15yh5cgnRD/q6RclPMLc=", "h1:qdVz+O0lLHhyf5YX3ujmoVvAGlKqvi+YOPUzVTqpKzY=",
"h1:rGjxue3mXRyQQqpywTXC4zK//JAtf0Cz7RP+uPMMJjw=", "h1:yTrPkdc9eQkxfPLBYydFf0fpcjarP5w0sdLPzekD9RQ=",
"zh:05943fef14c2028f4722bf078aa1889229e94302f7678cc6f63adb669d8ea612", "zh:0fe3987c927d81196c97504470ce4d26c3ad0014f8ee3d0c1be422d08cfcf49c",
"zh:26a163930a92a7408f7bbd0130064b84df8a232b500d8c6c3989952986308539", "zh:15c36dc69e058876921ac887213e1716217d159b7ee7f0f233e21fb35be85178",
"zh:41305feaaade55391447521ec309f3c038b631ca542907ad95132fab71a7e116", "zh:29d58d7b76dcb142a06d4edd15b8500fe6c1afb7f7c056ada17e2d42bb999fbd",
"zh:606919a930f0299948504adbdcd0f239a8af5c418f85741c48f8add370a3d038", "zh:33d313836c0e985186b3456c0946e062b27cacfcb08611d0a394f36db9ee1aef",
"zh:66963d5b445639511939fc508513fd31da3ee1d4ee1a565ee396c9532897a349", "zh:47e085e52e9b24ad85fa2988dbb8604256a970a6f53f7fa6aab04d8ae756a738",
"zh:6c981ec0c8545556395c43e2511861ab65ee9ecf2a960480e7889c3af0d23af3", "zh:4ba4f87571ca72fbc6c24ab71f2f7b5a086938262e2d8e5c0b39701ed52f8bbc",
"zh:7334a1bdb726ce1f1bf0a3155f30f84f65206980c229c832ff5f0b0718c44e0b", "zh:4c6bae97b543c5b328e1ecbcf7c976351b4b381654e9d3e569270dcab3ba816c",
"zh:75f6c86bf74511e605423332d113711c76c8028361a32282fb3359d6c7ecae9e",
"zh:7aebb1a01cfe8be54903853202ae06eba14ad99c37d230ed93ce7d6633e05e9b",
"zh:890df766e9b839623b1f0437355032a3c006226a6c200cd911e15ee1a9014e9f", "zh:890df766e9b839623b1f0437355032a3c006226a6c200cd911e15ee1a9014e9f",
"zh:9041d0e20c9ceea532de6eebf5cb3a27dad0bb49d3f5b5154be2a08d68fbbf1f", "zh:9ba7ab56537963db2449d217528a751469c9dc4e413dec3e3d63fd7daf3db4ef",
"zh:a6bbf65431a02be4df0ebb1cbe01185ad357ff6e33c01bd0558f59bed90c8f36", "zh:a3c48eda7e11b03b831f2a639797524bb335f155f0dff0e999cf3496994da8b3",
"zh:c6d075a31096f080c388dfe46036f451c0cc114c3311a4f46ab8dbe1938a202f", "zh:aab8f4814d55ef8c6c285d2496ae412437017d0fd1be70106f7b3a4a6e764feb",
"zh:dd8703f7b55b8bc8e10f8718bea889781100b18e932b04898995b63178c3d36e", "zh:b92b9beacf71ae894717c2036ceb68db52c9c43af4a01b8209eceae9f91a2c8e",
"zh:dd92a5cd4e133a4000e7e5bc8cce876ae0ed803543cedd2f3d590661ba244d04", "zh:da389285938e22e1249e6a00cebf12a9f67334743f0b3f66399e6881028bda11",
"zh:e024fdf121bebc48c1e6debea344c6d4f174117f3ae605fca6e13b9705d92d22", "zh:dadcc33d06e6f64a17d1965478af5e8bbdc971e92ec9b14e384c5d43861d63f7",
"zh:ee0e80c31b438e35fa1608f6a2f5824d2806db1e5e8b9f7a90986585c7bcb895", "zh:e090c916e6da685125194af4f0a1fd772494a0c63f3f16ab3741782e17f4a8f9",
"zh:fc2d4b705411b48f8c045981f9368a3ea2f74969dd6302008c31ff0bedd51f0a", "zh:e5881e00fa970c08e66e8079b47d69b76def6e7ff3bdc35b68d7811e5ece55d1",
"zh:eeebb25a066a6287d545c91c0fc264acee5b28174d0979faeebdac3bd14f0fff",
"zh:f368195116c9ce0181aa7527c51ae5e7ab23d42fb966acf4eddca344621ae339",
] ]
} }

176
cdn.tf
View file

@ -32,7 +32,7 @@
# cache_stale = ["offline", "updating"] # cache_stale = ["offline", "updating"]
# use_background_update = true # use_background_update = true
# block_ips = var.cdn_block_ips # block_ips = var.cdn_block_ips
# # 50 TB # # 50 TB
# limit_bandwidth = 50000000000000 # limit_bandwidth = 50000000000000
@ -52,6 +52,26 @@
### cran.rpkgs.com ### cran.rpkgs.com
locals {
rpkgs_slots = [
for pair in setproduct(
["amd64", "arm64"],
["resolute", "noble", "jammy", "rhel8", "rhel9", "rhel10", "alpine323", "alpine324"]
) : "${pair[0]}/${pair[1]}"
]
# The supported R minors: the current one plus the two previous, which is
# exactly what build-env-images installs as R_VERSION_LATEST / PREV1 / PREV2.
# These must stay in step. A minor listed here without a published index
# sends those clients to a 404; a published minor missing from this list
# sends them to CRAN for sources instead of serving the binaries we built.
rpkgs_supported_minors = ["4.4", "4.5", "4.6"]
# bunny.net serves every pull zone on <name>.b-cdn.net, so staging needs no
# DNS record and is never advertised.
rpkgs_test_hostname = "cran-rpkgs-test.b-cdn.net"
}
# The edge middleware that resolves the bare cran.rpkgs.com form to an # The edge middleware that resolves the bare cran.rpkgs.com form to an
# <arch>/<os> slot and routes PACKAGES* to the per-R-minor slot. The source of # <arch>/<os> slot and routes PACKAGES* to the per-R-minor slot. The source of
# truth is edge/rpkgs-router.ts; `tofu apply` publishes a new release. # truth is edge/rpkgs-router.ts; `tofu apply` publishes a new release.
@ -75,6 +95,13 @@ resource "bunnynet_compute_script_variable" "rpkgs_router_union_slots" {
required = false required = false
} }
resource "bunnynet_compute_script_variable" "rpkgs_router_known_minors" {
script = bunnynet_compute_script.rpkgs_router.id
name = "KNOWN_MINORS"
default_value = join(",", local.rpkgs_supported_minors)
required = false
}
resource "bunnynet_pullzone" "cran_rpkgs_com" { resource "bunnynet_pullzone" "cran_rpkgs_com" {
name = "cran-rpkgs" name = "cran-rpkgs"
@ -82,7 +109,7 @@ resource "bunnynet_pullzone" "cran_rpkgs_com" {
cache_expiration_time = 31919000 cache_expiration_time = 31919000
websockets_enabled = false websockets_enabled = false
errorpage_whitelabel = true errorpage_whitelabel = true
origin { origin {
type = "OriginUrl" type = "OriginUrl"
@ -147,6 +174,151 @@ resource "bunnynet_pullzone_hostname" "cran_rpkgs_com" {
tls_enabled = true tls_enabled = true
} }
### Staging zone for edge-router changes
# Every published <arch>/<os> slot. The staging zone enables per-minor routing
# for all of them at once; production adopts the same list only after
# `scripts/verify-r-minor-routing.sh --live` passes against staging.
# A second copy of the same router, bound to the same B2 origin, so UNION_SLOTS
# can be exercised end to end before production is touched.
resource "bunnynet_compute_script" "rpkgs_router_test" {
type = "middleware"
name = "rpkgs-router-test"
content = file("${path.module}/edge/rpkgs-router.ts")
}
resource "bunnynet_compute_script_variable" "rpkgs_router_test_union_slots" {
script = bunnynet_compute_script.rpkgs_router_test.id
name = "UNION_SLOTS"
default_value = join(",", local.rpkgs_slots)
required = false
}
# Without this the staging zone rewrites to PUBLIC_CDN_ORIGIN, so its redirects
# land on production and the test silently measures the wrong system.
resource "bunnynet_compute_script_variable" "rpkgs_router_test_extra_hosts" {
script = bunnynet_compute_script.rpkgs_router_test.id
name = "EXTRA_PUBLIC_HOSTS"
default_value = local.rpkgs_test_hostname
required = false
}
resource "bunnynet_compute_script_variable" "rpkgs_router_test_known_minors" {
script = bunnynet_compute_script.rpkgs_router_test.id
name = "KNOWN_MINORS"
default_value = join(",", local.rpkgs_supported_minors)
required = false
}
resource "bunnynet_pullzone" "cran_rpkgs_test" {
name = "cran-rpkgs-test"
cache_errors = false
cache_expiration_time = 31919000
websockets_enabled = false
errorpage_whitelabel = true
origin {
type = "OriginUrl"
url = "https://devxy-rpkgs-binaries.s3.eu-central-003.backblazeb2.com"
middleware_script = bunnynet_compute_script.rpkgs_router_test.id
}
routing {
filters = [
"scripting",
]
}
s3_auth_enabled = true
s3_auth_key = var.B2_S3_ACCESS_KEY
s3_auth_secret = var.B2_S3_SECRET_KEY
s3_auth_region = "eu-central-003"
cache_enabled = true
request_coalescing_enabled = true
block_post_requests = true
cache_vary_headers = ["User-Agent"]
# Staging carries only synthetic verification traffic, so the production
# ceilings would be pure headroom.
limit_requests = 500
limit_connections = 100
safehop_enabled = true
add_canonical_header = true
cache_stale = ["offline", "updating"]
block_ips = var.cdn_block_ips
# 1 TB
limit_bandwidth = 1000000000000
block_root_path = true
}
# Alliance SwissPass historically used a separate, manually configured pull
# zone. Adopt it so both public repositories use the same B2 origin, middleware
# release and cache behavior.
import {
to = bunnynet_pullzone.cran_allianceswisspass
id = "3265648"
}
resource "bunnynet_pullzone" "cran_allianceswisspass" {
name = "cran-allianceswisspass"
cache_errors = false
cache_expiration_time = 31919000
websockets_enabled = false
errorpage_whitelabel = true
origin {
type = "OriginUrl"
url = "https://devxy-rpkgs-binaries.s3.eu-central-003.backblazeb2.com"
middleware_script = bunnynet_compute_script.rpkgs_router.id
}
routing {
filters = [
"scripting",
]
}
s3_auth_enabled = true
s3_auth_key = var.B2_S3_ACCESS_KEY
s3_auth_secret = var.B2_S3_SECRET_KEY
s3_auth_region = "eu-central-003"
cache_enabled = true
request_coalescing_enabled = true
block_post_requests = true
cache_vary_headers = ["User-Agent"]
limit_requests = 5000
limit_connections = 1000
safehop_enabled = true
add_canonical_header = true
cache_stale = ["offline", "updating"]
block_ips = var.cdn_block_ips
# 50 TB
limit_bandwidth = 50000000000000
block_root_path = true
}
resource "bunnynet_pullzone_hostname" "cran_allianceswisspass" {
pullzone = bunnynet_pullzone.cran_allianceswisspass.id
name = "cran.allianceswisspass.devxy.io"
force_ssl = true
tls_enabled = true
}
# resource "bunnynet_storage_zone" "devxy-r-binaries" { # resource "bunnynet_storage_zone" "devxy-r-binaries" {
# name = "devxy-r-binaries-storage" # name = "devxy-r-binaries-storage"
# region = "DE" # region = "DE"

View file

@ -17,7 +17,12 @@ const UNION_SLOTS = 'amd64/alpine324';
const UA_R45_MUSL = 'R (4.5.3 x86_64-pc-linux-musl x86_64 linux-musl)'; const UA_R45_MUSL = 'R (4.5.3 x86_64-pc-linux-musl x86_64 linux-musl)';
const UA_R46_MUSL = 'R (4.6.0 x86_64-pc-linux-musl x86_64 linux-musl)'; const UA_R46_MUSL = 'R (4.6.0 x86_64-pc-linux-musl x86_64 linux-musl)';
const UA_R43_MUSL = 'R (4.3.3 x86_64-pc-linux-musl x86_64 linux-musl)';
const UA_R47_MUSL = 'R (4.7.0 x86_64-pc-linux-musl x86_64 linux-musl)';
const UA_R45_ALPINE = 'R/4.5.3 R (4.5.3 x86_64-pc-linux-musl x86_64 linux-musl) Alpine Linux 3.24'; const UA_R45_ALPINE = 'R/4.5.3 R (4.5.3 x86_64-pc-linux-musl x86_64 linux-musl) Alpine Linux 3.24';
const UA_R45_RESOLUTE = 'R/4.5.3 (Ubuntu 26.04) (aarch64-unknown-linux-gnu aarch64 linux-gnu)';
const UA_R45_FUTURE_UBUNTU =
'R/4.5.3 (Ubuntu 28.04; codename=dynamic-dugong) (aarch64-unknown-linux-gnu aarch64 linux-gnu)';
const UA_R45_DARWIN = 'R (4.5.1 aarch64-apple-darwin20 aarch64 darwin20)'; const UA_R45_DARWIN = 'R (4.5.1 aarch64-apple-darwin20 aarch64 darwin20)';
const UA_CURL = 'curl/8.0.1'; const UA_CURL = 'curl/8.0.1';
@ -93,6 +98,33 @@ Deno.test('rpkgs-router', async (t) => {
assertEquals(res.location, `https://cran.rpkgs.com${SLOT}/4.6/PACKAGES.gz`); assertEquals(res.location, `https://cran.rpkgs.com${SLOT}/4.6/PACKAGES.gz`);
}); });
// We publish binaries only for the supported window. An excluded minor has
// no slot we can serve safely, so it goes to CRAN for sources rather than
// to a 404 or to binaries built under another minor.
await t.step('sends an excluded R minor to CRAN for the index', async () => {
const res = await probe(`${SLOT}/PACKAGES.gz`, UA_R43_MUSL);
assertEquals(res.location, 'https://cran.r-project.org/src/contrib/PACKAGES.gz');
});
await t.step('sends a future R minor to CRAN too', async () => {
const res = await probe(`${SLOT}/PACKAGES.gz`, UA_R47_MUSL);
assertEquals(res.location, 'https://cran.r-project.org/src/contrib/PACKAGES.gz');
});
// The index and the tarballs R resolves against it have to come from the
// same place. Serving one from CRAN and the other from here would hand R a
// binary where it expects a source tarball.
await t.step('sends an excluded minor to CRAN for tarballs as well', async () => {
const res = await probe(`${SLOT}/foo_1.0.tar.gz`, UA_R43_MUSL);
assertEquals(res.location, 'https://cran.r-project.org/src/contrib/foo_1.0.tar.gz');
});
await t.step('leaves an excluded minor alone on a slot outside UNION_SLOTS', async () => {
const res = await probe(`${OTHER_SLOT}/PACKAGES.gz`, UA_R43_MUSL);
assertEquals(res.location, null);
assertEquals(res.status, 200);
});
await t.step('routes PACKAGES and PACKAGES.rds too', async () => { await t.step('routes PACKAGES and PACKAGES.rds too', async () => {
for (const file of ['PACKAGES', 'PACKAGES.rds']) { for (const file of ['PACKAGES', 'PACKAGES.rds']) {
const res = await probe(`${SLOT}/${file}`, UA_R45_MUSL); const res = await probe(`${SLOT}/${file}`, UA_R45_MUSL);
@ -117,6 +149,13 @@ Deno.test('rpkgs-router', async (t) => {
assertEquals(res.status, 200); assertEquals(res.status, 200);
}); });
await t.step('serves an archived binary when it exists', async () => {
const path = `${SLOT}/Archive/xml2/xml2_1.5.2.tar.gz`;
const res = await probe(path, UA_R45_MUSL);
assertEquals(res.status, 200);
assertEquals(res.location, null);
});
await t.step('does not redirect a path already under a minor', async () => { await t.step('does not redirect a path already under a minor', async () => {
const res = await probe(`${SLOT}/4.5/PACKAGES.gz`, UA_R45_MUSL); const res = await probe(`${SLOT}/4.5/PACKAGES.gz`, UA_R45_MUSL);
assertEquals(res.location, null); assertEquals(res.location, null);
@ -134,6 +173,16 @@ Deno.test('rpkgs-router', async (t) => {
assertEquals(res.location, `https://cran.rpkgs.com${SLOT}/4.5/PACKAGES.gz`); assertEquals(res.location, `https://cran.rpkgs.com${SLOT}/4.5/PACKAGES.gz`);
}); });
await t.step('resolves Ubuntu 26.04 to the resolute slot', async () => {
const res = await probe('/src/contrib/PACKAGES.gz', UA_R45_RESOLUTE);
assertEquals(res.location, 'https://cran.rpkgs.com/arm64/resolute/latest/src/contrib/PACKAGES.gz');
});
await t.step('resolves a future Ubuntu release from its codename', async () => {
const res = await probe('/src/contrib/PACKAGES.gz', UA_R45_FUTURE_UBUNTU);
assertEquals(res.location, 'https://cran.rpkgs.com/arm64/dynamic-dugong/latest/src/contrib/PACKAGES.gz');
});
await t.step('sends an unidentifiable distro to CRAN', async () => { await t.step('sends an unidentifiable distro to CRAN', async () => {
const res = await probe('/src/contrib/PACKAGES.gz', UA_R45_MUSL); const res = await probe('/src/contrib/PACKAGES.gz', UA_R45_MUSL);
assertEquals(res.location, 'https://cran.r-project.org/src/contrib/PACKAGES.gz'); assertEquals(res.location, 'https://cran.r-project.org/src/contrib/PACKAGES.gz');

View file

@ -27,6 +27,17 @@ import * as BunnySDK from 'https://esm.sh/@bunny.net/edgescript-sdk@0.12';
const PUBLIC_CDN_ORIGIN = 'https://cran.rpkgs.com'; const PUBLIC_CDN_ORIGIN = 'https://cran.rpkgs.com';
const CRAN_ORIGIN = 'https://cran.r-project.org'; const CRAN_ORIGIN = 'https://cran.r-project.org';
const PUBLIC_CDN_HOSTS = new Set([
'cran.rpkgs.com',
'cran.allianceswisspass.devxy.io',
// Staging hostnames, so the identical script can run on a test pull zone and
// redirect within itself. Without this a test zone rewrites to
// PUBLIC_CDN_ORIGIN, quietly exercising production instead of itself.
...(Deno.env.get('EXTRA_PUBLIC_HOSTS') ?? '')
.split(',')
.map((host) => host.trim())
.filter((host) => host.length > 0),
]);
/** Slots ("<arch>/<os>", comma separated) whose per-minor index is a union. */ /** Slots ("<arch>/<os>", comma separated) whose per-minor index is a union. */
const UNION_SLOTS = new Set( const UNION_SLOTS = new Set(
@ -36,6 +47,21 @@ const UNION_SLOTS = new Set(
.filter((slot) => slot.length > 0), .filter((slot) => slot.length > 0),
); );
/**
* R minors for which a per-minor index is actually published.
*
* contribPath() has no way to probe the origin, so a minor that is not
* published here must fall back to the flat index. Routing an unlisted minor
* would send that client to a 404 and it would see no packages at all - a
* silent, total failure rather than a degraded one.
*/
const KNOWN_MINORS = new Set(
(Deno.env.get('KNOWN_MINORS') ?? '4.4,4.5,4.6')
.split(',')
.map((minor) => minor.trim())
.filter((minor) => minor.length > 0),
);
/** `/<arch>/<os>/latest/src/contrib[/<rest>]` */ /** `/<arch>/<os>/latest/src/contrib[/<rest>]` */
const SLOT_PATH_REGEX = /^\/(amd64|arm64)\/([a-z0-9._-]+)\/latest\/src\/contrib\/?(.*)$/; const SLOT_PATH_REGEX = /^\/(amd64|arm64)\/([a-z0-9._-]+)\/latest\/src\/contrib\/?(.*)$/;
@ -47,13 +73,19 @@ const INDEX_FILE_REGEX = /^PACKAGES(\.gz|\.rds)?$/;
const SRC_CONTRIB_REGEX = /^\/src\/contrib\/(.+)$/; const SRC_CONTRIB_REGEX = /^\/src\/contrib\/(.+)$/;
/** A binary archive URL whose upstream source counterpart CRAN can serve. */
const ARCHIVE_TARBALL_REGEX =
/^\/(?:amd64|arm64)\/[a-z0-9._-]+\/latest\/src\/contrib\/Archive\/([^/]+)\/([^/]+\.tar\.gz)$/;
const MACOS_BIN_REGEX = const MACOS_BIN_REGEX =
/^\/bin\/macosx\/(big-sur-arm64|big-sur-x86_64|monterey-arm64|monterey-x86_64)\/contrib\/([0-9.]+)\/(.+)$/; /^\/bin\/macosx\/(big-sur-arm64|big-sur-x86_64|monterey-arm64|monterey-x86_64)\/contrib\/([0-9.]+)\/(.+)$/;
const RHEL_REGEX = /(almalinux|rocky)[^\d]*(\d+)/i; const RHEL_REGEX = /(almalinux|rocky)[^\d]*(\d+)/i;
const UBUNTU_REGEX = /Ubuntu ([\d.]+)/i; const UBUNTU_REGEX = /Ubuntu ([\d.]+)/i;
const UBUNTU_CODENAME_REGEX = /Ubuntu [\d.]+;\s*codename=([a-z][a-z0-9-]*)/i;
const UBUNTU_CODENAMES: Record<string, string> = { const UBUNTU_CODENAMES: Record<string, string> = {
'26.04': 'resolute',
'24.04': 'noble', '24.04': 'noble',
'22.04': 'jammy', '22.04': 'jammy',
}; };
@ -85,6 +117,22 @@ function redirectTo(location: string, status = 302): Response {
}); });
} }
function publicCdnOrigin(url: URL): string {
return PUBLIC_CDN_HOSTS.has(url.hostname) ? url.origin : PUBLIC_CDN_ORIGIN;
}
/**
* True when the client reports an R minor that we deliberately do not serve.
*
* A client that reports no minor at all is not "unsupported": non-R fetchers
* (mirror scripts, image builds) must keep getting the flat slot. Only a
* known-and-excluded minor falls through to CRAN.
*/
function isExcludedMinor(userAgent: string): boolean {
const rMinor = extractRMinor(userAgent);
return rMinor !== null && !KNOWN_MINORS.has(rMinor);
}
function extractRMinor(userAgent: string): string | null { function extractRMinor(userAgent: string): string | null {
for (const regex of R_MINOR_REGEXES) { for (const regex of R_MINOR_REGEXES) {
const match = userAgent.match(regex); const match = userAgent.match(regex);
@ -127,6 +175,11 @@ function parseSlot(userAgent: string): string | null {
const ubuntu = userAgent.match(UBUNTU_REGEX); const ubuntu = userAgent.match(UBUNTU_REGEX);
if (ubuntu) { if (ubuntu) {
const codenameMatch = userAgent.match(UBUNTU_CODENAME_REGEX);
if (codenameMatch) {
return `${arch}/${codenameMatch[1].toLowerCase()}`;
}
const codename = UBUNTU_CODENAMES[ubuntu[1]]; const codename = UBUNTU_CODENAMES[ubuntu[1]];
if (codename) { if (codename) {
return `${arch}/${codename}`; return `${arch}/${codename}`;
@ -161,8 +214,8 @@ function parseMacUserAgent(userAgent: string): { os: string; arch: string; rver:
* The contrib path a request should be served from, relative to the slot. * The contrib path a request should be served from, relative to the slot.
* *
* Returns the per-minor path for an index file when the slot is known to carry * Returns the per-minor path for an index file when the slot is known to carry
* a union index and the client's R minor is known; otherwise the flat path, * a union index and the client's R minor is one we publish; otherwise the flat
* which is what every client sees today. * path, which is what every client sees today.
*/ */
function contribPath(slot: string, rest: string, userAgent: string): string { function contribPath(slot: string, rest: string, userAgent: string): string {
const flat = rest ? `/${slot}/latest/src/contrib/${rest}` : `/${slot}/latest/src/contrib`; const flat = rest ? `/${slot}/latest/src/contrib/${rest}` : `/${slot}/latest/src/contrib`;
@ -172,7 +225,7 @@ function contribPath(slot: string, rest: string, userAgent: string): string {
} }
const rMinor = extractRMinor(userAgent); const rMinor = extractRMinor(userAgent);
return rMinor ? `/${slot}/latest/src/contrib/${rMinor}/${rest}` : flat; return rMinor && KNOWN_MINORS.has(rMinor) ? `/${slot}/latest/src/contrib/${rMinor}/${rest}` : flat;
} }
BunnySDK.net.http BunnySDK.net.http
@ -181,15 +234,14 @@ BunnySDK.net.http
const url = new URL(ctx.request.url); const url = new URL(ctx.request.url);
const path = normalizePathname(url.pathname); const path = normalizePathname(url.pathname);
const userAgent = ctx.request.headers.get('User-Agent') || ''; const userAgent = ctx.request.headers.get('User-Agent') || '';
const publicOrigin = publicCdnOrigin(url);
// macOS clients are served from CRAN's own binary tree. // macOS clients are served from CRAN's own binary tree.
const srcContrib = path.match(SRC_CONTRIB_REGEX); const srcContrib = path.match(SRC_CONTRIB_REGEX);
if (srcContrib && /darwin/.test(userAgent)) { if (srcContrib && /darwin/.test(userAgent)) {
const mac = parseMacUserAgent(userAgent); const mac = parseMacUserAgent(userAgent);
if (mac) { if (mac) {
return Promise.resolve( return Promise.resolve(redirectTo(`${publicOrigin}/bin/macosx/${mac.os}/contrib/${mac.rver}/${srcContrib[1]}`));
redirectTo(`${PUBLIC_CDN_ORIGIN}/bin/macosx/${mac.os}/contrib/${mac.rver}/${srcContrib[1]}`),
);
} }
} }
@ -209,11 +261,21 @@ BunnySDK.net.http
return Promise.resolve(ctx.request); return Promise.resolve(ctx.request);
} }
// An R minor outside the supported window has no binaries we can safely
// serve, so the whole interaction goes to CRAN: the index and the
// tarballs R will resolve against it. Serving the index from CRAN but
// tarballs from here would hand R a binary where it expects a source
// tarball, which fails in a far more confusing way than not being
// served at all.
if (UNION_SLOTS.has(slot) && isExcludedMinor(userAgent)) {
return Promise.resolve(redirectTo(`${CRAN_ORIGIN}/src/contrib/${rest}`));
}
const target = contribPath(slot, rest, userAgent); const target = contribPath(slot, rest, userAgent);
if (target === path) { if (target === path) {
return Promise.resolve(ctx.request); return Promise.resolve(ctx.request);
} }
return Promise.resolve(redirectTo(`${PUBLIC_CDN_ORIGIN}${target}`)); return Promise.resolve(redirectTo(`${publicOrigin}${target}`));
} }
// The bare `https://cran.rpkgs.com` form, resolved from the User-Agent. // The bare `https://cran.rpkgs.com` form, resolved from the User-Agent.
@ -223,13 +285,32 @@ BunnySDK.net.http
return Promise.resolve(redirectTo(`${CRAN_ORIGIN}${path}`)); return Promise.resolve(redirectTo(`${CRAN_ORIGIN}${path}`));
} }
if (UNION_SLOTS.has(slot) && isExcludedMinor(userAgent)) {
return Promise.resolve(redirectTo(`${CRAN_ORIGIN}${path}`));
}
const rest = srcContrib ? srcContrib[1] : ''; const rest = srcContrib ? srcContrib[1] : '';
return Promise.resolve(redirectTo(`${PUBLIC_CDN_ORIGIN}${contribPath(slot, rest, userAgent)}`)); return Promise.resolve(redirectTo(`${publicOrigin}${contribPath(slot, rest, userAgent)}`));
} }
return Promise.resolve(ctx.request); return Promise.resolve(ctx.request);
}) })
.onOriginResponse((ctx) => { .onOriginResponse(async (ctx) => {
const path = normalizePathname(new URL(ctx.request.url).pathname);
const archive = path.match(ARCHIVE_TARBALL_REGEX);
// Binary archives can be incomplete when an older build never succeeded.
// Preserve renv/remotes version restores by falling back to CRAN's source
// package only for an absent archived tarball. A requested version can be
// either archived upstream or still current, so probe the archive first.
// Other 404s remain visible.
if (ctx.response.status === 404 && archive) {
const archiveUrl = `${CRAN_ORIGIN}/src/contrib/Archive/${archive[1]}/${archive[2]}`;
const archiveResponse = await fetch(archiveUrl, { method: 'HEAD' });
const sourceUrl = archiveResponse.ok ? archiveUrl : `${CRAN_ORIGIN}/src/contrib/${archive[2]}`;
return redirectTo(sourceUrl);
}
ctx.response.headers.append('X-Via', 'MyMiddleware'); ctx.response.headers.append('X-Via', 'MyMiddleware');
return Promise.resolve(ctx.response); return Promise.resolve(ctx.response);
}); });

View file

@ -40,15 +40,11 @@ index 6f6a745..e407986 100644
if (is.null(name)) if (is.null(name))
return(tbbRoot) return(tbbRoot)
@@ -58,7 +58,7 @@ tbbCxxFlags <- function() { @@ -58,3 +58,3 @@ tbbCxxFlags <- function() {
flags <- c("-DRCPP_PARALLEL_USE_TBB=1")
# if TBB_INC is set, apply those library paths # if TBB_INC is set, apply those library paths
- tbbInc <- Sys.getenv("TBB_INC", unset = TBB_INC) - tbbInc <- Sys.getenv("TBB_INC", unset = TBB_INC)
+ tbbInc <- bincraftGetenv("TBB_INC", unset = TBB_INC) + tbbInc <- bincraftGetenv("TBB_INC", unset = TBB_INC)
if (!file.exists(tbbInc)) { if (!file.exists(tbbInc)) {
tbbInc <- system.file("include", package = "RcppParallel")
}
@@ -117,7 +117,7 @@ tbbLdFlags <- function() { @@ -117,7 +117,7 @@ tbbLdFlags <- function() {
} }

View file

@ -1,7 +1,7 @@
[ [
{ {
"package": "RcppParallel", "package": "RcppParallel",
"versions": ">=6.0.0", "versions": "6.2.1",
"platforms": ["*"], "platforms": ["*"],
"env": {}, "env": {},
"configure_args": [], "configure_args": [],

View file

@ -84,9 +84,20 @@ trap 'rm -rf "$project_dir"' EXIT
cd "$project_dir" cd "$project_dir"
"$uvr_bin" init --here --r-version "$r_full" "$uvr_bin" init --here --r-version "$r_full"
# --no-install: resolve and lock only. The install happens in the sync below, # --no-install resolves and locks only; retry because concurrent shards can
# which is the only command that honours --library. # expose short-lived DNS or CRAN-index failures and uvr rolls the manifest back
"$uvr_bin" add --no-install "$@" # cleanly after an unsuccessful resolution.
add_attempt=1
while ! "$uvr_bin" add --no-install "$@"; do
if [ "$add_attempt" -ge 4 ]; then
echo "error: uvr add failed after ${add_attempt} attempts" >&2
exit 1
fi
add_delay=$((add_attempt * 10))
echo "warning: uvr add attempt ${add_attempt} failed; retrying in ${add_delay}s" >&2
sleep "$add_delay"
add_attempt=$((add_attempt + 1))
done
# TEMPORARY (drop once the images ship a uvr above v0.4.5): the sync below runs # TEMPORARY (drop once the images ship a uvr above v0.4.5): the sync below runs
# `apt-get install` for every resolved system dependency without refreshing the # `apt-get install` for every resolved system dependency without refreshing the

View file

@ -2,7 +2,7 @@ terraform {
required_providers { required_providers {
bunnynet = { bunnynet = {
source = "registry.terraform.io/BunnyWay/bunnynet" source = "registry.terraform.io/BunnyWay/bunnynet"
version = "~> 0.17" version = "~> 0.18"
} }
} }
} }

View file

@ -18,35 +18,70 @@
# objects were replaced. The cost is a cold cache for everything else, which is # objects were replaced. The cost is a cold cache for everything else, which is
# why this is not used by the daily update path. # why this is not used by the daily update path.
# #
# All hostnames on the zone (cran.devxy.io, cran.allianceswisspass.devxy.io, # The public hostnames currently use separate pull zones, so callers must pass
# cran.rpkgs.com) share pull zone 3857050, so one purge covers all of them. # every zone that serves the repository. A zone can be identified by its
# numeric ID or by one of its hostnames; hostname lookup avoids persisting IDs
# that change when a zone is recreated.
# #
# Usage: # Usage:
# purge_cdn_zone.sh <BUNNYNET_API_KEY> <pull_zone_id> # purge_cdn_zone.sh <BUNNYNET_API_KEY> <pull_zone> [<pull_zone>...]
# #
set -euo pipefail set -euo pipefail
if (($# < 2)); then if (($# < 2)); then
echo "usage: $0 <api_key> <pull_zone_id>" >&2 echo "usage: $0 <api_key> <pull_zone> [<pull_zone>...]" >&2
exit 2 exit 2
fi fi
api_key="$1" api_key="$1"
zone_id="$2" shift
echo "Purging BunnyCDN pull zone ${zone_id}" resolve_zone_id() {
local zone="$1"
local response_file
local zone_id
status=$( if [[ "${zone}" =~ ^[0-9]+$ ]]; then
curl -sS -o /tmp/purge_zone_response.txt -w '%{http_code}' -X POST \ echo "${zone}"
return
fi
response_file=$(mktemp)
curl -sS -o "${response_file}" \
-H "AccessKey: ${api_key}" \ -H "AccessKey: ${api_key}" \
-H "Content-Length: 0" \ "https://api.bunny.net/pullzone"
"https://api.bunny.net/pullzone/${zone_id}/purgeCache" zone_id=$(
) jq -r --arg hostname "${zone}" \
'(.Items // .)[] | select(any(.Hostnames[]?; .Value == $hostname)) | .Id' \
"${response_file}"
)
rm -f "${response_file}"
if [[ "${status}" != "200" && "${status}" != "204" ]]; then if [[ -z "${zone_id}" ]]; then
echo "Purge of pull zone ${zone_id} failed with HTTP ${status}:" >&2 echo "Could not find BunnyCDN pull zone for hostname ${zone}" >&2
cat /tmp/purge_zone_response.txt >&2 exit 1
exit 1 fi
fi
echo "Purged pull zone ${zone_id} (HTTP ${status})" echo "${zone_id}"
}
for zone in "$@"; do
zone_id=$(resolve_zone_id "${zone}")
echo "Purging BunnyCDN pull zone ${zone_id}"
response_file="/tmp/purge_zone_response_${zone_id}.txt"
status=$(
curl -sS -o "${response_file}" -w '%{http_code}' -X POST \
-H "AccessKey: ${api_key}" \
-H "Content-Length: 0" \
"https://api.bunny.net/pullzone/${zone_id}/purgeCache"
)
if [[ "${status}" != "200" && "${status}" != "204" ]]; then
echo "Purge of pull zone ${zone_id} failed with HTTP ${status}:" >&2
cat "${response_file}" >&2
exit 1
fi
echo "Purged pull zone ${zone_id} (HTTP ${status})"
done

309
scripts/verify-r-minor-routing.sh Executable file
View file

@ -0,0 +1,309 @@
#!/usr/bin/env bash
#
# Verify per-R-minor index routing for cran.rpkgs.com across every published
# <arch>/<os> slot.
#
# The edge router (edge/rpkgs-router.ts) rewrites PACKAGES* requests to
# `contrib/<x.y>/` when the slot is listed in UNION_SLOTS and the client's
# User-Agent carries an R minor. Two properties have to hold before a slot may
# be added to UNION_SLOTS:
#
# 1. the per-minor index is a UNION of the per-minor and flat slots, so
# routing to it hides nothing the flat index carries; and
# 2. every R minor a client might report resolves to an index that exists,
# because contribPath() does not check existence and has no fallback.
#
# Modes:
# (default) Resolve routing decisions without depending on UNION_SLOTS being
# set. Safe to run before enabling: it reads the per-minor indexes
# directly and reproduces the router's target path.
# --live Additionally drive the real CDN with R User-Agents and assert the
# bytes served match the expected index. Only meaningful once the
# slot is in UNION_SLOTS.
#
# Usage:
# scripts/verify-r-minor-routing.sh
# scripts/verify-r-minor-routing.sh --live
# MINORS="4.4 4.5" SAMPLE=10 scripts/verify-r-minor-routing.sh
#
# Exits non-zero if any check fails.
set -uo pipefail
BASE=${BASE:-https://cran.rpkgs.com}
ARCHES=${ARCHES:-"amd64 arm64"}
DISTROS=${DISTROS:-"resolute noble jammy rhel8 rhel9 rhel10 alpine323 alpine324"}
# The supported window: the current R minor plus the two previous, matching
# build-env-images' R_VERSION_LATEST/PREV1/PREV2 and cdn.tf's
# local.rpkgs_supported_minors. Each of these must have a published index.
MINORS=${MINORS:-"4.4 4.5 4.6"}
# Minors we deliberately do not serve. These must have NO published index and,
# once routing is live, must be sent to CRAN for sources rather than 404ing or
# being handed binaries built under another minor.
EXCLUDED_MINORS=${EXCLUDED_MINORS:-"4.3"}
# How many Path: targets to HEAD-check per slot/minor. 0 disables.
SAMPLE=${SAMPLE:-5}
# Largest package-count shortfall a non-primary minor may have against the best
# minor on the same slot before coverage counts as uneven. A slot built under
# one R minor carries fewer per-minor binaries for the others; until that gap
# closes, "full coverage for ABI-sensitive packages" is not a claim we can make.
PARITY_TOLERANCE=${PARITY_TOLERANCE:-25}
LIVE=0
for arg in "$@"; do
case "$arg" in
--live) LIVE=1 ;;
-h | --help)
sed -n '2,32p' "$0"
exit 0
;;
*)
echo "unknown argument: $arg" >&2
exit 2
;;
esac
done
WORK=$(mktemp -d)
trap 'rm -rf "$WORK"' EXIT
PASS=0
FAIL=0
FAILURES=""
ok() {
PASS=$((PASS + 1))
printf ' ok %s\n' "$1"
}
bad() {
FAIL=$((FAIL + 1))
FAILURES="${FAILURES}\n - $1"
printf ' FAIL %s\n' "$1"
}
# Fetch a URL into a file, echoing the HTTP status. Cached per URL.
fetch() {
local url=$1 dest=$2 ua=${3:-}
if [ -s "$dest" ]; then
cat "$dest.status"
return 0
fi
local status
if [ -n "$ua" ]; then
status=$(curl -sS -A "$ua" -o "$dest" -w '%{http_code}' --max-time 120 "$url" 2>/dev/null)
else
status=$(curl -sS -o "$dest" -w '%{http_code}' --max-time 120 "$url" 2>/dev/null)
fi
echo "$status" > "$dest.status"
echo "$status"
}
head_status() {
curl -sS -o /dev/null -w '%{http_code}' -I --max-time 60 "$1" 2>/dev/null
}
# Package names from a gzipped PACKAGES index, sorted.
pkg_names() {
gunzip -c "$1" 2>/dev/null | awk '/^Package:/ {print $2}' | sort -u
}
# "<Package> <Path>" pairs for entries that carry a Path: field.
path_entries() {
gunzip -c "$1" 2>/dev/null | awk '
/^Package:/ { pkg = $2; ver = ""; path = "" }
/^Version:/ { ver = $2 }
/^Path:/ { path = $2 }
/^$/ { if (pkg != "" && path != "") print pkg, ver, path; pkg = "" }
END { if (pkg != "" && path != "") print pkg, ver, path }
'
}
# An R User-Agent of the shape R actually sends.
r_user_agent() {
printf 'R/%s.0 (Ubuntu 24.04; codename=noble) (x86_64-pc-linux-gnu x86_64 linux-gnu)' "$1"
}
echo "verify-r-minor-routing: $BASE"
echo " slots: $(echo "$ARCHES" | wc -w) arch x $(echo "$DISTROS" | wc -w) os"
echo " minors: $MINORS (excluded: $EXCLUDED_MINORS)"
echo " live: $LIVE"
echo
for arch in $ARCHES; do
for distro in $DISTROS; do
slot="$arch/$distro"
echo "$slot"
flat_url="$BASE/$slot/latest/src/contrib/PACKAGES.gz"
flat_file="$WORK/${arch}-${distro}-flat.gz"
flat_status=$(fetch "$flat_url" "$flat_file")
if [ "$flat_status" != "200" ]; then
bad "$slot flat index unreachable (HTTP $flat_status)"
continue
fi
pkg_names "$flat_file" > "$flat_file.names"
flat_count=$(wc -l < "$flat_file.names")
if [ "$flat_count" -lt 1000 ]; then
bad "$slot flat index has only $flat_count packages"
continue
fi
ok "$slot flat index: $flat_count packages"
for minor in $MINORS; do
minor_url="$BASE/$slot/latest/src/contrib/$minor/PACKAGES.gz"
minor_file="$WORK/${arch}-${distro}-${minor}.gz"
minor_status=$(fetch "$minor_url" "$minor_file")
# A minor the router would route to must exist, or clients on that R
# version get a 404 and see no packages at all.
if [ "$minor_status" != "200" ]; then
bad "$slot R $minor index missing (HTTP $minor_status) - routing would 404 for R $minor clients"
continue
fi
pkg_names "$minor_file" > "$minor_file.names"
minor_count=$(wc -l < "$minor_file.names")
# Union property: nothing the flat index carries may be missing here.
missing=$(comm -23 "$flat_file.names" "$minor_file.names" | head -5)
missing_count=$(comm -23 "$flat_file.names" "$minor_file.names" | wc -l)
if [ "$missing_count" -ne 0 ]; then
bad "$slot R $minor index is not a union: $missing_count flat packages absent (e.g. $(echo "$missing" | tr '\n' ' '))"
else
ok "$slot R $minor index: $minor_count packages, union holds"
fi
# Path: entries steer to per-minor binaries; they must resolve.
if [ "$SAMPLE" -gt 0 ]; then
path_entries "$minor_file" > "$minor_file.paths"
total_paths=$(wc -l < "$minor_file.paths")
broken=0
checked=0
while read -r pkg ver path; do
[ -z "${pkg:-}" ] && continue
tarball="$BASE/$slot/latest/src/contrib/$path/${pkg}_${ver}.tar.gz"
status=$(head_status "$tarball")
checked=$((checked + 1))
if [ "$status" != "200" ]; then
broken=$((broken + 1))
[ "$broken" -le 2 ] && printf ' broken target: %s (HTTP %s)\n' "$tarball" "$status"
fi
done < <(shuf -n "$SAMPLE" "$minor_file.paths" 2>/dev/null || head -n "$SAMPLE" "$minor_file.paths")
if [ "$broken" -ne 0 ]; then
bad "$slot R $minor: $broken/$checked sampled Path: targets do not resolve (of $total_paths total)"
elif [ "$checked" -gt 0 ]; then
ok "$slot R $minor: $checked/$checked sampled Path: targets resolve (of $total_paths total)"
fi
fi
# Live routing: what a real R client on this minor actually receives.
if [ "$LIVE" -eq 1 ]; then
ua=$(r_user_agent "$minor")
live_file="$WORK/${arch}-${distro}-${minor}-live.gz"
live_status=$(fetch "$flat_url" "$live_file" "$ua")
if [ "$live_status" != "200" ]; then
bad "$slot R $minor live request failed (HTTP $live_status)"
elif cmp -s "$live_file" "$minor_file"; then
ok "$slot R $minor live request served the per-minor index"
elif cmp -s "$live_file" "$flat_file"; then
bad "$slot R $minor live request served the FLAT index - slot not in UNION_SLOTS?"
else
bad "$slot R $minor live request served neither the per-minor nor the flat index"
fi
fi
done
# Coverage parity across minors. The union property only guarantees no
# client loses packages relative to the flat index; it says nothing about a
# 4.4 client seeing fewer packages than a 4.5 client on the same slot.
best=0
for minor in $MINORS; do
f="$WORK/${arch}-${distro}-${minor}.gz.names"
[ -s "$f" ] || continue
c=$(wc -l < "$f")
[ "$c" -gt "$best" ] && best=$c
done
if [ "$best" -gt 0 ]; then
uneven=""
for minor in $MINORS; do
f="$WORK/${arch}-${distro}-${minor}.gz.names"
[ -s "$f" ] || continue
c=$(wc -l < "$f")
gap=$((best - c))
[ "$gap" -gt "$PARITY_TOLERANCE" ] && uneven="$uneven R$minor:-$gap"
done
if [ -n "$uneven" ]; then
bad "$slot coverage uneven across minors (vs best $best):$uneven"
else
ok "$slot coverage parity across minors (best $best, all within $PARITY_TOLERANCE)"
fi
fi
# Excluded minors: no published index, and under --live a redirect to CRAN.
for minor in $EXCLUDED_MINORS; do
ex_url="$BASE/$slot/latest/src/contrib/$minor/PACKAGES.gz"
ex_status=$(curl -sS -o /dev/null -w '%{http_code}' --max-time 60 "$ex_url" 2>/dev/null)
if [ "$ex_status" = "200" ]; then
bad "$slot R $minor is excluded but an index is published - the two lists disagree"
else
ok "$slot R $minor correctly has no published index"
fi
if [ "$LIVE" -eq 1 ]; then
loc=$(curl -sS -o /dev/null -w '%{redirect_url}' -A "$(r_user_agent "$minor")" \
--max-time 60 "$flat_url" 2>/dev/null)
case "$loc" in
https://cran.r-project.org/*)
ok "$slot R $minor is sent to CRAN ($loc)"
;;
"")
bad "$slot R $minor was served directly instead of being sent to CRAN"
;;
*)
bad "$slot R $minor redirected somewhere unexpected: $loc"
;;
esac
fi
done
# A client whose User-Agent carries no R version must keep getting the flat
# index, never a per-minor one.
if [ "$LIVE" -eq 1 ]; then
plain_file="$WORK/${arch}-${distro}-plain.gz"
plain_status=$(fetch "$flat_url" "$plain_file" "curl/8.0.0")
if [ "$plain_status" != "200" ]; then
bad "$slot non-R User-Agent request failed (HTTP $plain_status)"
elif cmp -s "$plain_file" "$flat_file"; then
ok "$slot non-R User-Agent still served the flat index"
else
bad "$slot non-R User-Agent was routed away from the flat index"
fi
# Tarball requests must never be rewritten into a per-minor directory:
# flat-slot packages do not live there.
sample_pkg=$(gunzip -c "$flat_file" | awk '/^Package:/ {p=$2} /^Version:/ {print p, $2; exit}')
if [ -n "$sample_pkg" ]; then
# shellcheck disable=SC2086 # deliberate split into $1 (package) and $2 (version)
set -- $sample_pkg
tb="$BASE/$slot/latest/src/contrib/${1}_${2}.tar.gz"
tb_status=$(curl -sS -o /dev/null -w '%{http_code}' -A "$(r_user_agent 4.5)" --max-time 60 "$tb" 2>/dev/null)
if [ "$tb_status" = "200" ]; then
ok "$slot tarball request under an R User-Agent still resolves"
else
bad "$slot tarball ${1}_${2}.tar.gz broke under an R User-Agent (HTTP $tb_status)"
fi
fi
fi
done
done
echo
echo "passed: $PASS failed: $FAIL"
if [ "$FAIL" -ne 0 ]; then
printf 'failures:%b\n' "$FAILURES"
exit 1
fi