Author	SHA1	Message	Date
pat-s	a3004695a7	test(verify): gate on regressions against the generic slot, not fallback rate (#179) ... ## Motivation The readiness check added in #175 failed a slot when more than 10% of its per-minor entries were source fallbacks. That stopped being a meaningful signal once rpkgs/bincraft#113 and #114 landed. Since bincraft keeps a matching-minor generic binary out of a fallback's shadow, a surviving fallback means the generic slot's binary was built under a **different** minor — unsafe for that client anyway. Serving source there is correct, just slow. Failing on that share blocks slots that are genuinely ready: `amd64/noble` sits at 53% for 4.5 and 4.6 while regressing nobody. ## Change Gate on the thing that actually decides enablement: packages a client of minor M would receive as **source** through per-minor routing while the generic slot holds a binary built under **M itself**. That is strictly worse than not routing at all, and must be zero. Fallback share is still printed, as context rather than a verdict. ## Verification Measured across every reindexed slot and minor after the `reindex=all` run: zero regressions everywhere. | slot | 4.4 | 4.5 | 4.6 | |---|---|---|---| | amd64/noble | 0 | 0 | 0 | | amd64/jammy | 0 | 0 | 0 | | amd64/rhel9 | 0 | 0 | 0 | | amd64/rhel10 | 0 | 0 | 0 | | amd64/resolute | 0 | 0 | 0 | | arm64/noble | 0 | 0 | 0 | `shellcheck` clean; script exercised against the live indexes. Reviewed-on: #179	2026-08-31 09:55:44 +00:00
pat-s	85295a9495	fix(cdn): resolve a pull zone when the API answers with a bare array (#178) ... ## Motivation Every reindex reports `failure` at the purge step: ``` Purging BunnyCDN pull zone 3857050 Purged pull zone 3857050 (HTTP 204) jq: error (at /tmp/tmp.eFPFmO:0): Cannot index array with string "Items" Could not find BunnyCDN pull zone for hostname cran.allianceswisspass.devxy.io ``` `cran.rpkgs.com` purges fine. The Alliance zone never has, so it is still serving objects that rebuilds replaced, behind a ~370-day `cache_expiration_time`. ## The defect ```sh jq -r '(.Items // .)[] | ...' ``` This was meant to accept both response shapes. It accepts neither: indexing an array with a string is an **error** in jq, not a null, so `//` never gets the chance to substitute and the whole expression aborts. The listing endpoint answers with a bare array for this account, so the lookup has always failed. ## Change - Select the array explicitly by type instead of relying on `//` to absorb an error. - Check the HTTP status of the listing call. It was previously used unconditionally, so an auth or rate-limit failure surfaced as "could not find hostname" — pointing at the wrong thing entirely. - Fail when a hostname matches multiple zones rather than silently purging whichever jq emitted first. - Request `perPage=1000`, so a paginated response cannot silently truncate the zone list. ## Verification Ran the current `main` script and the fixed one against a stubbed `curl` returning an array-shaped listing: ``` === BEFORE (main) === Purged pull zone 3857050 (HTTP 204) jq: error (at ...): Cannot index array with string ("Items") Could not find BunnyCDN pull zone for hostname cran.allianceswisspass.devxy.io === AFTER === Purged pull zone 3857050 (HTTP 204) Purging BunnyCDN pull zone 222 Purged pull zone 222 (HTTP 204) ``` The jq expression was also checked against both an array-shaped and an object-shaped (`.Items`) response; the old one fails the array case, the new one handles both. `shellcheck` clean. Reviewed-on: #178	2026-08-31 09:55:37 +00:00
pat-s	aba2063ea0	feat(edge): gate per-minor routing on published minors and add a staging zone (#175) ... ## Motivation `UNION_SLOTS` is empty, so per-minor routing has never been exercised end to end. Before it can be enabled and advertised, two things were missing: a way to test it without pointing production at it, and evidence that the published indexes actually support it. Verifying the data first turned up a defect that would have broken users the moment the flag was flipped. ## The defect `contribPath()` redirects to `contrib/<minor>/` whenever the User-Agent carries any R minor, with no existence check and no fallback: ```ts const rMinor = extractRMinor(userAgent); return rMinor ? `/${slot}/latest/src/contrib/${rMinor}/${rest}` : flat; ``` Only `4.4`, `4.5` and `4.6` are published. `4.3` and `4.2` return 404 on all 16 slots. With `UNION_SLOTS` set, an R 4.3 client would be redirected to a non-existent index and see **zero** packages: a silent, total failure rather than a degraded one. R 4.3 is still advertised as supported on the website and in `docs/configuration.mdoc`, though `build-env-images` now pins only 4.6.0/4.5.3/4.4.3. ## Changes - **Gate routing on `KNOWN_MINORS`** (default `4.4,4.5,4.6`), falling back to the flat index for anything else. Unknown minor now behaves exactly as today. - **Honour `EXTRA_PUBLIC_HOSTS`.** `publicCdnOrigin()` falls back to the hardcoded `PUBLIC_CDN_ORIGIN` for any hostname not in `PUBLIC_CDN_HOSTS`, so a staging zone on a `b-cdn.net` hostname would redirect into *production* and silently measure the wrong system. This lets the identical script run on staging and redirect within itself. - **Add the `cran-rpkgs-test` pull zone** with `UNION_SLOTS` pre-enabled for all 16 slots, same B2 origin, served on the bunny default hostname so it needs no DNS record and is never advertised. - **Add `scripts/verify-r-minor-routing.sh`**, covering every `<arch>/<os>` slot: index reachability per minor, the union property against flat, `Path:` target resolution, coverage parity across minors, and with `--live` the real User-Agent routing, the non-R User-Agent case, and that tarballs are never rewritten. - **Cover the fallback in the edge test suite** for both an unpublished minor (4.3) and a future one (4.7). ## Findings from the full run 112 passed, 16 failed across the 16 slots. Every failure is the same: no R 4.3 index. All 16 slots carry union indexes that are supersets of flat, every sampled `Path:` target resolves, and all indexes were republished within minutes of each other, so the build side is healthy. Coverage is **not** yet even, which is why "full coverage for ABI-sensitive packages" is not a claim to make yet: | slot | flat | 4.4 | 4.5 | 4.6 | |---|---|---|---|---| | amd64/resolute | 24305 | 24402 | 24748 | 24395 | | amd64/alpine324 | 24397 | 24457 | 24744 | 24448 | | amd64/noble | 24780 | 24805 | 24805 | 24805 | On the R 4.5-built distros (`resolute`, `alpine324`, and their arm64 twins) a 4.4 or 4.6 client sees ~300 fewer packages than a 4.5 client. On `noble`/`jammy`/`rhel9`/`alpine323` the spread is under 5. The new parity check encodes this with a configurable `PARITY_TOLERANCE`. ## Verification - `just edge-test`: 18 steps pass. The two new steps were confirmed to fail with the `KNOWN_MINORS` gate removed and pass with it. - `tofu validate`: passes. **Not applied** - no bunny.net or state credentials were available, so the staging zone still needs a `tofu apply`. - `scripts/verify-r-minor-routing.sh`: full 16-slot run, results above. - `shellcheck`: clean. ## Not done here Applying the staging zone, then running `BASE=https://cran-rpkgs-test.b-cdn.net scripts/verify-r-minor-routing.sh --live` against it. Production `UNION_SLOTS` is deliberately left empty. Reviewed-on: #175	2026-08-30 21:20:16 +00:00
pat-s	9bded261ee	fix(cdn): restore Alliance pull-zone hostname (#166) ... ## Motivation Applying #165 recreated the Alliance SwissPass pull zone without its custom hostname because the hostname association was not represented in OpenTofu. The recreated zone also received a new numeric ID, making the weekly purge configuration stale. ## Changes - Manage `cran.allianceswisspass.devxy.io` as a pull-zone hostname with TLS and forced HTTPS. - Resolve the Alliance pull-zone ID from its hostname before purging instead of persisting a replaceable numeric ID. - Install `jq` in the purge step for the Bunny API lookup. ## Verification - Targeted `prek` hooks pass. - `tofu validate` passes. - `crow lint .crow/` passes. - `just edge-test` passes all 14 routing steps. - `bash -n scripts/purge_cdn_zone.sh` passes. ## Deployment Run `tofu apply` to restore the Alliance hostname on the recreated pull zone. Reviewed-on: #166	2026-08-13 14:13:04 +00:00
pat-s	a1c1f5e78f	fix(cdn): align repository routing across pull zones (#165) ... ## Motivation `cran.rpkgs.com` and `cran.allianceswisspass.devxy.io` serve the same B2 repository through separate Bunny pull zones, but only the first zone was managed and purged after weekly reindexing. This allowed the Alliance endpoint to retain stale repository metadata and left locked `renv` restores unable to retrieve versions whose binary archive object was absent. ## Changes - Adopt the Alliance SwissPass pull zone `3265648` into OpenTofu and configure it with the shared B2 origin and middleware script. - Purge both Bunny pull zones after the weekly rebuild reindex. - Preserve the requested public hostname in middleware redirects. - Redirect missing archived binaries to the corresponding CRAN source package, checking whether the version is archived or still current. - Cover the existing archived-binary passthrough behavior in the edge routing matrix. ## Verification - `prek run -a` - `just edge-test` - `crow lint .crow/` - `tofu validate` - `bash -n scripts/purge_cdn_zone.sh` ## Deployment Run `tofu apply` to adopt pull zone `3265648`, publish the middleware release, and align both pull zones. After the apply, rerun the Alliance SwissPass CI restore that requested `cli 3.6.5` and `AzureStor 3.7.1`. Reviewed-on: #165	2026-08-13 14:08:10 +00:00
pat-s	01b8ab43df	fix(rebuild): re-index and purge the CDN after a rebuild (#160) ... ## Problem The rebuild now works — `AATtools 0.0.3` was detected as a source fallback, built, and published: ``` ℹ `upload_single_binary()`: Replacing the CRAN source published for AATtools 0.0.3 … with the binary. ✔ Successfully uploaded package AATtools with tag 0.0.3. ``` But clients still get the source, and will for about a year: ``` $ curl -sI .../src/contrib/AATtools_0.0.3.tar.gz etag: "ea8127d953ca6a2f118ea49441772af6" # CRAN's source MD5 cdn-cache: HIT cdn-cachedat: 08/09/2026 16:43:32 # predates the 18:12 upload ``` Two causes, both specific to a rebuild: 1. **The slot is never re-indexed.** `weekly-rebuild-missing` has no `upload_package_index` step, so the index keeps the old MD5 and — for anything that had been served from source — no `Built` stamp. This one self-heals at the next `process-updates` run. 2. **The tarball URL is never purged.** A normal update publishes new packages at *new* URLs, so `purge_cdn_cache.sh` only needs the five index files. A rebuild replaces an object *in place*, and the zone caches tarballs for `cache_expiration_time = 31919000` (~370 days). This does not self-heal. Nothing about a stale package looks wrong from the outside, which is what makes it worth fixing rather than documenting. ## What this changes **Re-index at the end of a rebuild**, flat and per-minor, mirroring the tail of `process-updates`. The codename is detected from the image's `/etc/os-release` (as `local/packages-to-build.R` already does) rather than adding `OS_ID` to all 18 matrix rows. **Purge the zone afterwards**, via a new `scripts/purge_cdn_zone.sh`. One call to `POST /pullzone/{id}/purgeCache` covers every replaced object, and all three hostnames — `cran.devxy.io`, `cran.allianceswisspass.devxy.io`, `cran.rpkgs.com` — share pull zone `3857050`, confirmed from the `cdn-pullzone` response header. Purging per URL was the alternative and is worse here: ~13.5k rate-limited calls per arch, where a single missed call leaves a package silently stale. The cost of the zone purge is a cold cache for everything else, which is why it stays out of the daily update path — `purge_cdn_cache.sh` is untouched. The purge runs on failure too (`when: status: [success, failure]`): a rebuild that died part-way still replaced objects, and those are exactly the ones a stale edge keeps hiding. ## Verification `crow lint .crow/` reports all ten configs valid; `bash -n` on the new script passes; prek hooks pass. Not yet exercised against Bunny — it needs `BUNNYNET_API_KEY`, which is a CI secret. The failure mode is explicit rather than silent: any status other than 200/204 prints the response body and exits non-zero. Reviewed-on: #160	2026-08-10 06:30:35 +00:00
pat-s	228459c0ba	fix(ci): actually purge BunnyCDN cache in purge_cdn_cache.sh (#78) ... ## Summary While cleaning this up I noticed the script **defined** `purge_cdn_cache()` but never **called** it. Every CI run only declared the helper and exited cleanly without issuing a single curl. The PACKAGES freshness on cran.devxy.io / cran.rpkgs.com has been carried entirely by R's `Cache-Control: no-cache` header on the index files. Fixes in one shot: 1. **Actually run the purge.** The shifted args (`api_key`, `arch`, `os_id`, `domain…`) are now consumed inline and a curl POST is issued per (domain × resource). 2. **Drop `set -x`** — leaked the `AccessKey:` header into job logs. 3. **Drop the Python URL-encoder.** `curl -G --data-urlencode "url=…" --data "async=false" https://api.bunny.net/purge` does the same with no Python. The workflow purge steps can later switch from `alpine:3.23 + apk add bash curl` to a slimmer curl-only image. 4. **Add `src/contrib/Meta/archive.rds`** to the purged resource list. It's rewritten on every `process_cran_updates` run (see README) and was being served stale. ## Risks - This is the **first time** the script actually purges anything. If anything else (e.g. a downstream service) relied on the no-op behavior, this PR is the moment it stops being silent. I don't see any such caller. - Edge cache miss right after a purge means an origin S3 fetch — minor latency uptick on the first request per region per resource. Reviewed-on: #78	2026-06-08 08:31:03 +00:00
pat-s	b769276679	fix: bincraftr url for install	2025-05-29 17:15:05 +02:00
pat-s	4dbe4daa4b	transparent purging	2025-03-26 15:57:42 +01:00
pat-s	3e55e81a0e	feat: build-single for alpine321	2025-03-26 15:25:53 +01:00