feat(edge): route PACKAGES requests to the per-R-minor slot #152

Merged

pat-s merged 4 commits from feat/per-minor-edge-routing into main

2026-08-07 14:12:22 +00:00

pat-s commented

2026-08-07 14:10:15 +00:00

Owner

Problem

install.packages("curl") fails in reg.devxy.io/r/r-alpine:4.5-3.24 with "package 'curl' is not available for this version of R", on both arches.

curl is not missing from the repo: it is in …/latest/src/contrib/4.5/ and …/4.6/, the per-minor slots that base R cannot address. The image's repo URL resolves to …/latest/src/contrib, whose index does not list it. On amd64/alpine324 that is 2 886 packages invisible to install.packages() (23 on amd64/noble) — what issue #63 records as "missing binaries".

Two further findings while investigating:

The middleware only ever rewrote the bare cran.rpkgs.com/src/contrib/… form, and that form was broken for every Linux client on a stock R user agent: ALPINE_REGEX/UBUNTU_REGEX/RHEL_REGEX need a Posit-style UA that carries the distro, so stock R fell through to extractOs() and got redirected to /amd64/linux-musl/latest/…, a slot that does not exist.
PACKAGES* is served cdn-cache: BYPASS (bincraft uploads it no-store), so the middleware sees every index request and no purge is needed for routing changes to take effect.

What this changes

edge/rpkgs-router.ts — the middleware, now a reviewed file in this repo rather than dashboard state. It routes PACKAGES, PACKAGES.gz and PACKAGES.rds into …/src/contrib/<x.y>/ for slots listed in UNION_SLOTS, and nothing else.

Tarballs are deliberately left alone. R keeps the contriburl it asked for, not the one the redirect served it, so every tarball URL is resolved against the flat directory and the union index steers the per-minor ones with a Path: <x.y> field. Rewriting a tarball request here would send flat-slot packages into a directory that does not hold them.

Also in the script: the phantom linux-gnu/linux-musl fallback is gone (an unidentifiable distro goes to CRAN, as an unparseable UA already did), and every redirect carries Cache-Control: no-store since its target depends on the User-Agent. The macOS branches are unchanged.

cdn.tf — bunnynet_compute_script.rpkgs_router with content = file("edge/rpkgs-router.ts"), the UNION_SLOTS variable, and middleware_script pointing at the resource instead of the literal 29277.

UNION_SLOTS is empty, so merging and applying this changes no client's behaviour. A slot is added only once bincraft has republished its per-minor index as a union (rpkgs/bincraft#97); routing to a raw per-minor index would hide every package it does not carry. Rolling back is a variable edit, not a deploy.

specs/, plans/ — the design and the implementation plan, including the two approaches that were rejected (edge-side merge, moving the minor up the path) and why.

Verification

just edge-test runs 13 routing cases against the SDK's local server, so what is tested is the artifact that gets deployed; pass-through cases proxy to the real origin. All pass.

End to end, with the middleware in front of a locally built union index for amd64/alpine324 (31 507 records), inside the runtime image:

curl:     7.1.0  -> …/latest/src/contrib/4.5  -> curl_7.1.0.tar.gz      717 725 B
jsonlite: 2.0.0  -> …/latest/src/contrib      -> jsonlite_2.0.0.tar.gz  1 055 849 B

tofu validate passes. tofu plan has not been run: no BUNNYNET_API_KEY available in this environment.

Before applying

The script pre-dates this configuration, so it must be adopted, not created:

tofu import bunnynet_compute_script.rpkgs_router 29277
tofu plan

The plan should show an in-place content update and no replacement of the pull zone. Without the import, tofu creates a second script and repoints the zone at it. Note that name = "rpkgs-router" will rename the existing script on apply.

Not fixed here

install.packages("curl") on alpine324 will now resolve, then fail to build: that slot's tarballs are byte-identical CRAN source tarballs (no Meta/, no Built: in DESCRIPTION) which the index nevertheless stamps Built: R 4.5.3; …-linux-musl. Sampled: amd64/alpine324 3/12 binary, arm64/alpine324 13/30, amd64/noble 12/12, amd64/alpine323 17/20. That slot needs a rebuild, tracked separately.

## Problem `install.packages("curl")` fails in `reg.devxy.io/r/r-alpine:4.5-3.24` with "package 'curl' is not available for this version of R", on both arches. `curl` is not missing from the repo: it is in `…/latest/src/contrib/4.5/` and `…/4.6/`, the per-minor slots that base R cannot address. The image's repo URL resolves to `…/latest/src/contrib`, whose index does not list it. On `amd64/alpine324` that is 2 886 packages invisible to `install.packages()` (23 on `amd64/noble`) — what issue #63 records as "missing binaries". Two further findings while investigating: - The middleware only ever rewrote the bare `cran.rpkgs.com/src/contrib/…` form, and that form was broken for every Linux client on a stock R user agent: `ALPINE_REGEX`/`UBUNTU_REGEX`/`RHEL_REGEX` need a Posit-style UA that carries the distro, so stock R fell through to `extractOs()` and got redirected to `/amd64/linux-musl/latest/…`, a slot that does not exist. - `PACKAGES*` is served `cdn-cache: BYPASS` (bincraft uploads it `no-store`), so the middleware sees every index request and no purge is needed for routing changes to take effect. ## What this changes **`edge/rpkgs-router.ts`** — the middleware, now a reviewed file in this repo rather than dashboard state. It routes `PACKAGES`, `PACKAGES.gz` and `PACKAGES.rds` into `…/src/contrib/<x.y>/` for slots listed in `UNION_SLOTS`, and nothing else. Tarballs are deliberately left alone. R keeps the `contriburl` it *asked for*, not the one the redirect served it, so every tarball URL is resolved against the flat directory and the union index steers the per-minor ones with a `Path: <x.y>` field. Rewriting a tarball request here would send flat-slot packages into a directory that does not hold them. Also in the script: the phantom `linux-gnu`/`linux-musl` fallback is gone (an unidentifiable distro goes to CRAN, as an unparseable UA already did), and every redirect carries `Cache-Control: no-store` since its target depends on the User-Agent. The macOS branches are unchanged. **`cdn.tf`** — `bunnynet_compute_script.rpkgs_router` with `content = file("edge/rpkgs-router.ts")`, the `UNION_SLOTS` variable, and `middleware_script` pointing at the resource instead of the literal `29277`. `UNION_SLOTS` is empty, so merging and applying this changes no client's behaviour. A slot is added only once bincraft has republished its per-minor index as a union (rpkgs/bincraft#97); routing to a raw per-minor index would hide every package it does not carry. Rolling back is a variable edit, not a deploy. **`specs/`, `plans/`** — the design and the implementation plan, including the two approaches that were rejected (edge-side merge, moving the minor up the path) and why. ## Verification `just edge-test` runs 13 routing cases against the SDK's local server, so what is tested is the artifact that gets deployed; pass-through cases proxy to the real origin. All pass. End to end, with the middleware in front of a locally built union index for `amd64/alpine324` (31 507 records), inside the runtime image: ``` curl: 7.1.0 -> …/latest/src/contrib/4.5 -> curl_7.1.0.tar.gz 717 725 B jsonlite: 2.0.0 -> …/latest/src/contrib -> jsonlite_2.0.0.tar.gz 1 055 849 B ``` `tofu validate` passes. `tofu plan` has not been run: no `BUNNYNET_API_KEY` available in this environment. ## Before applying The script pre-dates this configuration, so it must be adopted, not created: ```sh tofu import bunnynet_compute_script.rpkgs_router 29277 tofu plan ``` The plan should show an in-place `content` update and no replacement of the pull zone. Without the import, tofu creates a second script and repoints the zone at it. Note that `name = "rpkgs-router"` will rename the existing script on apply. ## Not fixed here `install.packages("curl")` on `alpine324` will now *resolve*, then fail to build: that slot's tarballs are byte-identical CRAN **source** tarballs (no `Meta/`, no `Built:` in DESCRIPTION) which the index nevertheless stamps `Built: R 4.5.3; …-linux-musl`. Sampled: `amd64/alpine324` 3/12 binary, `arm64/alpine324` 13/30, `amd64/noble` 12/12, `amd64/alpine323` 17/20. That slot needs a rebuild, tracked separately.

pat-s added 4 commits

2026-08-07 14:10:16 +00:00

docs(specs): design per-R-minor routing for the CDN middleware 5a3b1c4564

- record that per-minor slots are unreachable from install.packages(), with
  the measured size of the gap on alpine324 and noble
- choose a union index written by bincraft over an edge-side merge, and note
  the two experiments that decide it (Path: .. resolves, gzcon does not read
  concatenated members)
- scope the edge script to PACKAGES* only, gated on a UNION_SLOTS variable
- move the script into the repo as a bunnynet_compute_script applied by tofu

docs(plans): add the implementation plan for per-R-minor edge routing 5a0a4fa200

feat(edge): route PACKAGES requests to the per-R-minor slot 628d19d650

- add edge/rpkgs-router.ts, which redirects PACKAGES, PACKAGES.gz and
  PACKAGES.rds into …/src/contrib/<x.y>/ for slots listed in UNION_SLOTS
- leave tarballs alone: R keeps the pre-redirect contrib URL, so the union
  index steers per-minor tarballs with a Path field instead
- stop resolving an unidentifiable distro to a phantom linux-gnu/linux-musl
  slot and send those clients to CRAN
- mark every redirect no-store, since the target depends on the User-Agent
- cover the routing matrix in edge/rpkgs-router.test.ts, run by just edge-test
- correct the spec and plan: the union index carries Path: <x.y> on per-minor
  records, not Path: .. on flat ones

feat(cdn): manage the edge middleware script from this repo 6916eb95f6

- add bunnynet_compute_script.rpkgs_router, sourced from edge/rpkgs-router.ts,
  so the middleware is reviewed and deployed like the rest of the config
- add the UNION_SLOTS script variable, empty until a slot carries a union index
- point the pull zone's middleware_script at the resource instead of the
  hard-coded id 29277, which the script is adopted from via tofu import

pat-s merged commit 2f732457d2 into main

2026-08-07 14:12:22 +00:00

pat-s referenced this pull request from a commit

2026-08-07 14:12:23 +00:00

feat(edge): route PACKAGES requests to the per-R-minor slot (#152)

pat-s deleted branch feat/per-minor-edge-routing

2026-08-07 14:12:23 +00:00