build-cran-binaries/specs/2026-08-07-per-minor-edge-routing-design.md

# Design: Routing clients to per-R-minor binary slots

Date: 2026-08-07
Status: Approved (pending spec review)

## Problem

`bincraft` routes ABI-"risky" packages to a per-minor slot `…/latest/src/contrib/<x.y>/` and indexes every directory independently (`upload_package_index()` calls `cranlike::update_PACKAGES()` on one prefix at a time).
Nothing unions those indices, and `contrib.url()` only ever yields `<repos>/src/contrib`, so no value of `options(repos)` can address a per-minor slot.
Only `uvr` resolves per-minor URLs, which means the per-minor slots are invisible to `install.packages()` by construction.

Measured on 2026-08-07:

| slot              | flat `src/contrib` | `src/contrib/4.5` | unique packages only in the per-minor slot |
| ----------------- | ------------------ | ----------------- | ------------------------------------------ |
| `amd64/alpine324` | 21 640             | 3 310             | 2 886                                      |
| `amd64/noble`     | 24 495             | 398               | 23                                         |

This is what issue #63 records as "missing binaries" on `alpine324`.
The packages are not missing; they are in a directory base R cannot reach.
The user-visible symptom in `reg.devxy.io/r/r-alpine:4.5-3.24` is:

```
> install.packages("curl")
Warning message:
package 'curl' is not available for this version of R
```

A second, unrelated defect exists on the same slot and is **out of scope here**: many `alpine324` tarballs are byte-identical CRAN _source_ tarballs that the index nevertheless stamps `Built: R 4.5.3; …-linux-musl`.
Routing exposes `curl`; only a rebuild of that slot makes it install.

## Goal

Let a stock `install.packages()` see one complete package list for its own R minor, without duplicating tarballs and without an R-version-varying cache key anywhere in the CDN.

## Key constraint that drives the design

R resolves a package's download URL from the index, not from the request path, and it keeps the `contriburl` it _asked for_ rather than the one it was redirected to.
Measured with `options(repos = …/latest)` against a middleware that redirects the index into `4.5/`:

```
curl available: TRUE
curl repo: …/latest/src/contrib          # the flat URL, not the 4.5 one it was served from
```

So the union index is always addressed relative to the **flat** directory, whatever path it was fetched from.
`available.packages()` honours a `Path:` field and folds it into the `Repository` column, which gives the whole routing for free:

- a per-minor record carries `Path: <x.y>`, so its tarball is fetched from `…/src/contrib/<x.y>/`
- a flat record carries no `Path`, so its tarball is fetched from `…/src/contrib/`

Verified end to end against the live CDN with a locally built union index for `amd64/alpine324` (31 507 records):

```
curl:     7.1.0  -> …/latest/src/contrib/4.5   -> curl_7.1.0.tar.gz     717 725 B
jsonlite: 2.0.0  -> …/latest/src/contrib       -> jsonlite_2.0.0.tar.gz 1 055 849 B
```

The corollary is that the edge script must **not** rewrite tarball requests: every tarball URL is already correct when it leaves the client, and redirecting one into `<x.y>/` would break exactly the flat packages the union is meant to preserve.

The complementary trick does not work: R's `gzcon()` reads only the first member of a concatenated gzip stream (10 291 of an expected 31 931 records), so an edge-side merge would have to fully decompress and recompress both indices and additionally 404 `PACKAGES.rds` to stop R preferring it.
That is why the union is produced in `bincraft`, not at the edge.

## Approaches considered

| Approach                                                    | Where the union lives                                        | Verdict                                                                                                     |
| ----------------------------------------------------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------- |
| **A. Union index written by `bincraft` (chosen)**           | per-minor `PACKAGES*`, per-minor entries carry `Path: <x.y>` | Edge does one redirect; `PACKAGES.rds` stays correct; no duplication                                        |
| B. Merge at the edge                                        | middleware fetches both indices, recompresses                | ~2 MB decompress/recompress per cache fill, cache key must include the R minor, breaks R's `.rds` fast path |
| C. Move the minor up the path (`latest/<x.y>/src/contrib/`) | addressable by `options(repos)` directly                     | No edge logic at all, but a full layout migration and breaks the published URL contract                     |

Chosen: **A**.

## Architecture

### bincraft: union index (separate PR)

After writing a per-minor index, republish it as a union of that slot and the flat slot:

1. Read the flat slot's `PACKAGES.rds` and the per-minor slot's own records.
2. Set `Path: <x.y>` on every per-minor record, so its tarball resolves into the per-minor directory.
3. Drop every flat record whose package is already present in the per-minor slot, so the per-minor build always wins, and leave the survivors without a `Path`.
4. Write the merged `PACKAGES`, `PACKAGES.gz` and `PACKAGES.rds` into `…/src/contrib/<x.y>/`.

Guard: refuse to publish a union with fewer records than the flat index it was built from.
A truncated union is worse than no union, because it silently removes packages from every client on that minor.

### Edge script (this repo)

The script routes `PACKAGES`, `PACKAGES.gz` and `PACKAGES.rds` requests, and nothing else.

```
normalize path
parseClient(UA) -> { rMinor, arch, os }      # rMinor from "R (4.5.3 …)" or "R/4.5.3"
darwin branches                               # unchanged
if path is /{arch}/{os}/latest/src/contrib/PACKAGES*
    already under /<x.y>/ ? pass through      # loop guard
    rMinor known && slot in UNION_SLOTS ? 302 -> …/src/contrib/<rMinor>/PACKAGES*
    else pass through                         # flat slot, today's behaviour
if path is /src/contrib/…                     # bare root
    resolve arch + os; unknown -> 302 to CRAN
    then apply the same PACKAGES* rule
else pass through
```

Redirects carry `Cache-Control: no-store`.
Every cacheable URL is therefore UA-independent, and no cache key has to vary by R version.

### Repaired bare-root detection

The bare `https://cran.rpkgs.com` form is currently broken for every Linux client that uses a stock R user agent.
`ALPINE_REGEX`, `UBUNTU_REGEX` and `RHEL_REGEX` only match a Posit-style user agent the user has to set by hand; stock R never carries the distro, so the script falls through to `extractOs()` and redirects to a slot that does not exist:

```
UA: R (4.5.3 x86_64-pc-linux-musl …)  ->  302 /amd64/linux-musl/latest/…   (404)
UA: R (4.5.3 x86_64-pc-linux-gnu  …)  ->  302 /amd64/linux-gnu/latest/…    (404)
```

The fallback to a phantom `linux-musl` / `linux-gnu` slot is removed.
An unidentifiable distro redirects to CRAN, which is the existing behaviour for an unparseable user agent.
The R _minor_ is always present in a stock user agent, so per-minor routing itself does not depend on distro detection.

### Rollout gate

`UNION_SLOTS` is a `bunnynet_compute_script_variable` listing the slots whose per-minor index is already a union.
It is empty by default, so deploying the script changes nothing until `bincraft` has backfilled a slot, and a rollback is a variable edit rather than a code deploy.
All slots currently carry `4.4`, `4.5` and `4.6`; a client on any other minor falls through to the flat slot.

### Deployment from this repo

The script is a file in the repo, applied by the existing OpenTofu configuration:

```
edge/rpkgs-router.ts          # the script
edge/rpkgs-router.test.ts     # UA x path -> expected Location matrix
cdn.tf                        # bunnynet_compute_script + _variable
```

Provider `BunnyWay/bunnynet` v0.17.0 (already pinned) ships `bunnynet_compute_script` with `content` loadable via `file()`, plus `bunnynet_compute_script_variable`.
`middleware_script = bunnynet_compute_script.rpkgs_router.id` replaces the hard-coded `29277`, after a one-time `tofu import` of the existing script.

## Error handling

- Unknown R minor, or a slot not listed in `UNION_SLOTS`: pass through to the flat slot.
  The client sees exactly today's behaviour.
- Unparseable distro on the bare-root form: redirect to CRAN.
- A request already under `…/src/contrib/<x.y>/`: pass through, so a redirect can never loop.
- A per-minor slot that does not exist for a listed minor: the client gets the origin's 404.
  `UNION_SLOTS` is the operator's assertion that the slot is ready, so this is a configuration error, not a runtime condition to paper over.

## Testing

Local, before any apply: `deno run -A edge/rpkgs-router.ts` serves the middleware against the real origin, so `edge/rpkgs-router.test.ts` drives the whole matrix against that local server.

- User agent matrix: R 4.4 / 4.5 / 4.6 on musl and gnu, both arches, Posit-style and stock forms, plus a darwin UA and a non-R UA.
- Path matrix: `PACKAGES`, `PACKAGES.gz`, `PACKAGES.rds`, a tarball, a path already under `4.5/`, and `/src/contrib/…` on the bare root.
- Assertion is the `Location` header (or its absence), not the body.

After apply, a smoke test against `cran.rpkgs.com`:

- `available.packages()` inside `reg.devxy.io/r/r-alpine:4.5-3.24` returns the union count, and `"curl" %in% rownames(...)` is `TRUE`.
- A flat-slot package still downloads from `…/src/contrib/`, and a per-minor package downloads from `…/src/contrib/<x.y>/`.

## Out of scope

- `Meta/archive.rds` stays flat-only, so `remotes::install_version()` does not see per-minor archives.
- The `alpine324` source-tarball defect: that slot serves CRAN sources stamped as binaries, and needs a rebuild independent of this work.
- Any change to how `uvr` resolves per-minor URLs; it already addresses the slots directly.

## Split of work

1. `bincraft`: union index writer plus its guard, and a re-index of one slot to validate.
2. This repo: `edge/rpkgs-router.ts`, its test matrix, and the `cdn.tf` resources with `UNION_SLOTS` empty.
3. Enable `UNION_SLOTS` slot by slot as `bincraft` backfills them.