Author	SHA1	Message	Date
pat-s	767dd9ee5d	fix(abi): inspect flat objects of packages CRAN has removed ... The backfill decided which objects were worth inspecting by looking the package up in CRAN's current index. CRAN lists only what it currently ships, so a package it has archived is absent, is never recognised as compiled, and is never inspected. No verdict is ever recorded for it. `union_index_records()` meanwhile reads the slot's *own* `NeedsCompilation`, finds `yes`, and drops the record when the source classifier flags the package and no safe verdict exists to clear it. The repair cannot rescue it either, because it works from recorded verdicts and there is no row. The record is therefore dropped from every per-minor index on every re-index, permanently. `mixedMem` and `vcpen`, both removed from CRAN, were stuck out of all three rhel8 per-minor indexes on both architectures that way, surviving a backfill, two re-indexes and a repair. Read `NeedsCompilation` from the slot's own flat index instead, which describes exactly the objects being inspected. It is fetched from the bucket rather than over HTTPS because this runs inside R, where the edge would route the request to a per-minor index. Keyed on `<package>_<version>` rather than the package name: a slot can carry several versions and only some of them compile.	2026-09-09 18:26:38 +00:00
pat-s	2f732457d2	feat(edge): route PACKAGES requests to the per-R-minor slot (#152) ... ## Problem `install.packages("curl")` fails in `reg.devxy.io/r/r-alpine:4.5-3.24` with "package 'curl' is not available for this version of R", on both arches. `curl` is not missing from the repo: it is in `…/latest/src/contrib/4.5/` and `…/4.6/`, the per-minor slots that base R cannot address. The image's repo URL resolves to `…/latest/src/contrib`, whose index does not list it. On `amd64/alpine324` that is 2 886 packages invisible to `install.packages()` (23 on `amd64/noble`) — what issue #63 records as "missing binaries". Two further findings while investigating: - The middleware only ever rewrote the bare `cran.rpkgs.com/src/contrib/…` form, and that form was broken for every Linux client on a stock R user agent: `ALPINE_REGEX`/`UBUNTU_REGEX`/`RHEL_REGEX` need a Posit-style UA that carries the distro, so stock R fell through to `extractOs()` and got redirected to `/amd64/linux-musl/latest/…`, a slot that does not exist. - `PACKAGES*` is served `cdn-cache: BYPASS` (bincraft uploads it `no-store`), so the middleware sees every index request and no purge is needed for routing changes to take effect. ## What this changes **`edge/rpkgs-router.ts`** — the middleware, now a reviewed file in this repo rather than dashboard state. It routes `PACKAGES`, `PACKAGES.gz` and `PACKAGES.rds` into `…/src/contrib/<x.y>/` for slots listed in `UNION_SLOTS`, and nothing else. Tarballs are deliberately left alone. R keeps the `contriburl` it *asked for*, not the one the redirect served it, so every tarball URL is resolved against the flat directory and the union index steers the per-minor ones with a `Path: <x.y>` field. Rewriting a tarball request here would send flat-slot packages into a directory that does not hold them. Also in the script: the phantom `linux-gnu`/`linux-musl` fallback is gone (an unidentifiable distro goes to CRAN, as an unparseable UA already did), and every redirect carries `Cache-Control: no-store` since its target depends on the User-Agent. The macOS branches are unchanged. **`cdn.tf`** — `bunnynet_compute_script.rpkgs_router` with `content = file("edge/rpkgs-router.ts")`, the `UNION_SLOTS` variable, and `middleware_script` pointing at the resource instead of the literal `29277`. `UNION_SLOTS` is empty, so merging and applying this changes no client's behaviour. A slot is added only once bincraft has republished its per-minor index as a union (rpkgs/bincraft#97); routing to a raw per-minor index would hide every package it does not carry. Rolling back is a variable edit, not a deploy. **`specs/`, `plans/`** — the design and the implementation plan, including the two approaches that were rejected (edge-side merge, moving the minor up the path) and why. ## Verification `just edge-test` runs 13 routing cases against the SDK's local server, so what is tested is the artifact that gets deployed; pass-through cases proxy to the real origin. All pass. End to end, with the middleware in front of a locally built union index for `amd64/alpine324` (31 507 records), inside the runtime image: ``` curl: 7.1.0 -> …/latest/src/contrib/4.5 -> curl_7.1.0.tar.gz 717 725 B jsonlite: 2.0.0 -> …/latest/src/contrib -> jsonlite_2.0.0.tar.gz 1 055 849 B ``` `tofu validate` passes. `tofu plan` has not been run: no `BUNNYNET_API_KEY` available in this environment. ## Before applying The script pre-dates this configuration, so it must be adopted, not created: ```sh tofu import bunnynet_compute_script.rpkgs_router 29277 tofu plan ``` The plan should show an in-place `content` update and no replacement of the pull zone. Without the import, tofu creates a second script and repoints the zone at it. Note that `name = "rpkgs-router"` will rename the existing script on apply. ## Not fixed here `install.packages("curl")` on `alpine324` will now *resolve*, then fail to build: that slot's tarballs are byte-identical CRAN **source** tarballs (no `Meta/`, no `Built:` in DESCRIPTION) which the index nevertheless stamps `Built: R 4.5.3; …-linux-musl`. Sampled: `amd64/alpine324` 3/12 binary, `arm64/alpine324` 13/30, `amd64/noble` 12/12, `amd64/alpine323` 17/20. That slot needs a rebuild, tracked separately. Reviewed-on: #152	2026-08-07 14:12:22 +00:00
pat-s	55a18fd87d	feat: patch registry + wiring for per-package patching (#103) ... ## Summary Adds the curated **patch registry** and wiring that drives bincraft's new package-patching mechanism (see bincraft PR `feat/package-patching`). Lets specific packages be patched (env/configure/Makevars overrides or source diffs) before pak installs them — including as transitive dependencies — so compiler-/OS-specific failures like RcppParallel's bundled TBB stop cascading. ## What's included - `local/patches/registry.json` — initial entry: RcppParallel with `RCPP_PARALLEL_USE_TBB=0` for alpine / ubuntu-2604, plus `local/patches/README.md` schema docs. - `local/validate-patches.R` — validates schema, referenced patch files, and ambiguous overlaps; clean failure + exit 1 (no stacktrace). - `.pre-commit-config.yaml` — a `validate-patches` hook (re-runs when the registry or the validator changes). - `local/build-one.R` / `local/build-all.R` — pass `patches = "local/patches"` to `bincraft::build_binary_package()`. - `specs/2026-06-30-package-patching-design.md` and `plans/2026-06-30-package-patching-implementation.md`. ## ⚠️ Merge ordering (blocker) This PR adds a `patches = ...` argument to `build_binary_package()` calls. The `.crow/*.yaml` workflows currently pin bincraft **v4.2.3**, which does not accept that argument — CI will error with `unused argument (patches=...)` until: 1. bincraft **v4.3.0** is released (PR `feat/package-patching`), and 2. the pin is bumped in `.crow/build-all-versions-install-deps.yaml`, `.crow/build-all-versions.yaml`, and `.crow/process-updates.yaml`. The `.crow` pin bump will be added to this PR once bincraft v4.3.0 is tagged. Do not merge before then. Reviewed-on: #103	2026-06-30 08:50:19 +00:00