Author	SHA1	Message	Date
pat-s	d9a652904e	fix(abi): inspect flat objects of packages CRAN has removed ... The backfill decided which objects were worth inspecting by looking the package up in CRAN's current index. CRAN lists only what it currently ships, so a package it has archived is absent, is never recognised as compiled, and is never inspected. No verdict is ever recorded for it. `union_index_records()` meanwhile reads the slot's *own* `NeedsCompilation`, finds `yes`, and drops the record when the source classifier flags the package and no safe verdict exists to clear it. The repair cannot rescue it either, because it works from recorded verdicts and there is no row. The record is therefore dropped from every per-minor index on every re-index, permanently. `mixedMem` and `vcpen`, both removed from CRAN, were stuck out of all three rhel8 per-minor indexes on both architectures that way, surviving a backfill, two re-indexes and a repair. Read `NeedsCompilation` from the slot's own flat index instead, which describes exactly the objects being inspected. It is fetched from the bucket rather than over HTTPS because this runs inside R, where the edge would route the request to a per-minor index. Keyed on `<package>_<version>` rather than the package name: a slot can carry several versions and only some of them compile.	2026-09-09 18:26:59 +00:00
pat-s	844ad9749b	fix(abi): enumerate the flat slot from S3, not through the edge (#203) ... ## Why The first backfill run reported: ``` amd64/resolute: 27105 flat binaries, 8187 compiled and worth inspecting Done: 300 inspected, 7 safe for every minor, 1 unsafe, 292 skipped ``` 292 of 300 skipped, and `27105` is larger than the flat index (~24k). Both have the same cause: the script read `PACKAGES.gz` over HTTPS from inside R, which sends an R User-Agent, so **the edge routed it to a per-minor index**. It then enumerated union records carrying `Path: <minor>`, whose objects do not exist at the flat path, and every download 404'd. An ironic failure given what this whole change is about, and one the trial run existed to catch. ## What changed - List the flat slot with `s3fs::s3_dir_ls()`, keeping only objects with no `<minor>/` prefix and nothing under `Archive/`. The bucket cannot be routed and is authoritative about what is actually in the flat slot. - Fetch each object from S3 too. The edge does not rewrite tarball requests, but it does cache them, and the verdict is about the object. - `NeedsCompilation` still comes from CRAN's own index, which is not behind our edge. ## Verification The next trial run is the check: `flat objects` should now match the flat index rather than exceed it, and `skipped` should be near zero. Reviewed-on: #203	2026-09-05 09:41:57 +00:00
pat-s	c216e44578	feat(abi): backfill per-object flat-slot safety verdicts (#202) ... ## Why bincraft #128 lets `union_index_records()` keep a carried flat record when that specific object is recorded safe. On its own it changes nothing for the records already being dropped, because they are legacy objects with no verdict: ``` 4.4: 16398 dropped across 9 indexes (~1822 each) 4.5: 20076 dropped across 9 indexes (~2230 each) 4.6: 15532 dropped across 9 indexes (~1726 each) ``` Every one of those resolves to CRAN source for clients the routing serves correctly. This backfill inspects the objects already in the bucket and writes their verdicts, which is what actually restores the coverage. ## How the verdict is made By symbols, not heuristics. A binary fails at `dyn.load()` under minor M exactly when it references a symbol M does not export, and the volatile set is derived by diffing the `libR.so` exports of the R builds installed in the image — so a new R release updates it with no curated list to maintain. That matters here because the source-level classifier calls **93%** of compiled packages risky (`LinkingTo: Rcpp` alone is enough), while a symbol scan of the same slot found **17** packages that break under 4.6 and **72** under 4.4. Recording the real answer per object is what makes the difference between ~2000 dropped records and the few dozen that genuinely need dropping. ## Safety - **Refuses to run where fewer than two R minors are installed.** With nothing to compare, every verdict would read "safe" — writing a bucket full of false clearances is far worse than writing none. - Skips packages with no compiled code: they cannot fail this way and are never dropped, so downloading them to prove it is wasted. - Resumable; already-recorded objects cost one query. `crow lint` passes. ## Sequencing Merge and release bincraft #128 first, then run this per slot, then re-index. Verifying the drop counts fall is the point of the exercise, so that re-index is where the fix is confirmed rather than assumed. Reviewed-on: #202	2026-09-05 09:35:47 +00:00