Commit graph build-cran-binaries/.crow/backfill-abi-classification.yaml
Author SHA1 Message Date
930a4acd12
fix(ci): install the R packages pipelines use instead of relying on the image
bincraft v5.5.1 dropped lubridate, dplyr, purrr, data.table and others from
its imports, so the pipelines' own calls now depend on whatever the build-env
image's site library or the shared R_LIBS_USER cache happens to hold. The arm64
redhat:9 image has no progressr under R 4.4.3, which fails every
process-cran-updates-redhat-9-arm64 run since v5.5.1.

- Install progressr and lubridate for process-updates, which call them inline.
- Install purrr for packages-to-build.R and data.table for the weekly audit.
- Drop the unused dplyr and progressr library() calls from packages-to-build.R.
- Install RPostgres in every pipeline that talks to the metadata DB, since
  bincraft silently skips the DB (and the per-minor index aborts) without it.
- Reinstall bincraft in the prune re-index step, whose /tmp library does not
  survive from the prune step.
2026-10-20 08:46:00 +00:00
853b924765 feat(abi): backfill classifier verdicts for pre-per-minor packages (#199)
All checks were successful
ci/crow/manual/rebuild-abi-gap/17 Pipeline was successful
ci/crow/manual/backfill-abi-classification Pipeline was successful
## Why

bincraft PR #124 makes the `abi_classification` cache authoritative for the risky-package set, breaking the circularity where a package counted as ABI-risky only once it had already been built per-minor.

On its own that changes nothing for the packages actually at risk. The cache only holds verdicts for packages that passed through the build pipeline after the per-minor scheme existed; everything older has no verdict, and no verdict means the flat binary is carried into every per-minor index. That is exactly how a `base64enc` built under R 4.4.3 reached an R 4.6.1 client:

```
base64enc.so: undefined symbol: SETLENGTH
```

This is the backfill that gives those packages a verdict.

## What it does

`local/backfill-abi-classification.R` enumerates a slot's flat index and runs each compiled package through `bincraft::classify_r_minor_sensitive()`, which looks the verdict up in the cache, and on a miss downloads the CRAN source, classifies it and stores the result.

The verdict is a property of the source, independent of OS, arch and R minor, so **one run populates the cache for every slot**. The arch/codename arguments only choose which flat index to enumerate.

Only `NeedsCompilation=yes` packages are classified: a package with no compiled code has no symbols to bind and cannot fail this way, so proving it by download would be wasted work. On amd64/resolute that is 4585 packages rather than 23218.

Re-running is cheap and resumable: cached verdicts return without touching the network, so a second run costs one query per package and downloads only the remainder.

## Verification

The enumeration was run against the live indexes and matches an independent symbol scan of the same slot exactly:

```
flat binaries: 23218 | due classification: 4585
base64enc included: TRUE
lobstr included:    TRUE
```

4585 is the same population I scanned with `nm -D --undefined-only` when establishing the blast radius, and both packages confirmed to break under 4.6 are in it.

`crow lint` passes.

## Sequencing

Merge after bincraft #124 and after a bincraft release the pipelines pick up, otherwise the verdicts are written but nothing reads them yet. Writing them early is harmless.

Then re-index the slots so the per-minor indexes are rebuilt with the widened risky set. Expect coverage in the per-minor indexes to *drop* slightly as newly-recognised risky packages are dropped rather than served from a mismatched minor; those then need per-minor builds, either through the normal pipeline or the targeted workflow in devxy/build-cran-binaries#198.

Reviewed-on: #199
2026-09-04 10:34:49 +00:00