| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| e77686affc |
fix(ci): install the R packages pipelines use instead of relying on the image (#238)
Some checks failed
ci/crow/cron/process-updates/5 Pipeline is running
ci/crow/cron/process-updates/7 Pipeline failed
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline is running
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline is running
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline is running
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline is running
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
Every `process-cran-updates-redhat-9-arm64` run since bincraft v5.5.1 fails with `there is no package called 'progressr'` (e.g. pipeline 14179). ## Cause bincraft v5.5.1 dropped `lubridate`, `dplyr`, `purrr`, `data.table`, `gert`, `gh` and `cranlike` from its imports. The pipelines call some of these directly, and so far got them from the build-env image's site library or the shared `R_LIBS_USER` cache. The arm64 `redhat:9` image has no `progressr` (and no `bincraft`) under R 4.4.3, and no `RPostgres`, `dplyr` or `purrr` under R 4.5.3. The amd64 image has all of them, which is why only this slot broke. ## Changes - `process-updates.yaml`: install `progressr` and `lubridate`, which the inline R calls use (`progressr::handlers`, `$INTERVAL`). - `build-all-versions*.yaml`: install `purrr` for `packages-to-build.R`. - `weekly-audit-missing.yaml`: install `data.table` for the audit script. - `local/packages-to-build.R`: drop the unused `dplyr` and `progressr` `library()` calls. - `reindex`, `weekly-rebuild-reindex`, `weekly-rebuild-missing`, `backfill-*`, `rebuild-abi-gap`, `repair-unsafe-flat-objects`, `prune-stale-flat-binaries`: install `RPostgres`, because bincraft quietly skips the metadata DB without it and the per-minor index build aborts. - `prune-stale-flat-binaries.yaml`: reinstall bincraft in the re-index step, since its `/tmp/R-pkgs` library doesn't survive from the prune step. ## Verification - `crow lint .crow/` passes; `prek` passes on the changed files. - The image contents were checked by exporting the arm64 and amd64 `build-env-redhat:9` images. - Not yet run in CI. Reviewed-on: #238 |
|||
| 853b924765 |
feat(abi): backfill classifier verdicts for pre-per-minor packages (#199)
## Why bincraft PR #124 makes the `abi_classification` cache authoritative for the risky-package set, breaking the circularity where a package counted as ABI-risky only once it had already been built per-minor. On its own that changes nothing for the packages actually at risk. The cache only holds verdicts for packages that passed through the build pipeline after the per-minor scheme existed; everything older has no verdict, and no verdict means the flat binary is carried into every per-minor index. That is exactly how a `base64enc` built under R 4.4.3 reached an R 4.6.1 client: ``` base64enc.so: undefined symbol: SETLENGTH ``` This is the backfill that gives those packages a verdict. ## What it does `local/backfill-abi-classification.R` enumerates a slot's flat index and runs each compiled package through `bincraft::classify_r_minor_sensitive()`, which looks the verdict up in the cache, and on a miss downloads the CRAN source, classifies it and stores the result. The verdict is a property of the source, independent of OS, arch and R minor, so **one run populates the cache for every slot**. The arch/codename arguments only choose which flat index to enumerate. Only `NeedsCompilation=yes` packages are classified: a package with no compiled code has no symbols to bind and cannot fail this way, so proving it by download would be wasted work. On amd64/resolute that is 4585 packages rather than 23218. Re-running is cheap and resumable: cached verdicts return without touching the network, so a second run costs one query per package and downloads only the remainder. ## Verification The enumeration was run against the live indexes and matches an independent symbol scan of the same slot exactly: ``` flat binaries: 23218 | due classification: 4585 base64enc included: TRUE lobstr included: TRUE ``` 4585 is the same population I scanned with `nm -D --undefined-only` when establishing the blast radius, and both packages confirmed to break under 4.6 are in it. `crow lint` passes. ## Sequencing Merge after bincraft #124 and after a bincraft release the pipelines pick up, otherwise the verdicts are written but nothing reads them yet. Writing them early is harmless. Then re-index the slots so the per-minor indexes are rebuilt with the widened risky set. Expect coverage in the per-minor indexes to *drop* slightly as newly-recognised risky packages are dropped rather than served from a mismatched minor; those then need per-minor builds, either through the normal pipeline or the targeted workflow in devxy/build-cran-binaries#198. Reviewed-on: #199 |
| Author | SHA1 | Date | |
|---|---|---|---|
| e77686affc | |||
| 853b924765 |
