| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 416a2d64d9 |
chore(alpine): drop the deprecated Alpine 3.22 slot (#213)
All checks were successful
ci/crow/manual/verify-repo-routing/11 Pipeline was successful
ci/crow/manual/verify-repo-routing/13 Pipeline was successful
ci/crow/manual/verify-repo-routing/2 Pipeline was successful
ci/crow/manual/verify-repo-routing/15 Pipeline was successful
ci/crow/manual/verify-repo-routing/1 Pipeline was successful
ci/crow/manual/verify-repo-routing/3 Pipeline was successful
ci/crow/manual/verify-repo-routing/4 Pipeline was successful
ci/crow/manual/verify-repo-routing/7 Pipeline was successful
ci/crow/manual/verify-repo-routing/12 Pipeline was successful
ci/crow/manual/verify-repo-routing/8 Pipeline was successful
ci/crow/manual/verify-repo-routing/9 Pipeline was successful
ci/crow/manual/verify-repo-routing/14 Pipeline was successful
ci/crow/manual/verify-repo-routing/10 Pipeline was successful
ci/crow/manual/verify-repo-routing/16 Pipeline was successful
ci/crow/manual/verify-repo-routing/5 Pipeline was successful
ci/crow/manual/verify-repo-routing/6 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
## Why Alpine 3.22 is deprecated and should be dropped. It was never a published slot to begin with. `local.rpkgs_slots` in `cdn.tf` enumerates eight distros and alpine322 is not one of them, so `UNION_SLOTS` never gated per-minor rewriting for it. The website's `DistroId` union lists the same eight. Every request to the slot therefore fell through to the flat index, which holds R 4.5 builds. ## Evidence The fleet-wide routing verification (pipeline 12591, `verify_routing=all`) measured this end to end. Across all 16 published slots, both architectures, three R minors and both contractual URL forms: **1056 of 1056 install-and-load checks passed.** Every failure in the run was alpine322, identical on amd64 and arm64: ``` amd64/alpine322 R 4.4 via arch|generic|flat: 8 of 11 amd64/alpine322 R 4.6 via arch|generic|flat: 10 of 11 arm64/alpine322 R 4.4 via arch|generic|flat: 8 of 11 arm64/alpine322 R 4.6 via arch|generic|flat: 10 of 11 ``` The tell is that all three forms agree. On a routed slot `arch` and `generic` diverge from `flat`, because that divergence *is* the routing. On alpine322 there is nothing to diverge from. R 4.5 clients were 11 of 11 on every form, because flat happens to be what they need. The data itself is not broken, only unreachable. `alpine322/contrib/4.4/data.table_1.18.6.1.tar.gz` is genuinely `Built: R 4.4.3` and the 4.6 object is `Built: R 4.6.0`. Correct objects, correct per-minor indexes, never served. ## What changed - alpine-322 matrix rows and dropdown options removed from the 12 pipelines that carried them, so `=all` no longer fans out to a slot nobody can reach. - `3.22` removed from the `OS_VERSION` dropdowns in `build-all-versions.yaml` and `repair-built-stamp.yaml`; the latter's default moves to `3.23`. - Stale `3.22` examples updated in `local/packages-to-build.R`. `crow lint .crow/` reports all 18 configs valid. ## Also done outside this diff The six alpine-322 cron jobs are deleted from the Crow server, since a cron whose matrix row is gone keeps firing against nothing: ``` 5 process-cran-updates-alpine-322-amd64 0 5/8 * * * 6 process-cran-updates-alpine-322-arm64 30 5/8 * * * 68 weekly-audit-missing-alpine-322-amd64 20 0 * * 0 69 weekly-audit-missing-alpine-322-arm64 30 0 * * 0 82 weekly-rebuild-missing-alpine-322-amd64 0 4 * * 0 83 weekly-rebuild-missing-alpine-322-arm64 10 4 * * 0 ``` They are listed here so they can be recreated if this is ever reversed. ## Not done The published objects and indexes under `amd64/alpine322` and `arm64/alpine322` are left in the bucket. Deleting them is a separate decision, and nothing routes to them in the meantime. Reviewed-on: #213 |
|||
| f3984f337d |
fix(ci): authenticate uvr's GitHub resolution in the index workflows (#204)
All checks were successful
ci/crow/cron/weekly-rebuild-missing/35 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/3 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/36 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/12 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/40 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/41 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/42 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/14 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/46 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/48 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/47 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/16 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
## Why
The post-backfill re-index (12257) reported failure. Five slots could not write their indexes:
```
WARN Failed to fetch git Remote pat-s/cranlike@s3 (GitHub API error: 403 Forbidden);
falling back to registry resolution
! Attempt 1/3 to update PACKAGES failed (unused argument (built = built)). Retrying in 5s...
Error: Failed to update PACKAGES after 3 attempts.
Last error: unused argument (built = built)
```
uvr resolves bincraft's dependencies through the GitHub API, including the `pat-s/cranlike@s3` fork. Unauthenticated, that is rate limited. uvr does not fail on the 403 — it **warns and falls back to the registry copy of cranlike**, which has no `built` parameter, so `update_PACKAGES(built = ...)` dies three retries later.
Affected: alpine323, rhel8, rhel9, rhel10, noble. The remaining 13 slots resolved the fork fine and indexed correctly, which is what makes this insidious: the same pipeline passes or fails on identical input depending on the rate-limit window.
`build-all-versions.yaml` and `rebuild-abi-gap.yaml` already pass `GITHUB_PAT`, which is why they were never affected.
## What changed
Adds the `GITHUB_PAT` secret to `reindex.yaml`, `weekly-rebuild-reindex.yaml` and `backfill-flat-safety.yaml`, with a comment recording why its absence is quiet rather than loud.
`crow lint` passes on all three.
## Context
The re-index itself did what it was meant to. Drops per per-minor index fell from ~1800-2200 to:
```
4.4: avg 129 4.5: avg 172 4.6: avg 145
```
so the per-object safety verdicts recovered roughly 93% of the coverage while still dropping every object that genuinely cannot load. This PR is about the five slots that never got to apply it.
Reviewed-on: #204
|
|||
| e1114bc145 |
fix(ci): give the index writers the metadata database password (#201)
All checks were successful
ci/crow/manual/reindex/11 Pipeline was successful
ci/crow/manual/reindex/9 Pipeline was successful
ci/crow/manual/reindex/15 Pipeline was successful
ci/crow/manual/reindex/13 Pipeline was successful
ci/crow/manual/reindex/17 Pipeline was successful
ci/crow/manual/reindex/1 Pipeline was successful
ci/crow/manual/reindex/3 Pipeline was successful
ci/crow/manual/reindex/5 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/manual/reindex/2 Pipeline was successful
ci/crow/manual/reindex/4 Pipeline was successful
ci/crow/manual/reindex/8 Pipeline was successful
ci/crow/manual/reindex/6 Pipeline was successful
ci/crow/manual/reindex/10 Pipeline was successful
ci/crow/manual/reindex/12 Pipeline was successful
ci/crow/manual/reindex/14 Pipeline was successful
ci/crow/manual/reindex/16 Pipeline was successful
ci/crow/manual/reindex/18 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
## Why Pipeline 12207 logs one of these per slot: ``` Error: connection to server at "r-binaries.devxy.io" (116.203.165.191), port 15432 failed: fe_sendauth: no password supplied ``` bincraft's index build calls `abi_cache_risky_packages()` to decide which packages may be carried from the flat slot into a per-minor index. `reindex.yaml` and `weekly-rebuild-reindex.yaml` never passed `PGPASS`, so the query cannot connect. It fails soft by design, which is what makes this worth fixing rather than tolerating: the run still reports **success** while using an empty cache-derived risky set. The index is then written as though no package were known risky from the classifier, which is the exact state the cache was added to fix. `build-all-versions.yaml` already passes `PGPASS`, which is why classification behaved correctly there and silently degraded here. ## What changed Adds the `PGPASS` secret to both index-writing workflows, with a comment explaining why its absence is invisible rather than loud. `crow lint` passes on both files. ## Note The tarball-derived half of the risky set still worked, so indexes written without this are not wrong so much as incomplete: they miss packages that are known risky from classification but have no per-minor tarball yet. A re-index after this merges applies the full set. Reviewed-on: #201 |
|||
| a2923bf063 |
fix(cdn): purge the staging zone too (#181)
## Motivation `cran-rpkgs-test` was added as a second pull zone on the same B2 origin, but it was never added to `BUNNY_PULLZONES`. It therefore keeps serving pre-reindex indexes behind the same ~370 day `cache_expiration_time` as production, and nothing ever refreshes it. That is not cosmetic. The zone exists to be measured, and a verification run against it measures whatever the edge still holds: ``` production: regressions=0 AGHmatrix Path=NA Built=R 4.5.3; x86_64-pc staging : regressions=161 AGHmatrix Path=4.5 Built=(none) ``` Same objects, same origin, 161 phantom regressions. I chased that number through two wrong diagnoses before noticing production and staging disagreed. ## Change Add `cran-rpkgs-test.b-cdn.net` to the purge list in both reindex pipelines. ## Note A `Cache-Control: no-cache` request header is not a substitute. It was added to the verification script and did **not** clear this: bunny does not honour it for these objects. Purging is the mechanism that works. Reviewed-on: #181 |
|||
| f3077677d7 |
ci: add a reindex-only manual workflow (#177)
## Motivation `weekly-rebuild-reindex` exists to run after `weekly-rebuild-missing`: it declares `depends_on: weekly-rebuild-missing` and is gated on that workflow's `weekly_rebuild_missing` variable. Triggering it manually therefore also starts hours of package rebuilds. That is the wrong tool when only the index needs regenerating. After rpkgs/bincraft#113 (v5.1.5), which changes how `union_index_records()` decides what a per-minor index steers to, every object in the bucket is already correct and only `PACKAGES*` is stale. Rebuilding to fix an index is pure waste, and the natural cron would take a full cycle to reach every slot. ## Change Adds `.crow/reindex.yaml`: the index half on its own, manual only, no dependency on a rebuild. It reuses the same matrix and the same steps as `weekly-rebuild-reindex` — install the latest bincraft release, republish the generic index, loop the installed R versions republishing each per-minor index, purge the edge. No package is built. Gated on a new `reindex` variable so it cannot be started by the rebuild gate, defaulting to `none` so a manual pipeline creation (which instantiates every file in `.crow/`) matches no matrix row. ```sh crow pipeline create devxy/build-cran-binaries --var reindex=all crow pipeline create devxy/build-cran-binaries --var reindex=ubuntu-2404-amd64 ``` ## Verification - `crow lint .crow/` passes. - Gate is manual-only and evaluates `reindex`, with no `depends_on` and no `runs_on` carried over from the rebuild coupling. Reviewed-on: #177 |
| Author | SHA1 | Date | |
|---|---|---|---|
| 416a2d64d9 | |||
| f3984f337d | |||
| e1114bc145 | |||
| a2923bf063 | |||
| f3077677d7 |