Commit graph build-cran-binaries/.crow/reindex.yaml
Author SHA1 Message Date
f3984f337d fix(ci): authenticate uvr's GitHub resolution in the index workflows (#204)
All checks were successful
ci/crow/cron/weekly-rebuild-missing/35 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/3 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/36 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/12 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/40 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/41 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/42 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/14 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/46 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/48 Pipeline was successful
ci/crow/cron/weekly-rebuild-missing/47 Pipeline was successful
ci/crow/cron/weekly-rebuild-reindex/16 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
## Why

The post-backfill re-index (12257) reported failure. Five slots could not write their indexes:

```
WARN Failed to fetch git Remote pat-s/cranlike@s3 (GitHub API error: 403 Forbidden);
     falling back to registry resolution
! Attempt 1/3 to update PACKAGES failed (unused argument (built = built)). Retrying in 5s...
Error: Failed to update PACKAGES after 3 attempts.
       Last error: unused argument (built = built)
```

uvr resolves bincraft's dependencies through the GitHub API, including the `pat-s/cranlike@s3` fork. Unauthenticated, that is rate limited. uvr does not fail on the 403 — it **warns and falls back to the registry copy of cranlike**, which has no `built` parameter, so `update_PACKAGES(built = ...)` dies three retries later.

Affected: alpine323, rhel8, rhel9, rhel10, noble. The remaining 13 slots resolved the fork fine and indexed correctly, which is what makes this insidious: the same pipeline passes or fails on identical input depending on the rate-limit window.

`build-all-versions.yaml` and `rebuild-abi-gap.yaml` already pass `GITHUB_PAT`, which is why they were never affected.

## What changed

Adds the `GITHUB_PAT` secret to `reindex.yaml`, `weekly-rebuild-reindex.yaml` and `backfill-flat-safety.yaml`, with a comment recording why its absence is quiet rather than loud.

`crow lint` passes on all three.

## Context

The re-index itself did what it was meant to. Drops per per-minor index fell from ~1800-2200 to:

```
4.4: avg 129    4.5: avg 172    4.6: avg 145
```

so the per-object safety verdicts recovered roughly 93% of the coverage while still dropping every object that genuinely cannot load. This PR is about the five slots that never got to apply it.

Reviewed-on: #204
2026-09-05 15:59:16 +00:00
e1114bc145 fix(ci): give the index writers the metadata database password (#201)
All checks were successful
ci/crow/manual/reindex/11 Pipeline was successful
ci/crow/manual/reindex/9 Pipeline was successful
ci/crow/manual/reindex/15 Pipeline was successful
ci/crow/manual/reindex/13 Pipeline was successful
ci/crow/manual/reindex/17 Pipeline was successful
ci/crow/manual/reindex/1 Pipeline was successful
ci/crow/manual/reindex/3 Pipeline was successful
ci/crow/manual/reindex/5 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/manual/reindex/2 Pipeline was successful
ci/crow/manual/reindex/4 Pipeline was successful
ci/crow/manual/reindex/8 Pipeline was successful
ci/crow/manual/reindex/6 Pipeline was successful
ci/crow/manual/reindex/10 Pipeline was successful
ci/crow/manual/reindex/12 Pipeline was successful
ci/crow/manual/reindex/14 Pipeline was successful
ci/crow/manual/reindex/16 Pipeline was successful
ci/crow/manual/reindex/18 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
## Why

Pipeline 12207 logs one of these per slot:

```
Error: connection to server at "r-binaries.devxy.io" (116.203.165.191), port 15432 failed:
  fe_sendauth: no password supplied
```

bincraft's index build calls `abi_cache_risky_packages()` to decide which packages may be carried from the flat slot into a per-minor index. `reindex.yaml` and `weekly-rebuild-reindex.yaml` never passed `PGPASS`, so the query cannot connect.

It fails soft by design, which is what makes this worth fixing rather than tolerating: the run still reports **success** while using an empty cache-derived risky set. The index is then written as though no package were known risky from the classifier, which is the exact state the cache was added to fix.

`build-all-versions.yaml` already passes `PGPASS`, which is why classification behaved correctly there and silently degraded here.

## What changed

Adds the `PGPASS` secret to both index-writing workflows, with a comment explaining why its absence is invisible rather than loud.

`crow lint` passes on both files.

## Note

The tarball-derived half of the risky set still worked, so indexes written without this are not wrong so much as incomplete: they miss packages that are known risky from classification but have no per-minor tarball yet. A re-index after this merges applies the full set.

Reviewed-on: #201
2026-09-04 19:26:55 +00:00
a2923bf063 fix(cdn): purge the staging zone too (#181)
All checks were successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
## Motivation

`cran-rpkgs-test` was added as a second pull zone on the same B2 origin, but it was never added to `BUNNY_PULLZONES`. It therefore keeps serving pre-reindex indexes behind the same ~370 day `cache_expiration_time` as production, and nothing ever refreshes it.

That is not cosmetic. The zone exists to be measured, and a verification run against it measures whatever the edge still holds:

```
production: regressions=0    AGHmatrix Path=NA   Built=R 4.5.3; x86_64-pc
staging   : regressions=161  AGHmatrix Path=4.5  Built=(none)
```

Same objects, same origin, 161 phantom regressions. I chased that number through two wrong diagnoses before noticing production and staging disagreed.

## Change

Add `cran-rpkgs-test.b-cdn.net` to the purge list in both reindex pipelines.

## Note

A `Cache-Control: no-cache` request header is not a substitute. It was added to the verification script and did **not** clear this: bunny does not honour it for these objects. Purging is the mechanism that works.

Reviewed-on: #181
2026-08-31 10:31:35 +00:00
f3077677d7 ci: add a reindex-only manual workflow (#177)
## Motivation

`weekly-rebuild-reindex` exists to run after `weekly-rebuild-missing`: it declares `depends_on: weekly-rebuild-missing` and is gated on that workflow's `weekly_rebuild_missing` variable. Triggering it manually therefore also starts hours of package rebuilds.

That is the wrong tool when only the index needs regenerating. After rpkgs/bincraft#113 (v5.1.5), which changes how `union_index_records()` decides what a per-minor index steers to, every object in the bucket is already correct and only `PACKAGES*` is stale. Rebuilding to fix an index is pure waste, and the natural cron would take a full cycle to reach every slot.

## Change

Adds `.crow/reindex.yaml`: the index half on its own, manual only, no dependency on a rebuild.

It reuses the same matrix and the same steps as `weekly-rebuild-reindex` — install the latest bincraft release, republish the generic index, loop the installed R versions republishing each per-minor index, purge the edge. No package is built.

Gated on a new `reindex` variable so it cannot be started by the rebuild gate, defaulting to `none` so a manual pipeline creation (which instantiates every file in `.crow/`) matches no matrix row.

```sh
crow pipeline create devxy/build-cran-binaries --var reindex=all
crow pipeline create devxy/build-cran-binaries --var reindex=ubuntu-2404-amd64
```

## Verification

- `crow lint .crow/` passes.
- Gate is manual-only and evaluates `reindex`, with no `depends_on` and no `runs_on` carried over from the rebuild coupling.

Reviewed-on: #177
2026-08-31 09:55:30 +00:00