Commit graph build-cran-binaries/.crow/reindex.yaml
Author SHA1 Message Date
1cf473acf8
fix(ci): authenticate uvr's GitHub resolution in the index workflows
uvr resolves bincraft's dependencies through the GitHub API, among them the
pat-s/cranlike@s3 fork. Unauthenticated that is rate limited, and the failure is
quiet:

  WARN Failed to fetch git Remote pat-s/cranlike@s3 (GitHub API error: 403 Forbidden);
       falling back to registry resolution
  Error: Failed to update PACKAGES after 3 attempts.
         Last error: unused argument (built = built)

uvr warns, falls back to the registry copy of cranlike, and the index write then
dies because upstream cranlike has no `built` parameter. Five slots failed to
re-index that way -- alpine323, rhel8, rhel9, rhel10 and noble -- while the rest
of the run reported success.

It is intermittent by nature, since it depends on the rate-limit window, so the
same pipeline can pass and fail on identical input.

build-all-versions.yaml and rebuild-abi-gap.yaml already pass GITHUB_PAT, which
is why they resolve the fork reliably and these did not.
2026-09-05 13:59:46 +00:00
e1114bc145 fix(ci): give the index writers the metadata database password (#201)
All checks were successful
ci/crow/manual/reindex/11 Pipeline was successful
ci/crow/manual/reindex/9 Pipeline was successful
ci/crow/manual/reindex/15 Pipeline was successful
ci/crow/manual/reindex/13 Pipeline was successful
ci/crow/manual/reindex/17 Pipeline was successful
ci/crow/manual/reindex/1 Pipeline was successful
ci/crow/manual/reindex/3 Pipeline was successful
ci/crow/manual/reindex/5 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/manual/reindex/2 Pipeline was successful
ci/crow/manual/reindex/4 Pipeline was successful
ci/crow/manual/reindex/8 Pipeline was successful
ci/crow/manual/reindex/6 Pipeline was successful
ci/crow/manual/reindex/10 Pipeline was successful
ci/crow/manual/reindex/12 Pipeline was successful
ci/crow/manual/reindex/14 Pipeline was successful
ci/crow/manual/reindex/16 Pipeline was successful
ci/crow/manual/reindex/18 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
## Why

Pipeline 12207 logs one of these per slot:

```
Error: connection to server at "r-binaries.devxy.io" (116.203.165.191), port 15432 failed:
  fe_sendauth: no password supplied
```

bincraft's index build calls `abi_cache_risky_packages()` to decide which packages may be carried from the flat slot into a per-minor index. `reindex.yaml` and `weekly-rebuild-reindex.yaml` never passed `PGPASS`, so the query cannot connect.

It fails soft by design, which is what makes this worth fixing rather than tolerating: the run still reports **success** while using an empty cache-derived risky set. The index is then written as though no package were known risky from the classifier, which is the exact state the cache was added to fix.

`build-all-versions.yaml` already passes `PGPASS`, which is why classification behaved correctly there and silently degraded here.

## What changed

Adds the `PGPASS` secret to both index-writing workflows, with a comment explaining why its absence is invisible rather than loud.

`crow lint` passes on both files.

## Note

The tarball-derived half of the risky set still worked, so indexes written without this are not wrong so much as incomplete: they miss packages that are known risky from classification but have no per-minor tarball yet. A re-index after this merges applies the full set.

Reviewed-on: #201
2026-09-04 19:26:55 +00:00
a2923bf063 fix(cdn): purge the staging zone too (#181)
All checks were successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
## Motivation

`cran-rpkgs-test` was added as a second pull zone on the same B2 origin, but it was never added to `BUNNY_PULLZONES`. It therefore keeps serving pre-reindex indexes behind the same ~370 day `cache_expiration_time` as production, and nothing ever refreshes it.

That is not cosmetic. The zone exists to be measured, and a verification run against it measures whatever the edge still holds:

```
production: regressions=0    AGHmatrix Path=NA   Built=R 4.5.3; x86_64-pc
staging   : regressions=161  AGHmatrix Path=4.5  Built=(none)
```

Same objects, same origin, 161 phantom regressions. I chased that number through two wrong diagnoses before noticing production and staging disagreed.

## Change

Add `cran-rpkgs-test.b-cdn.net` to the purge list in both reindex pipelines.

## Note

A `Cache-Control: no-cache` request header is not a substitute. It was added to the verification script and did **not** clear this: bunny does not honour it for these objects. Purging is the mechanism that works.

Reviewed-on: #181
2026-08-31 10:31:35 +00:00
f3077677d7 ci: add a reindex-only manual workflow (#177)
## Motivation

`weekly-rebuild-reindex` exists to run after `weekly-rebuild-missing`: it declares `depends_on: weekly-rebuild-missing` and is gated on that workflow's `weekly_rebuild_missing` variable. Triggering it manually therefore also starts hours of package rebuilds.

That is the wrong tool when only the index needs regenerating. After rpkgs/bincraft#113 (v5.1.5), which changes how `union_index_records()` decides what a per-minor index steers to, every object in the bucket is already correct and only `PACKAGES*` is stale. Rebuilding to fix an index is pure waste, and the natural cron would take a full cycle to reach every slot.

## Change

Adds `.crow/reindex.yaml`: the index half on its own, manual only, no dependency on a rebuild.

It reuses the same matrix and the same steps as `weekly-rebuild-reindex` — install the latest bincraft release, republish the generic index, loop the installed R versions republishing each per-minor index, purge the edge. No package is built.

Gated on a new `reindex` variable so it cannot be started by the rebuild gate, defaulting to `none` so a manual pipeline creation (which instantiates every file in `.crow/`) matches no matrix row.

```sh
crow pipeline create devxy/build-cran-binaries --var reindex=all
crow pipeline create devxy/build-cran-binaries --var reindex=ubuntu-2404-amd64
```

## Verification

- `crow lint .crow/` passes.
- Gate is manual-only and evaluates `reindex`, with no `depends_on` and no `runs_on` carried over from the rebuild coupling.

Reviewed-on: #177
2026-08-31 09:55:30 +00:00