| Filename | Latest commit message | Latest commit date |
|---|---|---|
pat-s
642e07e1d6
Some checks are pending
ci/crow/manual/build-all-versions/5 Pipeline is running
ci/crow/manual/build-all-versions/6 Pipeline is running
ci/crow/manual/build-all-versions/8 Pipeline is running
ci/crow/manual/build-all-versions/7 Pipeline is running
ci/crow/manual/build-all-versions-install-deps/2 Pipeline was successful
ci/crow/manual/build-all-versions-install-deps/1 Pipeline was successful
ci/crow/manual/build-all-versions/1 Pipeline is running
ci/crow/manual/build-all-versions/2 Pipeline is running
ci/crow/cron/process-updates/8 Pipeline is pending
ci/crow/manual/build-all-versions/3 Pipeline is running
ci/crow/cron/process-updates/9 Pipeline is pending
ci/crow/manual/build-all-versions/4 Pipeline is running
ci/crow/cron/process-updates/3 Pipeline is pending
## Motivation
`arm64/alpine324` reported nothing to build while thousands were missing:
```
line 49: Precomputed 7192 package versions (6871 r-minor-sensitive) <- install-deps agent
line 99: Total# of remaining package versions: 43 (sensitive_only=TRUE) <- a build shard
line 101: Skipped 0 package versions already attempted under R 4.4; 0 remaining
```
Both numbers come from the **same pipeline**. The same run's index step dropped 2407 packages as missing for 4.4 and 2436 for 4.6.
## Cause
```r
if (!all(file.exists(package_cache_files))) { ... recompute ... }
```
Existence is not freshness. The snapshot describes S3 and CRAN state when it was written, and the cache volume is per-agent — the file's own comment says so. An agent that ran an earlier pipeline keeps serving that pipeline's answer forever, and no later fix to how the snapshot is computed (#189) can reach it.
## Change
Recompute when the snapshot is stale as well as when it is missing. Keyed on the pipeline when the CI exposes an identifier (`CI_PIPELINE_NUMBER`, `CI_BUILD_NUMBER`, `CI_PIPELINE_ID`), so a new pipeline recomputes once per agent and its shards then share the result. Off CI, or when none is set, an age check with a two hour default (`PACKAGE_SNAPSHOT_TTL_HOURS`).
## Verification
| scenario | decision |
|---|---|
| files missing | RECOMPUTE |
| same pipeline id | reuse |
| **new pipeline id** | **RECOMPUTE** |
| no CI var, recent file | reuse |
| no CI var, aged out | RECOMPUTE |
I could not confirm which identifier Crow actually sets — none is referenced anywhere in this repo — so all three are tried and the age check backs them up. If none is present the behaviour is the age path, which is still correct, just coarser.
Reviewed-on: #190
|
||
| .. | ||
| patches | ||
| tests | ||
| archive-missed-pkgs.R | ||
| build-all.R | ||
| build-one.R | ||
| check-cdn-cache.sh | ||
| check-NA.R | ||
| debug-packages-writing.R | ||
| dedupe-audit-issue.R | ||
| detect-duplicates.R | ||
| excluded-packages.json | ||
| failing-builds-classify.R | ||
| failing-builds-report.R | ||
| fetch-rebuild-packages-from-issue.R | ||
| find-R-api-packages.sh | ||
| install-bincraft.R | ||
| label-removed-cran-packages.R | ||
| last-processed-by-platform.R | ||
| migrate-s3-hetzner-to-backblaze.sh | ||
| missing-cran-packages-db.R | ||
| missing-cran-packages.R | ||
| missing-packages-in-index.R | ||
| packages-to-build.R | ||
| packages-without-any-binary.R | ||
| proposal-tracking-lib.R | ||
| proposal-tracking.R | ||
| propose-patches.R | ||
| query-pkgs-without-old-versions.R | ||
| r-minor-helpers.R | ||
| rebuild-missing-helpers.R | ||
| rebuild-missing.R | ||
| repair-built-stamp.R | ||
| test-package-loading.R | ||
| trial-build-patch.R | ||
| trial-build-registry.R | ||
| uvr-install.sh | ||
| validate-patches.R | ||
| weekly-missing-binaries-audit.R | ||