| Filename | Latest commit message | Latest commit date |
|---|---|---|
pat-s
d4f093923a
Some checks failed
ci/crow/manual/build-all-versions-install-deps/1 Pipeline was successful
ci/crow/manual/build-all-versions-install-deps/2 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/manual/build-all-versions/7 Pipeline was canceled
ci/crow/manual/build-all-versions/6 Pipeline was canceled
ci/crow/manual/build-all-versions/8 Pipeline was canceled
ci/crow/manual/build-all-versions/1 Pipeline was canceled
ci/crow/manual/build-all-versions/5 Pipeline was canceled
ci/crow/manual/build-all-versions/3 Pipeline was canceled
ci/crow/manual/build-all-versions/2 Pipeline was canceled
ci/crow/manual/build-all-versions/4 Pipeline was canceled
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
## Why #191 gave the existence cache a per-minor view of the slot, but the paths it filters never carried a minor prefix, so the fix could not take effect. `s3fs::s3_dir_ls()` returns keys with the `s3://` scheme attached: ``` s3://devxy-rpkgs-binaries/amd64/resolute/latest/src/contrib/4.4/oeli_0.7.6.tar.gz ``` The contrib prefix was stripped as a *fixed substring*, so it was removed from the middle of the key and the scheme survived: ``` s3://4.4/oeli_0.7.6.tar.gz ``` That leading `s3://` defeats both `^<minor>/` and `^[0-9]+\.[0-9]+/`, so all 21212 per-minor objects on amd64/resolute were classified as flat-slot objects. Pipeline 12011 shows it exactly: ``` sensitive pass: 0 files (0 of 119053 objects apply to this pass) primary pass: 119053 files (119053 of 119053 objects apply to this pass) ``` Each sensitive pass therefore ran with an empty cache and recompiled all 10248 sensitive packages it already had. ## What changed - `local/packages-to-build.R`: anchor the contrib prefix and swallow an optional `s3://` with it. - `local/packages-to-build.R`: abort when the strip leaves fewer per-minor paths than the listing held. The failure mode is silent and only surfaces as a multi-hour rebuild, and both counts derive from the same listing so they must agree exactly. - `local/packages-to-build.R` / `local/build-all.R`: mark the cache with a `slot_relative` attribute and read that, instead of sniffing for a `/`. A slot-relative cache for a slot with no per-minor or Archive object holds bare names too, and would have been misread as legacy and used unfiltered, which makes a per-minor pass believe the flat slot's binaries are its own and build nothing. - `scripts/purge_cdn_zone.sh`: indent the jq continuation lines by a multiple of two. This is unrelated, but it fails editorconfig-checker on `main` and blocks `prek run -a` for everyone. jq ignores the whitespace, and both response shapes still resolve. ## Verification Replaying the real key shapes through the old and new code: ``` sensitive(4.6) primary OLD paths: 0 8 <- reproduces production NEW paths: 3 3 Invariant NEW: listing=4 stripped=4 -> PASS Invariant OLD: listing=4 stripped=0 -> ABORT (would have caught this) ``` Per-minor `Archive/` objects are attributed to their minor, and the flat bucket keeps its own `Archive/`. `prek run -a` passes. Pipelines 12011-12015 were stopped rather than left to spend hours recompiling what they already had. Their uploads are not lost, so a fresh run inherits them. Reviewed-on: #192 |
||
| .. | ||
| patches | ||
| tests | ||
| archive-missed-pkgs.R | ||
| build-all.R | ||
| build-one.R | ||
| check-cdn-cache.sh | ||
| check-NA.R | ||
| debug-packages-writing.R | ||
| dedupe-audit-issue.R | ||
| detect-duplicates.R | ||
| excluded-packages.json | ||
| failing-builds-classify.R | ||
| failing-builds-report.R | ||
| fetch-rebuild-packages-from-issue.R | ||
| find-R-api-packages.sh | ||
| install-bincraft.R | ||
| label-removed-cran-packages.R | ||
| last-processed-by-platform.R | ||
| migrate-s3-hetzner-to-backblaze.sh | ||
| missing-cran-packages-db.R | ||
| missing-cran-packages.R | ||
| missing-packages-in-index.R | ||
| packages-to-build.R | ||
| packages-without-any-binary.R | ||
| proposal-tracking-lib.R | ||
| proposal-tracking.R | ||
| propose-patches.R | ||
| query-pkgs-without-old-versions.R | ||
| r-minor-helpers.R | ||
| rebuild-missing-helpers.R | ||
| rebuild-missing.R | ||
| repair-built-stamp.R | ||
| test-package-loading.R | ||
| trial-build-patch.R | ||
| trial-build-registry.R | ||
| uvr-install.sh | ||
| validate-patches.R | ||
| weekly-missing-binaries-audit.R | ||