| Filename | Latest commit message | Latest commit date |
|---|---|---|
pat-s
94e6c697cf
Some checks failed
ci/crow/manual/build-all-versions-install-deps/2 Pipeline was successful
ci/crow/manual/build-all-versions-install-deps/1 Pipeline was successful
ci/crow/manual/build-all-versions/8 Pipeline was canceled
ci/crow/manual/build-all-versions/7 Pipeline was canceled
ci/crow/manual/build-all-versions/6 Pipeline was canceled
ci/crow/manual/build-all-versions/5 Pipeline was canceled
ci/crow/manual/build-all-versions/1 Pipeline was canceled
ci/crow/manual/build-all-versions/3 Pipeline was canceled
ci/crow/manual/build-all-versions/4 Pipeline was canceled
ci/crow/manual/build-all-versions/2 Pipeline was canceled
ci/crow/cron/process-updates/7 Pipeline was successful
## Motivation
`arm64/alpine324` (pipeline 11953) finished in minutes having uploaded 57 packages, and reported:
```
Skipped 0 package versions already attempted under R 4.4; 0 remaining
Skipped 0 package versions already attempted under R 4.6; 3 remaining
```
The same run's index step dropped **2407** packages as missing for 4.4 and **2436** for 4.6. Nothing to build, and thousands missing — the candidate list is wrong.
## Two omissions
**1. Per-minor objects mask the per-minor candidates.**
```r
s3_pkgs <- s3fs::s3_dir_ls(".../latest/src/contrib", recurse = TRUE)
file_names <- basename(s3_pkgs)
```
`recurse = TRUE` walks `4.4/`, `4.5/`, `4.6/`; `basename()` throws the directory away. `4.5/curl_1.0.tar.gz` and `curl_1.0.tar.gz` collapse to one name, so a package present under **any** R minor counts as built for **all** of them — pruning exactly the packages a per-minor pass exists to build.
Per-minor objects are now excluded, and presence in a specific minor is decided downstream where the running R version is known: `build-all.R` filters on it, and `build_binary_package()` checks the per-minor path per package and skips what is already there.
`Archive/` is kept. Those are versions built and later superseded; dropping them would make every archived version look unbuilt.
Validated against real path shapes:
| path | |
|---|---|
| `curl_1.0.tar.gz` | keep |
| `4.4/curl_1.0.tar.gz` | exclude |
| `4.6/rlang_1.3.0.tar.gz` | exclude |
| `Archive/curl/curl_0.9.tar.gz` | keep |
| `PACKAGES.gz` | keep |
**2. The error query ignores `r_version`.**
```sql
SELECT error_occurred FROM single_builds
WHERE name = $1 AND tag = $2 AND platform = $3 AND arch = $4
```
A failure under the primary minor drops the package from every other minor's candidate list. This is the same omission fixed in `local/build-all.R` (#187) and in bincraft's `check_package_error()` (rpkgs/bincraft#119). This is the third and last consumer of that table — I have grepped the rest; `bincraft::R/cran-internal.R` also reads it, but to list packages present rather than to skip, where the R minor does not apply.
## Expected effect
The per-minor passes get real candidate lists. Expect slots that reported "0 remaining" to report thousands, and correspondingly long runs.
There is a cost: the list is no longer pruned by per-minor presence, so each pass asks `build_binary_package()` about packages that may already exist, and it answers `already exists in S3 ... Skipping build` per package. Slower per pass, and correct — the pruning it replaces was removing the wrong things.
Reviewed-on: #189
|
||
| .. | ||
| patches | ||
| tests | ||
| archive-missed-pkgs.R | ||
| build-all.R | ||
| build-one.R | ||
| check-cdn-cache.sh | ||
| check-NA.R | ||
| debug-packages-writing.R | ||
| dedupe-audit-issue.R | ||
| detect-duplicates.R | ||
| excluded-packages.json | ||
| failing-builds-classify.R | ||
| failing-builds-report.R | ||
| fetch-rebuild-packages-from-issue.R | ||
| find-R-api-packages.sh | ||
| install-bincraft.R | ||
| label-removed-cran-packages.R | ||
| last-processed-by-platform.R | ||
| migrate-s3-hetzner-to-backblaze.sh | ||
| missing-cran-packages-db.R | ||
| missing-cran-packages.R | ||
| missing-packages-in-index.R | ||
| packages-to-build.R | ||
| packages-without-any-binary.R | ||
| proposal-tracking-lib.R | ||
| proposal-tracking.R | ||
| propose-patches.R | ||
| query-pkgs-without-old-versions.R | ||
| r-minor-helpers.R | ||
| rebuild-missing-helpers.R | ||
| rebuild-missing.R | ||
| repair-built-stamp.R | ||
| test-package-loading.R | ||
| trial-build-patch.R | ||
| trial-build-registry.R | ||
| uvr-install.sh | ||
| validate-patches.R | ||
| weekly-missing-binaries-audit.R | ||