fix(repair-built-stamp): patch the published PACKAGES.rds once bincraft drops cranlike (#236)
Some checks failed
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline failed
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
Some checks failed
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline failed
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
Follow-up to https://codefloe.com/rpkgs/bincraft/pulls/228 (bincraft#33), which drops the cranlike and desc forks. ## Why After that change, `upload_package_index()` no longer keeps a `PACKAGES.db`. It reuses the records of the slot's published `PACKAGES.rds`, as long as each tarball is still listed with the same ETag. `local/repair-built-stamp.R` patched `PACKAGES.db`, so it would stop having any effect. ## Changes - `local/repair-built-stamp.R` now patches `Built` in each slot's `PACKAGES.rds` and then re-emits the index. - Only the slot's own rows are patched: the generic slot's rows without a `Path`, or a per-minor slot's rows whose `Path` is its minor. - A missing stamp is not counted as broken, since bincraft clears it on purpose for source fallbacks. - With a bincraft release that still loads cranlike, the script patches `PACKAGES.db` as before, so merge order does not matter. - `bincraft::built_stamp()` was never exported, so the script failed at that line before this change; it now calls `bincraft:::built_stamp()`. - The `GITHUB_PAT` comments in `reindex`, `weekly-rebuild-reindex` and `backfill-flat-safety` now give the current reason: bincraft reads DESCRIPTIONs from raw.githubusercontent.com. - The README section on index files describes `upload_package_index()` instead of the forks. ## Testing Tested against a moto S3 bucket, with bincraft installed from the bincraft#228 branch: - A slot was indexed, then 3 of its 5 `Built` stamps were set to `R 4.5.0; NA; ...`. - The dry run reported 5 entries, 3 broken. - `--apply` rewrote the 3 stamps, and the re-emit logged `Index records: 5 reused, 0 read, 0 unreadable.` - The published `PACKAGES` then had a valid stamp on all 5 records. - With the released bincraft, which still loads cranlike, the script chose `PACKAGES.db`. The stale `PACKAGES.db` objects in the slots can be deleted once a bincraft release with #228 is in use. Reviewed-on: #236
This commit is contained in:
parent
8f659ef4f7
commit
c61d951875
1 changed files with 117 additions and 45 deletions
|
|
@ -103,11 +103,13 @@ Preview the selection without building or accessing storage or build metadata:
|
|||
|
||||
### Creating/Updating the PACKAGES Index Files
|
||||
|
||||
Currently, the {cranlike} and {desc} packages only work with files on a local file system.
|
||||
Tools such as `tools::write_PACKAGES()` and {cranlike} only index files on a local file system.
|
||||
This is infeasible if the goal is to store binaries in S3.
|
||||
Storing binaries permanently on a disk-based file system would incur significantly higher costs, especially when operating in the cloud.
|
||||
|
||||
Hence, [{cranlike}](https://devxy.gitlab.io/r-package-binaries/bincraft/) and [{desc}](https://github.com/pat-s/desc/tree/description-from-remote) were forked to handle files in S3 (through {s3fs}).
|
||||
Hence, `bincraft::upload_package_index()` builds the index from a listing of the slot in S3 (through {s3fs}).
|
||||
Each record is read from the package's CRAN `DESCRIPTION` on the GitHub mirror of CRAN, falling back to the `DESCRIPTION` inside the tarball for a package the mirror does not have.
|
||||
The previously published `PACKAGES.rds` serves as a cache, so only new or replaced tarballs are read.
|
||||
|
||||
### Resources
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue