build-cran-binaries/.crow/repair-built-stamp.yaml
pat-s c61d951875
Some checks failed
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline failed
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
fix(repair-built-stamp): patch the published PACKAGES.rds once bincraft drops cranlike (#236)
Follow-up to https://codefloe.com/rpkgs/bincraft/pulls/228 (bincraft#33), which drops the cranlike and desc forks.

## Why

After that change, `upload_package_index()` no longer keeps a `PACKAGES.db`.
It reuses the records of the slot's published `PACKAGES.rds`, as long as each tarball is still listed with the same ETag.
`local/repair-built-stamp.R` patched `PACKAGES.db`, so it would stop having any effect.

## Changes

- `local/repair-built-stamp.R` now patches `Built` in each slot's `PACKAGES.rds` and then re-emits the index.
  - Only the slot's own rows are patched: the generic slot's rows without a `Path`, or a per-minor slot's rows whose `Path` is its minor.
  - A missing stamp is not counted as broken, since bincraft clears it on purpose for source fallbacks.
  - With a bincraft release that still loads cranlike, the script patches `PACKAGES.db` as before, so merge order does not matter.
  - `bincraft::built_stamp()` was never exported, so the script failed at that line before this change; it now calls `bincraft:::built_stamp()`.
- The `GITHUB_PAT` comments in `reindex`, `weekly-rebuild-reindex` and `backfill-flat-safety` now give the current reason: bincraft reads DESCRIPTIONs from raw.githubusercontent.com.
- The README section on index files describes `upload_package_index()` instead of the forks.

## Testing

Tested against a moto S3 bucket, with bincraft installed from the bincraft#228 branch:

- A slot was indexed, then 3 of its 5 `Built` stamps were set to `R 4.5.0; NA; ...`.
- The dry run reported 5 entries, 3 broken.
- `--apply` rewrote the 3 stamps, and the re-emit logged `Index records: 5 reused, 0 read, 0 unreadable.`
- The published `PACKAGES` then had a valid stamp on all 5 records.
- With the released bincraft, which still loads cranlike, the script chose `PACKAGES.db`.

The stale `PACKAGES.db` objects in the slots can be deleted once a bincraft release with #228 is in use.

Reviewed-on: #236
2026-10-18 11:55:21 +00:00

109 lines
3.4 KiB
YAML

### Manual repair of a slot whose PACKAGES index advertises a broken `Built`
### stamp (e.g. `Built: R 4.5.0; NA; ...`).
#
# uvr matches the stamp's platform triple plus R minor to decide binary vs
# source, so an unusable triple turns a whole slot source-only. See
# local/repair-built-stamp.R for why this patches the published PACKAGES.rds in
# place instead of forcing a full reparse.
#
# Run with `dry_run: true` first: it reports how many entries are broken per
# slot and changes nothing. Pick the R version the slot should advertise, which
# is the R_VERSION its entry in .crow/process-updates.yaml uses.
#
# The gate variable is `repair_built_stamp`, not `target_arch`: `target_arch` is
# what build-all-versions and build-all-versions-install-deps gate on, so a
# manual run passing it would start a full rebuild alongside this repair. Every
# pipeline here gates on a variable named after itself for exactly that reason.
#
# crow pipeline create --branch main \
# --var repair_built_stamp=arm64 --var OS=alpine --var OS_VERSION=3.23 \
# --var R_VERSION=4.5.3 --var dry_run=true devxy/build-cran-binaries
variables:
# Gates this pipeline. A manual pipeline creation instantiates every file in
# .crow/, and a declared default is applied even when the run never passed
# this variable, so the default must be a value that matches no matrix row.
repair_built_stamp:
description: 'Architecture of the slot to repair, or "none" to run nothing.'
options:
- none
- amd64
- arm64
default: none
OS:
description: 'Base OS image name.'
options:
- alpine
- redhat
- ubuntu
default: alpine
OS_VERSION:
description: 'OS image tag. Must match OS (alpine: 3.23/3.24; redhat: 8/9/10; ubuntu: jammy/noble/resolute).'
options:
- '3.23'
- '3.24'
- '8'
- '9'
- '10'
- 'jammy'
- 'noble'
- 'resolute'
default: '3.23'
R_VERSION:
description: 'R version whose stamp the slot should advertise.'
options:
- 4.5.3
- 4.4.3
default: 4.5.3
dry_run:
description: 'Report what would change without writing anything.'
options:
- 'true'
- 'false'
default: 'true'
when:
- event: manual
evaluate: 'repair_built_stamp == "${ARCH}"'
skip_clone: true
labels:
platform: linux/${ARCH}
group: rpkgs-${ARCH}
matrix:
include:
- ARCH: amd64
- ARCH: arm64
steps:
- name: 'Repair Built stamp'
image: 'reg.devxy.io/rpkgs/build-env-${OS}:${OS_VERSION}'
pull: true
environment:
B2_S3_ACCESS_KEY:
from_secret: B2_S3_ACCESS_KEY
B2_S3_SECRET_KEY:
from_secret: B2_S3_SECRET_KEY
REPO_RO_TOKEN:
from_secret: REPO_RO_TOKEN
GIT_USER: pat-s
commands:
- git clone -q https://pat-s:$$REPO_RO_TOKEN@git.devxy.io/devxy/build-cran-binaries.git .
- /opt/R/$R_VERSION/bin/Rscript local/install-bincraft.R
- /opt/R/$R_VERSION/bin/R -q -e 'packageVersion("bincraft")'
- |
if [ "$dry_run" = "false" ]; then
/opt/R/$R_VERSION/bin/Rscript local/repair-built-stamp.R "$ARCH" --apply
else
/opt/R/$R_VERSION/bin/Rscript local/repair-built-stamp.R "$ARCH"
fi
backend_options:
docker:
resources:
requests:
memory: 2Gi
cpu: 1000m
limits:
memory: 8Gi
cpu: 2000m