build-cran-binaries/.crow/archive-missed-packages.yaml
pat-s 72de7ae6d1
fix(index): add a slot repair for a broken Built stamp and retire alpine 3.21
`arm64/alpine321` (22,930 entries) and `arm64/alpine322` (24,696 entries)
advertise `Built: R <ver>; NA; ...`. uvr matches the stamp's platform triple
plus R minor to pick binary over source, so no client matches those slots and
both silently serve as source-only, which is the regression the stamp was added
to prevent. The tarballs themselves carry the correct triple; only the index is
wrong.

Re-running the normal index update does not fix it: `upload_package_index()`
reuses the slot's remote `PACKAGES.db` and cranlike's `update_db()` only
reparses files whose md5 changed. Dropping `PACKAGES.db` to force a full reparse
does work, but for an S3 repo cranlike reads each package's metadata from the
CRAN source mirror on GitHub, so a 25k-entry slot means 25k requests to
raw.githubusercontent.com and a real risk of being rate-limited part-way.

Only the `Built` column is wrong, so correct it in place: patch the column in
`PACKAGES.db`, put it back, and let `upload_package_index()` re-emit `PACKAGES*`
from it. `update_db()` always rewrites the index files even when nothing was
reparsed, so no tarball is re-read and nothing is fetched from GitHub.

Alpine 3.21 is EOL: the website advertises only 3.23/3.24 and
`process-updates.yaml` dropped it, so remove its two leftover matrix entries
from the archive pipeline rather than repairing that slot.
2026-08-07 09:05:52 +00:00

77 lines
2.2 KiB
YAML

### Required extra vars
# woodpecker-cli pipeline create --var task=archive-missed-packages --branch=main --log-level=info 7
when:
- event: manual
evaluate: 'task == "archive-missed-packages"'
- event: cron
cron: archive-missed-packages
matrix:
include:
- CODENAME: jammy
ARCH: amd64
- CODENAME: jammy
ARCH: arm64
- CODENAME: noble
ARCH: amd64
- CODENAME: noble
ARCH: arm64
- CODENAME: redhat-8
ARCH: amd64
- CODENAME: redhat-8
ARCH: arm64
- CODENAME: redhat-9
ARCH: amd64
- CODENAME: redhat-9
ARCH: arm64
- CODENAME: redhat-10
ARCH: amd64
- CODENAME: redhat-10
ARCH: arm64
- CODENAME: alpine322
ARCH: amd64
- CODENAME: alpine322
ARCH: arm64
- CODENAME: alpine323
ARCH: amd64
- CODENAME: alpine323
ARCH: arm64
steps:
- name: 'Archive missed packages'
image: reg.devxy.io/rpkgs/build-env-alpine:3.24
pull: true
environment:
RED_HAT_DEV_PW:
from_secret: RED_HAT_DEV_PW
B2_S3_ACCESS_KEY:
from_secret: B2_S3_ACCESS_KEY
B2_S3_SECRET_KEY:
from_secret: B2_S3_SECRET_KEY
PGPASS:
from_secret: PGPASS
REPO_RO_TOKEN:
from_secret: REPO_RO_TOKEN
GITHUB_PAT:
from_secret: GITHUB_PAT
# normal env vars
GIT_USER: pat-s
R_VERSION: 4.5.3
commands:
- /opt/R/$R_VERSION/bin/Rscript local/install-bincraft.R
- /opt/R/$R_VERSION/bin/R -q -e 'packageVersion("bincraft")'
- /opt/R/$R_VERSION/bin/R -q -e 'bincraft::process_unarchived_pkgs(Sys.getenv("CODENAME"), Sys.getenv("ARCH"), s3_endpoint = "https://s3.eu-central-003.backblazeb2.com", s3_region = "eu-central-003", s3_bucket = "devxy-rpkgs-binaries", s3_access_key_id = Sys.getenv("B2_S3_ACCESS_KEY"), s3_secret_access_key = Sys.getenv("B2_S3_SECRET_KEY"), workers = 2L)'
backend_options:
kubernetes:
resources:
requests:
memory: 1Gi
cpu: 2000m
limits:
memory: 1200Mi
cpu: 2000m
tolerations:
- key: 'CI'
operator: 'Equal'
value: 'true'
effect: 'NoSchedule'