ci: add a reindex-only manual workflow (#177)

## Motivation `weekly-rebuild-reindex` exists to run after `weekly-rebuild-missing`: it declares `depends_on: weekly-rebuild-missing` and is gated on that workflow's `weekly_rebuild_missing` variable. Triggering it manually therefore also starts hours of package rebuilds. That is the wrong tool when only the index needs regenerating. After rpkgs/bincraft#113 (v5.1.5), which changes how `union_index_records()` decides what a per-minor index steers to, every object in the bucket is already correct and only `PACKAGES*` is stale. Rebuilding to fix an index is pure waste, and the natural cron would take a full cycle to reach every slot. ## Change Adds `.crow/reindex.yaml`: the index half on its own, manual only, no dependency on a rebuild. It reuses the same matrix and the same steps as `weekly-rebuild-reindex` — install the latest bincraft release, republish the generic index, loop the installed R versions republishing each per-minor index, purge the edge. No package is built. Gated on a new `reindex` variable so it cannot be started by the rebuild gate, defaulting to `none` so a manual pipeline creation (which instantiates every file in `.crow/`) matches no matrix row. ```sh crow pipeline create devxy/build-cran-binaries --var reindex=all crow pipeline create devxy/build-cran-binaries --var reindex=ubuntu-2404-amd64 ``` ## Verification - `crow lint .crow/` passes. - Gate is manual-only and evaluates `reindex`, with no `depends_on` and no `runs_on` carried over from the rebuild coupling. Reviewed-on: #177
2026-08-31 09:55:30 +00:00 · 2026-08-31 09:55:30 +00:00 · f3077677d7
commit f3077677d7
parent aba2063ea0
1 changed files with 188 additions and 0 deletions
--- a/.crow/reindex.yaml
+++ b/.crow/reindex.yaml
@ -0,0 +1,188 @@
+# Re-index every slot without rebuilding anything.
+#
+# `weekly-rebuild-reindex` exists to run after `weekly-rebuild-missing`, so it
+# depends on that workflow and shares its gate: triggering it manually also
+# starts hours of package rebuilds. That is the wrong tool when only the index
+# needs regenerating - after a bincraft release that changes how the index is
+# written, for instance, where the objects in the bucket are already correct
+# and only `PACKAGES*` is stale.
+#
+# This workflow does the index half on its own. It installs the latest bincraft
+# release, republishes the generic and per-R-minor indexes for each slot, and
+# purges the edge. No package is built.
+#
+# Trigger with the `reindex` variable set to `all` or to a single
+# `<os>-<arch>`, e.g.
+#
+#   crow pipeline create devxy/build-cran-binaries --var reindex=all
+
+variables:
+  # A manual pipeline creation instantiates every file in .crow/, so the
+  # default must match no matrix row.
+  reindex:
+    description: "Re-index target: a specific <os>-<arch>, 'all' for every slot, or 'none'."
+    options:
+      - none
+      - all
+      - alpine-322-amd64
+      - alpine-322-arm64
+      - alpine-323-amd64
+      - alpine-323-arm64
+      - alpine-324-amd64
+      - alpine-324-arm64
+      - redhat-8-amd64
+      - redhat-8-arm64
+      - redhat-9-amd64
+      - redhat-9-arm64
+      - redhat-10-amd64
+      - redhat-10-arm64
+      - ubuntu-2204-amd64
+      - ubuntu-2204-arm64
+      - ubuntu-2404-amd64
+      - ubuntu-2404-arm64
+      - ubuntu-2604-amd64
+      - ubuntu-2604-arm64
+    default: none
+
+when:
+  - event: manual
+    evaluate: 'reindex == "all" || reindex == "${OS}-${ARCH}"'
+
+skip_clone: true
+
+labels:
+  group: rpkgs-${ARCH}
+
+matrix:
+  include:
+    - OS: alpine-322
+      ARCH: amd64
+      R_VERSION: 4.5.3
+      IMG: alpine:3.22
+    - OS: alpine-322
+      ARCH: arm64
+      R_VERSION: 4.5.3
+      IMG: alpine:3.22
+    - OS: alpine-323
+      ARCH: amd64
+      R_VERSION: 4.5.3
+      IMG: alpine:3.23
+    - OS: alpine-323
+      ARCH: arm64
+      R_VERSION: 4.5.3
+      IMG: alpine:3.23
+    - OS: alpine-324
+      ARCH: amd64
+      R_VERSION: 4.5.3
+      IMG: alpine:3.24
+    - OS: alpine-324
+      ARCH: arm64
+      R_VERSION: 4.5.3
+      IMG: alpine:3.24
+    - OS: redhat-8
+      ARCH: amd64
+      R_VERSION: 4.4.3
+      IMG: redhat:8
+    - OS: redhat-8
+      ARCH: arm64
+      R_VERSION: 4.4.3
+      IMG: redhat:8
+    - OS: redhat-9
+      ARCH: amd64
+      R_VERSION: 4.4.3
+      IMG: redhat:9
+    - OS: redhat-9
+      ARCH: arm64
+      R_VERSION: 4.4.3
+      IMG: redhat:9
+    - OS: redhat-10
+      ARCH: amd64
+      R_VERSION: 4.5.3
+      IMG: redhat:10
+    - OS: redhat-10
+      ARCH: arm64
+      R_VERSION: 4.5.3
+      IMG: redhat:10
+    - OS: ubuntu-2204
+      ARCH: amd64
+      R_VERSION: 4.4.3
+      IMG: ubuntu:jammy
+    - OS: ubuntu-2204
+      ARCH: arm64
+      R_VERSION: 4.4.3
+      IMG: ubuntu:jammy
+    - OS: ubuntu-2404
+      ARCH: amd64
+      R_VERSION: 4.4.3
+      IMG: ubuntu:noble
+    - OS: ubuntu-2404
+      ARCH: arm64
+      R_VERSION: 4.4.3
+      IMG: ubuntu:noble
+    - OS: ubuntu-2604
+      ARCH: amd64
+      R_VERSION: 4.4.3
+      IMG: ubuntu:resolute
+    - OS: ubuntu-2604
+      ARCH: arm64
+      R_VERSION: 4.4.3
+      IMG: ubuntu:resolute
+
+steps:
+  - name: 'Re-index the slot'
+    image: reg.devxy.io/rpkgs/build-env-${IMG}
+    pull: true
+    environment:
+      OTEL_R_TRACES_EXPORTER: none
+      OTEL_R_LOGS_EXPORTER: none
+      OTEL_R_METRICS_EXPORTER: none
+      RED_HAT_DEV_PW:
+        from_secret: RED_HAT_DEV_PW
+      B2_S3_ACCESS_KEY:
+        from_secret: B2_S3_ACCESS_KEY
+      B2_S3_SECRET_KEY:
+        from_secret: B2_S3_SECRET_KEY
+      REPO_RO_TOKEN:
+        from_secret: REPO_RO_TOKEN
+      GIT_USER: pat-s
+      R_LIBS_USER: /mnt/cache/R-pkgs
+      R_VERSION: ${R_VERSION}
+      PLATFORM: ${OS}
+      ARCH: ${ARCH}
+    commands:
+      - git clone -q https://pat-s:$$REPO_RO_TOKEN@git.devxy.io/devxy/build-cran-binaries.git .
+      - mkdir -p /mnt/cache/R-pkgs
+      - rm -rf /mnt/cache/R-pkgs/00LOCK-*
+      - /opt/R/$R_VERSION/bin/Rscript local/install-bincraft.R
+      # The codename is detected from the image's /etc/os-release.
+      - /opt/R/$R_VERSION/bin/R -q -e 'library(bincraft); upload_package_index(s3_endpoint = "https://s3.eu-central-003.backblazeb2.com", s3_region = "eu-central-003", s3_bucket = "devxy-rpkgs-binaries", s3_access_key_id = Sys.getenv("B2_S3_ACCESS_KEY"), s3_secret_access_key = Sys.getenv("B2_S3_SECRET_KEY"))'
+      - |
+        for RBIN in /opt/R/[0-9]*/bin/R; do
+          RMINOR=$(basename "$(dirname "$(dirname "$RBIN")")" | cut -d. -f1-2)
+          /opt/R/$R_VERSION/bin/R -q -e "library(bincraft); upload_package_index(r_minor = '$RMINOR', s3_endpoint = 'https://s3.eu-central-003.backblazeb2.com', s3_region = 'eu-central-003', s3_bucket = 'devxy-rpkgs-binaries', s3_access_key_id = Sys.getenv('B2_S3_ACCESS_KEY'), s3_secret_access_key = Sys.getenv('B2_S3_SECRET_KEY'))" || true
+        done
+
+  - name: Purge CDN cache
+    image: reg.devxy.io/docker.io/library/alpine:3.24
+    environment:
+      OTEL_R_TRACES_EXPORTER: none
+      OTEL_R_LOGS_EXPORTER: none
+      OTEL_R_METRICS_EXPORTER: none
+      BUNNYNET_API_KEY:
+        from_secret: BUNNYNET_API_KEY
+      # cran.rpkgs.com and cran.allianceswisspass.devxy.io are on separate
+      # Bunny pull zones, so both must be purged after the shared origin changes.
+      BUNNY_PULLZONES: '3857050 cran.allianceswisspass.devxy.io'
+    commands:
+      - apk add --no-cache -q bash curl jq
+      # Crow carries the checkout from the re-index step into this step.
+      - bash scripts/purge_cdn_zone.sh "$BUNNYNET_API_KEY" $BUNNY_PULLZONES
+    # Runs on every row rather than on one designated slot: a cron fires only
+    # its own slot's row, so gating on a named slot would leave every other
+    # slot unpurged. A manual "all" run therefore purges the zone 18 times,
+    # which is a cheap API call and rare.
+    #
+    # Run it even when the re-index above failed: the objects were still
+    # replaced, and a stale edge is exactly what keeps them hidden.
+    when:
+      - status: [success, failure]