refactor(slots): cut the comments down to the essentials

Addresses review on #214.

This change will:

- Trim the header comments of `local/check-slots.R`, `local/retire-slot.R` and
  `.crow/retire-slot.yaml` to what a reader needs, dropping the narrative about
  how Alpine 3.22 got into that state.
- Reduce the README section to the ordered steps and the two constraints that
  matter.

Behaviour is unchanged: against the tree at 523424b the script still reproduces
#213's structural changes, differing only in the same prose-only file, and both
guards still fire.

.crow/retire-slot.yaml

@@ -1,27 +1,16 @@
 # Delete a retired slot's objects from the bucket.
 #
-# The config half of a retirement (`.crow/*.yaml` matrices) is a commit, so it
+# The config half is a commit, so it runs on a workstation:
-# happens on a workstation:
+#   just retire-slot <codename> --apply
 #
-#   Rscript local/retire-slot.R <codename> --config-only --apply
+# This half needs the B2 credentials, which only exist in CI.
+# `local/retire-slot.R` refuses any slot still listed in `local.rpkgs_slots`,
+# so this cannot delete a live slot. Both architectures go together.
 #
-# The asset half needs the B2 credentials, which only exist in CI, so it
+# Dry run unless RETIRE_APPLY is 'yes':
-# happens here. `local/retire-slot.R` refuses any slot still listed in
-# `local.rpkgs_slots` in cdn.tf, so this pipeline cannot delete a live slot
-# however it is invoked.
-#
-# Dry run unless RETIRE_APPLY is 'yes'. The dry run reports the object count,
-# which is the number worth looking at before deleting a five-figure prefix.
 #
 #   crow pipeline create devxy/build-cran-binaries --branch main \
 #     --var retire_slot=alpine322 --var RETIRE_APPLY=yes
-#
-# Retiring a distro retires both architectures: `local.rpkgs_slots` is a
-# setproduct over both, so there is no way to publish one without the other,
-# and the script deletes `amd64/<codename>/` and `arm64/<codename>/` together.
-#
-# The slot's cron jobs are not in this repository and are not removed here. The
-# script prints them by name, with the API call, at the end of the run.
 
 variables:
   # A manual pipeline creation instantiates every file in .crow/, so the

README.md

@@ -200,39 +200,25 @@ Technically, no date-based snapshots are planned, so this component from the Pos
 
 ## Retiring a slot
 
-A slot is only reachable by clients if it appears in `local.rpkgs_slots` in `cdn.tf`.
+A slot is only reachable if it is listed in `local.rpkgs_slots` in `cdn.tf`, which feeds `UNION_SLOTS` in `edge/rpkgs-router.ts`.
-That list feeds the `UNION_SLOTS` variable and `edge/rpkgs-router.ts` gates all per-R-minor index rewriting on it.
+An unrouted slot serves the flat index to every client, so its binaries fail at `dyn.load()` under every R minor but one.
-A slot missing from it is not merely unadvertised, it is broken: every request falls through to the flat index, which holds a single build per package, so clients on any R minor other than the one the slot was built against die at `dyn.load()`.
 
-Alpine 3.22 sat in that state for months, built nightly and never routed.
+In order:
-Retire a slot in this order:
 
 1. Remove it from `local.rpkgs_slots` in `cdn.tf` and from `DistroId` in the website's `src/src/data/repos.ts`, then `tofu apply`.
-   This is the decision; everything below is cleanup.
+2. `just retire-slot <codename> --apply`, then commit.
-2. Strip it from the pipeline matrices and commit the result:
+3. Delete its cron jobs on the Crow server; `local/retire-slot.R` prints them with the API call.
-
+4. Delete its objects:
-   ```sh
-   # dry run
-   just retire-slot alpine322
-   # write the changes
-   just retire-slot alpine322 --apply
-   ```
-
-3. Delete its cron jobs on the Crow server.
-   They are not in this repository, and removing a matrix row does not remove the cron that targets it.
-   `local/retire-slot.R` prints the ones it finds, with the API call.
-4. Delete its objects from the bucket, which needs the B2 credentials and therefore runs in CI:
 
    ```sh
    crow pipeline create devxy/build-cran-binaries --branch main \
-     --var retire_slot=alpine322 --var RETIRE_APPLY=yes
+     --var retire_slot=<codename> --var RETIRE_APPLY=yes
    ```
 
-`local/retire-slot.R` refuses any slot still listed in `cdn.tf`, so steps 2 and 4 cannot run against a live slot.
+Steps 2 and 4 refuse any slot still listed in `cdn.tf`.
-Doing step 4 before step 2 is the one ordering that actively breaks things: the next nightly would rebuild everything just deleted.
+Running step 4 before step 2 means the next nightly rebuilds everything just deleted.
 
-`just check-slots` fails when any pipeline still targets a slot the edge does not route, which is the drift that put Alpine 3.22 where it was.
+`just check-slots` fails when a pipeline still targets an unrouted slot.
-It is one-directional on purpose: a published slot missing from _some_ pipeline is only reported, because several pipelines cover a subset deliberately.
 
 ## Helpers
 

local/check-slots.R

@@ -1,47 +1,20 @@
-### Fail when a pipeline still targets a slot the edge does not route.
+### Fail when a pipeline targets a slot that is not in `local.rpkgs_slots`.
 #
-# Usage:
 #   Rscript local/check-slots.R
 #
-# A slot is only reachable by clients if it appears in `local.rpkgs_slots` in
+# That list feeds `UNION_SLOTS`, which `edge/rpkgs-router.ts` gates per-R-minor
-# `cdn.tf`. That list feeds the `UNION_SLOTS` variable, and
+# routing on. An unrouted slot serves the flat index to every client, so its
-# `edge/rpkgs-router.ts` gates all per-R-minor index rewriting on it. A slot
+# binaries fail at dyn.load() under every R minor but one.
-# missing from it is not merely unadvertised: every request falls through to
-# the flat index, which holds a single build per package, so clients on any R
-# minor other than the one the slot was built against die at dyn.load().
 #
-# Alpine 3.22 sat in that state for months. It was built nightly, its per-minor
+# One-directional: a published slot missing from some pipeline is only
-# indexes were correct, and it was never routed. `verify_routing=all` found it
+# reported, since several pipelines cover a subset on purpose.
-# only because the run happened to include it: 1056 of 1056 checks passed on
-# the 16 published slots and every failure in the run was alpine322, on both
-# architectures, identical across all three URL forms.
-#
-# The check is deliberately one-directional. A pipeline targeting an unpublished
-# slot is an error, because it spends build capacity on binaries nobody can
-# reach and turns `verify_routing=all` permanently red for a reason that is not
-# a defect. A published slot missing from *some* pipeline is only reported:
-# several pipelines cover a subset on purpose (archive-missed-packages has no
-# ubuntu rows, weekly-rebuild-missing has no alpine-324 rows), and the coverage
-# a pipeline needs is its own business.
-#
-# The website's `DistroId` union in `src/src/data/repos.ts` must agree with
-# `cdn.tf` too, but it lives in another repository so it cannot be checked here.
 
 options(error = function() {
   cat("ERROR:", geterrmessage(), "\n", file = stdout())
   q(status = 1)
 })
 
-# `cdn.tf` is the source of truth, so read it rather than restating the list.
+# Parses the distro list out of the setproduct's second array.
-# The shape this depends on is the second string array inside the
-# `rpkgs_slots` local, i.e. the distro list of the setproduct:
-#
-#   rpkgs_slots = [
-#     for pair in setproduct(
-#       ["amd64", "arm64"],
-#       ["resolute", "noble", ...]
-#     ) : "${pair[0]}/${pair[1]}"
-#   ]
 published_codenames <- function(path = "cdn.tf") {
   if (!file.exists(path)) {
     stop(
@@ -59,27 +32,23 @@ published_codenames <- function(path = "cdn.tf") {
   }
   block <- paste(lines[start:min(start + 10L, length(lines))], collapse = "\n")
   arrays <- regmatches(block, gregexpr("\\[[^]]*\\]", block))[[1L]]
-  # The first array is the setproduct's arch list, the second is the distros.
   if (length(arrays) < 2L) {
     stop(
-      "The `rpkgs_slots` local no longer has the two-array setproduct shape this parser expects. Update local/check-slots.R alongside cdn.tf.", # nolint
+      "`rpkgs_slots` no longer has the two-array setproduct shape this parses.",
       call. = FALSE
     )
   }
-  codenames <- regmatches(arrays[[2L]], gregexpr('"[^"]+"', arrays[[2L]]))[[1L]]
+  codenames <- gsub(
-  codenames <- gsub('"', "", codenames)
+    '"',
+    "",
+    regmatches(arrays[[2L]], gregexpr('"[^"]+"', arrays[[2L]]))[[1L]]
+  )
   if (length(codenames) == 0L) {
-    stop(
+    stop("Parsed an empty distro list from cdn.tf.", call. = FALSE)
-      "Parsed an empty distro list from cdn.tf; refusing to report every pipeline slot as unpublished.",
-      call. = FALSE
-    )
   }
   codenames
 }
 
-# Pipelines name a slot `<os>-<version>`, the edge and the bucket name it by
-# codename. The ubuntu half of this mapping is also in `edge/rpkgs-router.ts`
-# (UBUNTU_CODENAMES) and `local/build-all.R`.
 UBUNTU_CODENAMES <- c(
   resolute = "ubuntu-2604",
   noble = "ubuntu-2404",
@@ -102,26 +71,23 @@ as_pipeline_os <- function(codename) {
   )
 }
 
-# Structural lines only, so a codename mentioned in a comment is not a finding.
+# Structural lines only, so a codename in a comment is not a finding.
-# Three shapes carry a slot: a matrix row (`- OS: alpine-323`), a continuation
-# key (`CODENAME: alpine323`), and a dropdown option (`- alpine-323-amd64`).
 slots_in_pipeline <- function(path) {
   lines <- readLines(path, warn = FALSE)
-  hits <- character(0)
   row <- regmatches(
     lines,
     regexec("^\\s*-?\\s*(OS|CODENAME|OS_ID):\\s*(\\S+)\\s*$", lines)
   )
-  hits <- c(
-    hits,
-    vapply(row, function(m) if (length(m) == 3L) m[[3L]] else NA_character_, "")
-  )
   opt <- regmatches(
     lines,
     regexec("^\\s*-\\s*([a-z0-9]+-[a-z0-9]+)-(amd64|arm64)\\s*$", lines)
   )
   hits <- c(
-    hits,
+    vapply(
+      row,
+      function(m) if (length(m) == 3L) m[[3L]] else NA_character_,
+      ""
+    ),
     vapply(opt, function(m) if (length(m) == 4L) m[[2L]] else NA_character_, "")
   )
   hits <- hits[!is.na(hits)]
@@ -140,7 +106,7 @@ if (anyNA(published_os)) {
     call. = FALSE
   )
 }
-# A slot is written either way depending on the file, so accept both spellings.
+# Files spell a slot either way, so accept both.
 published <- unique(c(published_os, codenames))
 
 cat(sprintf(
@@ -151,8 +117,7 @@ cat(sprintf(
 files <- sort(list.files(".crow", pattern = "\\.yaml$", full.names = TRUE))
 findings <- list()
 for (f in files) {
-  found <- slots_in_pipeline(f)
+  stale <- setdiff(slots_in_pipeline(f), published)
-  stale <- setdiff(found, published)
   if (length(stale) > 0L) {
     findings[[f]] <- stale
     cat(sprintf(
@@ -163,11 +128,13 @@ for (f in files) {
   }
 }
 
-covered <- unique(unlist(lapply(files, slots_in_pipeline)))
+uncovered <- setdiff(
-uncovered <- setdiff(published_os, covered)
+  published_os,
+  unique(unlist(lapply(files, slots_in_pipeline)))
+)
 if (length(uncovered) > 0L) {
   cat(sprintf(
-    "\nNote: %s appear(s) in no pipeline at all. That is a gap, not necessarily a bug.\n",
+    "\nNote: %s appear(s) in no pipeline at all.\n",
     paste(sort(uncovered), collapse = ", ")
   ))
 }
@@ -178,7 +145,7 @@ if (length(findings) == 0L) {
 }
 
 cat(sprintf(
-  "\n%d file(s) target a slot the edge does not route. Retire it with:\n  just retire-slot <codename>\n",
+  "\n%d file(s) target an unrouted slot. Retire it with:\n  just retire-slot <codename>\n",
   length(findings)
 ))
 quit(status = 1)

local/retire-slot.R

@@ -1,39 +1,20 @@
 ### Retire a slot: strip it from the pipelines and delete its objects from S3.
 #
-# Usage:
 #   Rscript local/retire-slot.R <codename> [--apply] [--config-only|--assets-only]
 #
-#   Rscript local/retire-slot.R alpine322                 # dry run, both halves
-#   Rscript local/retire-slot.R alpine322 --config-only    # edit .crow/, no S3
-#   Rscript local/retire-slot.R alpine322 --assets-only --apply
-#
 # Retiring a distro retires both architectures, because `local.rpkgs_slots` in
-# `cdn.tf` is a setproduct over `["amd64", "arm64"]` and there is no way to
+# `cdn.tf` is a setproduct over them.
-# publish one without the other.
 #
-# Three things have to happen and nothing ties them together on its own:
+# Order: remove the slot from `local.rpkgs_slots` (and the website's
+# `DistroId`) and apply that first, then run this. Deleting objects while a
+# pipeline still targets the slot just means the next nightly rebuilds them.
 #
-#   1. the slot leaves `local.rpkgs_slots` in `cdn.tf` (and `DistroId` in the
+# Cron jobs live on the Crow server, not here, and are printed at the end.
-#      website's `src/src/data/repos.ts`), which is what stops the edge routing
-#      it. Do this first and by hand: it is the decision, the rest is cleanup.
-#   2. the slot leaves every `.crow/*.yaml` matrix, which is what stops the
-#      fleet building, indexing and verifying it. This script does that.
-#   3. the objects leave the bucket. This script does that too.
 #
-# A fourth step cannot be scripted from here: the slot's cron jobs live on the
+# Only structural YAML is edited. Prose mentioning the slot is left alone, so
-# Crow server, not in this repository, and removing a matrix row does not remove
+# grep for the name afterwards.
-# the cron that targets it. The script prints the ones it finds by name at the
-# end, with the API call to delete them.
 #
-# Dry run by default. Nothing is edited or deleted without --apply.
+# Dry run by default.
-#
-# Order matters. Deleting the objects while a pipeline still targets the slot
-# means the next nightly rebuilds them.
-#
-# Only structural YAML is edited: matrix rows, dropdown options and the
-# `OS_VERSION` tag. Prose that happens to mention the slot (a usage example in
-# a header comment, a dropdown `description:`) is left alone, because rewriting
-# free text automatically is guesswork. Grep for the name afterwards.
 
 options(error = function() {
   cat("ERROR:", geterrmessage(), "\n", file = stdout())
@@ -53,15 +34,14 @@ if (length(args) != 1L || (config_only && assets_only)) {
 }
 codename <- args[[1L]]
 
-# `alpine322` is how the bucket, the edge and `CODENAME:` spell it; `alpine-322`
+# The bucket and `CODENAME:` say `alpine322`; `OS:` and the options say
-# is how `OS:` and the dropdown options spell it. Both have to go.
+# `alpine-322`. Both have to go.
 pipeline_os <- if (grepl("^rhel[0-9]+$", codename)) {
   sub("^rhel", "redhat-", codename)
 } else if (grepl("^alpine[0-9]+$", codename)) {
   sub("^alpine", "alpine-", codename)
 } else {
-  # Single bracket, so an unknown codename yields NA and the check below
+  # Single bracket so an unknown codename yields NA, not an error.
-  # reports it rather than raising "subscript out of bounds".
   c(resolute = "ubuntu-2604", noble = "ubuntu-2404", jammy = "ubuntu-2204")[
     codename
   ]
@@ -79,9 +59,8 @@ image_tag <- if (grepl("^alpine-", pipeline_os)) {
   codename
 }
 
-# Refuse to retire a slot that is still routed. Without this the script is a
+# Refuse a slot that is still routed: deleting its objects would leave the edge
-# fast way to break a live slot: the edge would keep sending clients to indexes
+# serving indexes that name files which no longer exist.
-# whose objects no longer exist, and every install would 404.
 tf <- readLines("cdn.tf", warn = FALSE)
 start <- grep("rpkgs_slots\\s*=", tf)
 if (length(start) != 1L) {
@@ -110,9 +89,7 @@ cat(sprintf(
 
 ## 1. Pipeline definitions ----------------------------------------------------
 
-# Drops a matrix row and everything indented under it, a dropdown option, and
+# Line-based on purpose: a YAML round trip would reflow all 18 files.
-# an `OS_VERSION` entry for the retired tag. Line-based on purpose: round
-# tripping through a YAML parser would reflow all 18 files and bury the change.
 strip_slot <- function(lines) {
   keep <- rep(TRUE, length(lines))
   i <- 1L
@@ -153,8 +130,7 @@ strip_slot <- function(lines) {
     i <- i + 1L
   }
   out <- lines[keep]
-  # A dropdown whose default was the retired tag would reject its own default,
+  # A dropdown cannot default to an option that no longer exists.
-  # so move it to the first surviving option of the same block.
   bad_default <- grepl(sprintf("^\\s*default:\\s*'%s'\\s*$", image_tag), out)
   if (any(bad_default)) {
     for (k in which(bad_default)) {
@@ -162,8 +138,7 @@ strip_slot <- function(lines) {
       if (length(opts) == 0L) {
         next
       }
-      # Walk back to the first option of this block rather than the last, so the
+      # Walk back to this block's first option, not the nearest one.
-      # default lands on the lowest surviving version instead of an unrelated one.
       first <- max(opts)
       while (first > 1L && grepl("^\\s*-\\s*'[^']+'\\s*$", out[[first - 1L]])) {
         first <- first - 1L
@@ -213,7 +188,7 @@ if (!config_only) {
   library(s3fs, quietly = TRUE)
   if (!nzchar(Sys.getenv("B2_S3_ACCESS_KEY"))) {
     stop(
-      "B2_S3_ACCESS_KEY is unset. B2 needs credentials even to list, and an unauthenticated list returns empty, which is indistinguishable from an already-clean slot.", # nolint
+      "B2_S3_ACCESS_KEY is unset. B2 lists nothing without credentials, which is indistinguishable from an already-clean slot.", # nolint
       call. = FALSE
     )
   }
@@ -248,7 +223,7 @@ if (!config_only) {
   cat("\n")
 }
 
-## 3. What is left for a human ------------------------------------------------
+## 3. Crons ------------------------------------------------------------------
 
 if (!apply_changes) {
   cat("Dry run: nothing was edited or deleted. Re-run with --apply.\n\n")
@@ -259,9 +234,7 @@ cat(sprintf(
   "  crow-at ci.devxy.io repo cron ls devxy/build-cran-binaries | grep %s\n",
   pipeline_os
 ))
-cat(
+cat("  # `crow repo cron rm` 404s here; the API base is /api/v1\n")
-  "  # `crow repo cron rm` 404s on this server version; use the API, whose base is /api/v1\n"
-)
 cat("  eval \"$(crow-at --export ci.devxy.io)\"\n")
 cat("  curl -s -X DELETE -H \"Authorization: Bearer $CROW_TOKEN\" \\\n")
 cat("    https://ci.devxy.io/api/v1/repos/5/cron/<cron-id>\n")