docs: implementation plans for R-minor-sensitive builds
Split into two sequenced plans: bincraft 4.2.0 enhancements (per-minor index + classifier-driven process_cran_updates) and the pipeline integration (full + iterative). Spec revised to classify at the orchestration layer instead of a build_binary_package "auto" mode.
This commit is contained in:
parent
59c3e0960c
commit
d328f261d2
1 changed files with 962 additions and 14 deletions
471
docs/superpowers/plans/2026-06-13-pipeline-r-minor-builds.md
Normal file
471
docs/superpowers/plans/2026-06-13-pipeline-r-minor-builds.md
Normal file
|
|
@ -0,0 +1,471 @@
|
|||
# Pipeline R-minor Builds Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make the full (`build-all-versions-*`) and iterative (`process-updates-*`) pipelines build R-minor-sensitive packages once per installed R minor, and everything else once into the generic slot.
|
||||
|
||||
**Architecture:** A precompute step classifies packages (cheap CRAN-metadata rules + source download only for the ambiguous compiled subset) and emits a sensitivity flag. Build scripts pass a concrete `is_r_minor_sensitive` per package on the primary-R pass and re-run a sensitive-only pass under each additional `/opt/R/*` minor. Per-minor `PACKAGES` indexes are uploaded for each touched slot.
|
||||
|
||||
**Tech Stack:** R scripts under `local/`, crow/woodpecker CI YAML under `.crow/`, bincraft `>= 4.2.0` (see `2026-06-13-bincraft-r-minor-enhancements.md`).
|
||||
|
||||
**HARD DEPENDENCY:** bincraft `4.2.0` (the companion plan) must be merged and released first — Tasks 4-6 call `r_minor_detection`/`upload_package_index(r_minor=)` which only exist there. Tasks 1-3 (this repo's R scripts) can be written and unit-tested before the release.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: pure metadata classifier helper
|
||||
|
||||
**Files:**
|
||||
- Create: `local/r-minor-helpers.R`
|
||||
- Test: `local/tests/test-r-minor-helpers.R`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```r
|
||||
# local/tests/test-r-minor-helpers.R
|
||||
source(file.path("local", "r-minor-helpers.R"))
|
||||
|
||||
test_that("pure-r (NeedsCompilation != yes) is not sensitive", {
|
||||
expect_identical(classify_from_metadata("no", NA, c("Rcpp")), "not-sensitive")
|
||||
expect_identical(classify_from_metadata("", "Rcpp", c("Rcpp")), "not-sensitive")
|
||||
})
|
||||
|
||||
test_that("LinkingTo a risky dep is sensitive (version constraints stripped)", {
|
||||
expect_identical(classify_from_metadata("yes", "Rcpp (>= 1.0)", c("Rcpp")), "sensitive")
|
||||
expect_identical(classify_from_metadata("yes", "R6,\n cpp11", c("Rcpp", "cpp11")), "sensitive")
|
||||
})
|
||||
|
||||
test_that("compiled but no risky LinkingTo is ambiguous (needs source)", {
|
||||
expect_identical(classify_from_metadata("yes", "R6", c("Rcpp", "cpp11")), "ambiguous")
|
||||
expect_identical(classify_from_metadata("yes", NA, c("Rcpp")), "ambiguous")
|
||||
})
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `R -q -e 'library(testthat); testthat::test_file("local/tests/test-r-minor-helpers.R")'`
|
||||
Expected: FAIL — `cannot open file 'local/r-minor-helpers.R'` / function not found.
|
||||
|
||||
- [ ] **Step 3: Write the helper**
|
||||
|
||||
```r
|
||||
# local/r-minor-helpers.R
|
||||
# Metadata-only ABI triage so the full build avoids downloading every source.
|
||||
# Mirrors bincraft::abi_classify rules 1-2; "ambiguous" packages still need a
|
||||
# source grep via bincraft::needs_per_minor_recompile().
|
||||
|
||||
classify_from_metadata <- function(needs_compilation, linking_to, risky_deps) {
|
||||
nc <- if (length(needs_compilation) == 0L || is.na(needs_compilation)) {
|
||||
""
|
||||
} else {
|
||||
tolower(trimws(needs_compilation))
|
||||
}
|
||||
if (!identical(nc, "yes")) {
|
||||
return("not-sensitive")
|
||||
}
|
||||
lt <- if (length(linking_to) == 0L || is.na(linking_to)) "" else linking_to
|
||||
linked <- trimws(unlist(strsplit(lt, "[,\n]")))
|
||||
linked <- sub("\\s*\\(.*\\)$", "", linked) # strip "(>= x)" constraints
|
||||
linked <- linked[nzchar(linked)]
|
||||
if (any(linked %in% risky_deps)) {
|
||||
return("sensitive")
|
||||
}
|
||||
"ambiguous"
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `R -q -e 'library(testthat); testthat::test_file("local/tests/test-r-minor-helpers.R")'`
|
||||
Expected: PASS (3/3).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add local/r-minor-helpers.R local/tests/test-r-minor-helpers.R
|
||||
git commit -m "feat(local): metadata-only ABI triage helper"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: add `r_minor_sensitive` to `packages-to-build.R`
|
||||
|
||||
**Files:**
|
||||
- Modify: `local/packages-to-build.R` (after `pkgs` is finalized, ~line 142)
|
||||
|
||||
- [ ] **Step 1: Append the classification block**
|
||||
|
||||
At the end of `local/packages-to-build.R` (after `setorder(pkgs, Package, Version)`), add:
|
||||
|
||||
```r
|
||||
### R-minor sensitivity (classify once per package, applied to all versions)
|
||||
source(file.path("local", "r-minor-helpers.R"))
|
||||
risky_deps <- bincraft::abi_risky_linking_deps()
|
||||
|
||||
release_meta <- data.table(
|
||||
Package = cran_release$Package,
|
||||
NeedsCompilation = cran_release$NeedsCompilation,
|
||||
LinkingTo = cran_release$LinkingTo
|
||||
)
|
||||
|
||||
meta <- release_meta[Package %in% unique(pkgs$Package)]
|
||||
meta[, triage := mapply(
|
||||
classify_from_metadata,
|
||||
NeedsCompilation,
|
||||
LinkingTo,
|
||||
MoreArgs = list(risky_deps = risky_deps)
|
||||
)]
|
||||
|
||||
# Only the "ambiguous" compiled packages need a source grep.
|
||||
ambiguous <- meta[triage == "ambiguous", Package]
|
||||
sensitive_ambiguous <- character()
|
||||
if (length(ambiguous) > 0L) {
|
||||
tmp_src <- file.path(tempdir(), "abi_src")
|
||||
dir.create(tmp_src, showWarnings = FALSE, recursive = TRUE)
|
||||
sens <- vapply(ambiguous, function(pkg) {
|
||||
out <- tryCatch({
|
||||
dl <- utils::download.packages(
|
||||
pkg, destdir = tmp_src,
|
||||
repos = "https://cloud.r-project.org", quiet = TRUE
|
||||
)
|
||||
isTRUE(as.logical(bincraft::needs_per_minor_recompile(dl[1L, 2L])))
|
||||
}, error = function(e) TRUE) # fail safe: treat as sensitive
|
||||
out
|
||||
}, logical(1L))
|
||||
sensitive_ambiguous <- ambiguous[sens]
|
||||
}
|
||||
|
||||
sensitive_pkgs <- unique(c(meta[triage == "sensitive", Package], sensitive_ambiguous))
|
||||
pkgs[, r_minor_sensitive := Package %in% sensitive_pkgs]
|
||||
sprintf("R-minor-sensitive packages: %s of %s", length(sensitive_pkgs), uniqueN(pkgs$Package))
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify the script parses and the column is added (offline smoke)**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
R -q -e '
|
||||
library(data.table)
|
||||
source("local/r-minor-helpers.R")
|
||||
pkgs <- data.table(Package = c("A","B"), Version = c("1","1"))
|
||||
cran_release <- data.frame(Package = c("A","B"),
|
||||
NeedsCompilation = c("no","yes"), LinkingTo = c(NA,"Rcpp"),
|
||||
stringsAsFactors = FALSE)
|
||||
risky_deps <- c("Rcpp")
|
||||
release_meta <- as.data.table(cran_release)
|
||||
meta <- release_meta[Package %in% unique(pkgs$Package)]
|
||||
meta[, triage := mapply(classify_from_metadata, NeedsCompilation, LinkingTo,
|
||||
MoreArgs = list(risky_deps = risky_deps))]
|
||||
sensitive_pkgs <- meta[triage == "sensitive", Package]
|
||||
pkgs[, r_minor_sensitive := Package %in% sensitive_pkgs]
|
||||
stopifnot(identical(pkgs$r_minor_sensitive, c(FALSE, TRUE)))
|
||||
cat("OK\n")'
|
||||
```
|
||||
Expected: prints `OK` (B classified sensitive via LinkingTo, A not).
|
||||
|
||||
- [ ] **Step 3: Persist the sensitive subset in install-deps**
|
||||
|
||||
In `.crow/build-all-versions-install-deps-amd64.yaml` and `...-arm64.yaml`, change the precompute command (line ~38) from saving only `pkgs_to_build.rds` to also saving the subset:
|
||||
|
||||
```yaml
|
||||
- /opt/R/$R_VERSION/bin/R -q -e "source('local/packages-to-build.R'); saveRDS(pkgs, '/mnt/cache/packages/pkgs_to_build.rds'); saveRDS(pkgs[r_minor_sensitive == TRUE], '/mnt/cache/packages/r_minor_sensitive_pkgs.rds'); sprintf('Precomputed %s package versions (%s r-minor-sensitive)', nrow(pkgs), nrow(pkgs[r_minor_sensitive == TRUE]))"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add local/packages-to-build.R .crow/build-all-versions-install-deps-amd64.yaml .crow/build-all-versions-install-deps-arm64.yaml
|
||||
git commit -m "feat(full): classify r-minor sensitivity in install-deps precompute"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: per-package flag + `--sensitive-only` in `build-all.R`
|
||||
|
||||
**Files:**
|
||||
- Modify: `local/build-all.R`
|
||||
- Test: `local/tests/test-build-all-args.R` (new — covers the arg/flag parsing seam only)
|
||||
|
||||
- [ ] **Step 1: Write the failing test for the arg parser**
|
||||
|
||||
```r
|
||||
# local/tests/test-build-all-args.R
|
||||
source(file.path("local", "r-minor-helpers.R"))
|
||||
|
||||
test_that("parse_build_args splits flags from positionals", {
|
||||
a <- parse_build_args(c("--sensitive-only", "4", "2", "8"))
|
||||
expect_true(a$sensitive_only)
|
||||
expect_identical(a$split_into, 4L)
|
||||
expect_identical(a$split_index, 2L)
|
||||
expect_identical(a$ncpus, 8L)
|
||||
|
||||
b <- parse_build_args(c("4", "2", "8"))
|
||||
expect_false(b$sensitive_only)
|
||||
expect_identical(b$split_into, 4L)
|
||||
})
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add `parse_build_args` to the helpers and run the test**
|
||||
|
||||
Append to `local/r-minor-helpers.R`:
|
||||
|
||||
```r
|
||||
parse_build_args <- function(args) {
|
||||
sensitive_only <- "--sensitive-only" %in% args
|
||||
pos <- args[!startsWith(args, "--")]
|
||||
list(
|
||||
sensitive_only = sensitive_only,
|
||||
split_into = as.integer(pos[1L]),
|
||||
split_index = as.integer(pos[2L]),
|
||||
ncpus = as.integer(pos[3L])
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
Run: `R -q -e 'library(testthat); testthat::test_file("local/tests/test-build-all-args.R")'`
|
||||
Expected: PASS (1/1, 5 expectations).
|
||||
|
||||
- [ ] **Step 3: Rewire `build-all.R` to use the parser, the flag, and the per-row sensitivity**
|
||||
|
||||
Replace the arg-parsing header and the `mapply` build loop of `local/build-all.R`. New top:
|
||||
|
||||
```r
|
||||
sink(stdout(), type = "message")
|
||||
options(crayon.enabled = TRUE, future.globals.onReference = NULL)
|
||||
source(file.path("local", "r-minor-helpers.R"))
|
||||
|
||||
args <- commandArgs(trailingOnly = TRUE)
|
||||
parsed <- parse_build_args(args)
|
||||
split_into <- parsed$split_into
|
||||
split_index <- parsed$split_index
|
||||
ncpus <- parsed$ncpus
|
||||
sensitive_only <- parsed$sensitive_only
|
||||
options(Ncpus = ncpus)
|
||||
|
||||
library(bincraft, quietly = TRUE)
|
||||
library(future)
|
||||
plan("sequential")
|
||||
|
||||
pkgs <- if (sensitive_only) {
|
||||
readRDS("/mnt/cache/packages/r_minor_sensitive_pkgs.rds")
|
||||
} else {
|
||||
readRDS("/mnt/cache/packages/pkgs_to_build.rds")
|
||||
}
|
||||
# Back-compat: tolerate an older RDS without the column (treat all as non-sensitive)
|
||||
if (is.null(pkgs$r_minor_sensitive)) pkgs$r_minor_sensitive <- FALSE
|
||||
sprintf("Total# of remaining package versions: %s (sensitive_only=%s)", nrow(pkgs), sensitive_only)
|
||||
```
|
||||
|
||||
Keep the existing chunk-split, exclude-list, and `s3_cache` lines unchanged. Then replace the `mapply(...)` call with:
|
||||
|
||||
```r
|
||||
n <- nrow(chunk)
|
||||
mapply(function(pkg, ver, sens, i) {
|
||||
cat(sprintf("[%d/%d] %s_%s (r_minor_sensitive=%s)\n", i, n, pkg, ver, sens))
|
||||
bincraft::build_binary_package(
|
||||
pkg,
|
||||
tag = ver,
|
||||
is_r_minor_sensitive = isTRUE(sens),
|
||||
s3_endpoint = "https://s3.eu-central-003.backblazeb2.com",
|
||||
s3_region = "eu-central-003",
|
||||
s3_bucket = "devxy-rpkgs-binaries",
|
||||
s3_access_key_id = Sys.getenv("B2_S3_ACCESS_KEY"),
|
||||
s3_secret_access_key = Sys.getenv("B2_S3_SECRET_KEY"),
|
||||
s3_package_cache = s3_cache,
|
||||
metadata_db_host = "r-binaries.devxy.io",
|
||||
metadata_db_name = "build_metadata",
|
||||
metadata_db_table = "single_builds",
|
||||
metadata_db_user = "rpkgs",
|
||||
metadata_db_password = Sys.getenv("PGPASS"),
|
||||
metadata_db_sslmode = "require",
|
||||
metadata_db_port = 15432,
|
||||
archive = TRUE,
|
||||
upload = TRUE,
|
||||
store_build_metadata = TRUE
|
||||
)
|
||||
}, chunk$Package, chunk$Version, chunk$r_minor_sensitive, seq_len(n))
|
||||
```
|
||||
|
||||
Note: in `--sensitive-only` mode every row already has `r_minor_sensitive == TRUE`, so the same loop builds them into per-minor slots; no special-casing needed.
|
||||
|
||||
- [ ] **Step 4: Verify the script parses**
|
||||
|
||||
Run: `R -q -e 'invisible(parse(file="local/build-all.R")); cat("parsed OK\n")'`
|
||||
Expected: `parsed OK`.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add local/build-all.R local/r-minor-helpers.R local/tests/test-build-all-args.R
|
||||
git commit -m "feat(full): per-package r-minor flag and --sensitive-only mode in build-all.R"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: multi-R loop + per-minor index in `build-all-versions-*.yaml`
|
||||
|
||||
> Depends on bincraft 4.2.0 (`upload_package_index(r_minor=)`).
|
||||
|
||||
**Files:**
|
||||
- Modify: `.crow/build-all-versions-amd64.yaml`, `.crow/build-all-versions-arm64.yaml`
|
||||
|
||||
- [ ] **Step 1: Add the sensitive-only multi-R pass after the primary build**
|
||||
|
||||
In each file's `commands:` (after the existing `Rscript local/build-all.R $SPLIT_INTO $SPLIT_INDEX $NCPUS` line and before the `process_unarchived_pkgs` line), insert this multiline command:
|
||||
|
||||
```yaml
|
||||
- |
|
||||
PRIMARY_MINOR=$(echo "$R_VERSION" | cut -d. -f1-2)
|
||||
for RBIN in /opt/R/*/bin/R; do
|
||||
RV=$(basename "$(dirname "$(dirname "$RBIN")")")
|
||||
RMINOR=$(echo "$RV" | cut -d. -f1-2)
|
||||
[ "$RMINOR" = "$PRIMARY_MINOR" ] && continue
|
||||
echo "=== R-minor-sensitive pass under R $RV ==="
|
||||
R_VERSION="$RV" $XVFB $XVFB_ARGS -n $SPLIT_INDEX -- "$RBIN" Rscript local/build-all.R --sensitive-only $SPLIT_INTO $SPLIT_INDEX $NCPUS 2>&1 || true
|
||||
done
|
||||
```
|
||||
|
||||
(`$XVFB`/`$XVFB_ARGS` are already defined on the preceding line in this workflow.) Note the primary pass still runs `build-all.R` without `--sensitive-only`, building everything and routing sensitive packages into the primary minor slot.
|
||||
|
||||
- [ ] **Step 2: Add a per-minor index upload step**
|
||||
|
||||
Append a new step after the build step (mirroring the existing CDN/index pattern). Add to `commands` of a new `Upload per-minor indexes` step (or extend the existing index handling) the following, which uploads the generic index plus one per discovered minor:
|
||||
|
||||
```yaml
|
||||
- |
|
||||
CODENAME=$(/opt/R/$R_VERSION/bin/Rscript -e "cat(bincraft::set_codename(NULL))")
|
||||
# generic slot
|
||||
/opt/R/$R_VERSION/bin/R -q -e "bincraft::upload_package_index(codename = '$CODENAME', s3_endpoint = 'https://s3.eu-central-003.backblazeb2.com', s3_region = 'eu-central-003', s3_bucket = 'devxy-rpkgs-binaries', s3_access_key_id = Sys.getenv('B2_S3_ACCESS_KEY'), s3_secret_access_key = Sys.getenv('B2_S3_SECRET_KEY'))"
|
||||
for RBIN in /opt/R/*/bin/R; do
|
||||
RMINOR=$(basename "$(dirname "$(dirname "$RBIN")")" | cut -d. -f1-2)
|
||||
/opt/R/$R_VERSION/bin/R -q -e "bincraft::upload_package_index(codename = '$CODENAME', r_minor = '$RMINOR', s3_endpoint = 'https://s3.eu-central-003.backblazeb2.com', s3_region = 'eu-central-003', s3_bucket = 'devxy-rpkgs-binaries', s3_access_key_id = Sys.getenv('B2_S3_ACCESS_KEY'), s3_secret_access_key = Sys.getenv('B2_S3_SECRET_KEY'))" || true
|
||||
done
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Validate YAML**
|
||||
|
||||
Run: `R -q -e 'invisible(lapply(c(".crow/build-all-versions-amd64.yaml",".crow/build-all-versions-arm64.yaml"), yaml::yaml.load_file)); cat("yaml OK\n")'`
|
||||
Expected: `yaml OK`.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add .crow/build-all-versions-amd64.yaml .crow/build-all-versions-arm64.yaml
|
||||
git commit -m "feat(full): sensitive-only multi-R passes and per-minor index upload"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: classifier + multi-R in all `process-updates-*.yaml`
|
||||
|
||||
> Depends on bincraft 4.2.0 (`r_minor_detection`, `upload_package_index(r_minor=)`).
|
||||
|
||||
**Files (14):** every `.crow/process-updates-<plat>-<arch>.yaml`:
|
||||
`alpine-322`, `alpine-323`, `redhat-8`, `redhat-9`, `redhat-10`, `ubuntu-2204`, `ubuntu-2404` × `amd64`, `arm64`.
|
||||
|
||||
The transform is identical in shape; only the already-present `platform=`, `R_VERSION`, `ARCH`, and the index `codename=` differ per file (leave those as-is).
|
||||
|
||||
- [ ] **Step 1: Edit the `process_cran_updates` call (primary pass)**
|
||||
|
||||
In each file, inside the single long `bincraft::process_cran_updates(...)` argument list, add `r_minor_detection = 'classifier', ` immediately before `s3_endpoint = ...`. Leave all other args unchanged.
|
||||
|
||||
- [ ] **Step 2: Add the sensitive-only multi-R pass**
|
||||
|
||||
Immediately after the `process_cran_updates(...)` command line, insert (preserve the per-file `'<platform>'` string and `$INTERVAL`):
|
||||
|
||||
```yaml
|
||||
- |
|
||||
PRIMARY_MINOR=$(echo "$R_VERSION" | cut -d. -f1-2)
|
||||
for RBIN in /opt/R/*/bin/R; do
|
||||
RV=$(basename "$(dirname "$(dirname "$RBIN")")")
|
||||
RMINOR=$(echo "$RV" | cut -d. -f1-2)
|
||||
[ "$RMINOR" = "$PRIMARY_MINOR" ] && continue
|
||||
echo "=== R-minor-sensitive update pass under R $RV ==="
|
||||
xvfb-run "$RBIN" -q -e "options(crayon.enabled = TRUE, Ncpus = 4, future.globals.onReference = NULL); bincraft::process_cran_updates(interval = $INTERVAL, platform = '<PLATFORM>', process_updated = TRUE, process_new = FALSE, process_removed = FALSE, r_minor_detection = 'classifier', r_minor_sensitive_only = TRUE, s3_endpoint = 'https://s3.eu-central-003.backblazeb2.com', s3_region = 'eu-central-003', s3_bucket = 'devxy-rpkgs-binaries', s3_access_key_id = Sys.getenv('B2_S3_ACCESS_KEY'), s3_secret_access_key = Sys.getenv('B2_S3_SECRET_KEY'), metadata_db_host = 'r-binaries.devxy.io', metadata_db_name = 'build_metadata', metadata_db_table = 'single_builds', metadata_db_user = 'rpkgs', metadata_db_password = Sys.getenv('PGPASS'), metadata_db_sslmode = 'require', metadata_db_port = 15432, archive = TRUE, upload = TRUE, store_build_metadata = TRUE)" || true
|
||||
done
|
||||
```
|
||||
|
||||
Replace `<PLATFORM>` with that file's existing platform string (e.g. `alpine-323`, `redhat-9`, `ubuntu-2404`). `process_removed = FALSE` on the extra passes (removals are handled once by the primary pass).
|
||||
|
||||
- [ ] **Step 3: Add per-minor index upload**
|
||||
|
||||
After the existing `upload_package_index(codename = "<CODENAME>", ...)` line in each file, append (reusing the file's existing `codename`):
|
||||
|
||||
```yaml
|
||||
- |
|
||||
for RBIN in /opt/R/*/bin/R; do
|
||||
RMINOR=$(basename "$(dirname "$(dirname "$RBIN")")" | cut -d. -f1-2)
|
||||
/opt/R/$R_VERSION/bin/R -q -e 'library(bincraft); upload_package_index(codename = "<CODENAME>", r_minor = "'"$RMINOR"'", s3_endpoint = "https://s3.eu-central-003.backblazeb2.com", s3_region = "eu-central-003", s3_bucket = "devxy-rpkgs-binaries", s3_access_key_id = Sys.getenv("B2_S3_ACCESS_KEY"), s3_secret_access_key = Sys.getenv("B2_S3_SECRET_KEY"))' || true
|
||||
done
|
||||
```
|
||||
|
||||
Replace `<CODENAME>` with the file's existing codename value (e.g. `alpine323`, `rhel9`, `ubuntu2404`).
|
||||
|
||||
- [ ] **Step 4: Verify every file changed and parses**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
R -q -e 'fs <- Sys.glob(".crow/process-updates-*.yaml"); invisible(lapply(fs, yaml::yaml.load_file)); cat(length(fs), "files parse OK\n")'
|
||||
grep -L "r_minor_detection = 'classifier'" .crow/process-updates-*.yaml
|
||||
```
|
||||
Expected: `14 files parse OK`, and the `grep -L` prints **nothing** (every file contains the classifier arg).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add .crow/process-updates-*.yaml
|
||||
git commit -m "feat(updates): classifier-driven r-minor builds + per-minor index across platforms"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: pin bincraft 4.2.0, remove the standalone workflow
|
||||
|
||||
**Files:**
|
||||
- Modify: all workflows pinning bincraft (`grep -rln "bincraft.git@v" .crow/`), and `.crow/build-all-versions-install-deps-*.yaml` (which installs from `main`)
|
||||
- Delete: `.crow/build-r-minor-sensitive-packages.yaml`
|
||||
|
||||
- [ ] **Step 1: Bump the pin**
|
||||
|
||||
Run to find pins: `grep -rn "bincraft" .crow/ | grep -E "@v4\.1\.1|bincraft.git"`. In every `process-updates-*` file, change `bincraft.git@v4.1.1` (and the `packageVersion("bincraft") != "4.1.1"` guard) to `4.2.0`.
|
||||
|
||||
- [ ] **Step 2: Remove the superseded workflow**
|
||||
|
||||
```bash
|
||||
git rm .crow/build-r-minor-sensitive-packages.yaml
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Verify no stale references**
|
||||
|
||||
Run: `grep -rn "build-r-minor-sensitive\|@v4.1.1\|!= \"4.1.1\"" .crow/`
|
||||
Expected: no matches.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add .crow/
|
||||
git commit -m "chore: pin bincraft 4.2.0 and drop superseded standalone r-minor workflow"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: integration smoke test (manual, gated on bincraft 4.2.0 release)
|
||||
|
||||
- [ ] **Step 1: Run one iterative platform manually** via crow against a short interval and confirm: a known risky package (e.g. one LinkingTo Rcpp) lands under `…/contrib/<x.y>/` for each installed minor, and a pure-r package lands only in the generic slot.
|
||||
|
||||
- [ ] **Step 2: Confirm a client install resolves the per-minor slot.** From an R `4.4` and an R `4.5` container:
|
||||
```r
|
||||
install.packages("<risky_pkg>", repos = "https://cran.devxy.io/<codename>")
|
||||
library(<risky_pkg>) # must load without "undefined symbol"
|
||||
```
|
||||
Expected: loads under both minors. If the per-minor `PACKAGES` is missing, revisit bincraft Task 2.
|
||||
|
||||
- [ ] **Step 3: Spot-check the full build** on one platform with a small `SPLIT_INTO`, verifying the sensitive-only extra passes ran and produced per-minor artifacts.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- **Spec coverage:** install-deps precompute → Tasks 1-2; `build-all.R` per-row flag + `--sensitive-only` → Task 3; full-build multi-R loop + per-minor index → Task 4; iterative classifier + multi-R + per-minor index → Task 5; remove standalone workflow + version pin → Task 6; per-minor index client-serviceability risk → Task 7 verification.
|
||||
- **Type/name consistency:** `classify_from_metadata` and `parse_build_args` live in `local/r-minor-helpers.R` and are used in Tasks 2-3; the RDS column is `r_minor_sensitive` everywhere; the subset file is `/mnt/cache/packages/r_minor_sensitive_pkgs.rds` in install-deps (Task 2) and `build-all.R` (Task 3).
|
||||
- **Placeholders:** `<PLATFORM>`/`<CODENAME>` in Task 5 are explicit per-file substitutions (the values already exist in each file), not unfilled blanks. No "TBD"/"handle errors" placeholders.
|
||||
- **Dependency ordering:** Tasks 1-3 are pure R-script work, unit-testable now; Tasks 4-6 are gated on bincraft 4.2.0; Task 7 is post-release verification.
|
||||
Loading…
Reference in a new issue