build-cran-binaries/docs/superpowers/specs/2026-06-13-r-minor-sensitive-builds-design.md

R-minor-sensitive binary builds — design

Problem

CRAN binaries are currently built once, under a single R minor version, and served from a single generic slot (…/latest/src/contrib/). That is wrong for the minority of packages whose compiled code reaches into volatile R internals: a binary built under R 4.5 will fail to load under R 4.4 with an undefined symbol error.

bincraft v4.1.0+ can now detect these packages automatically via the ABI classifier (abi_classify() / needs_per_minor_recompile(), added in bincraft PR #49, https://codefloe.com/rpkgs/bincraft/pulls/49). The exec-env images now ship multiple R minor versions under /opt/R/. This design uses both to build R-minor-specific binaries for the sensitive packages only, in both the iterative (process-updates-*) and full (build-all-versions-*) pipelines.

Background: how bincraft already behaves

• abi_classify(path) returns a tier: pure-r (~78.6%), safe-compiled (~7.7%), or risky (~13.6%). needs_per_minor_recompile(path) is the boolean wrapper (TRUE iff risky). Both need the package source (DESCRIPTION + src/).
• build_binary_package(…, is_r_minor_sensitive = TRUE) uploads the artifact into a per-minor slot …/latest/src/contrib/<major.minor>/ and records r_version in the build metadata. With FALSE it uses the generic slot. The minor is derived from the running interpreter (R.version), so producing a 4.4 binary requires running /opt/R/4.4.x/bin/R.
• process_cran_updates() orchestrates the iterative flow with a single is_r_minor_sensitive bool for the whole run and no per-package classification or R-version loop.
• upload_package_index() writes/uploads PACKAGES* for the generic slot only; it has no per-minor support. The current standalone r-minor workflow never builds a per-minor index, so per-minor slots are effectively unservable today.

Core model

Collapse the three tiers to one boolean per package:

r_minor_sensitive := (abi_classify(pkg)$tier == "risky")

group	tiers	built under	slot
non-sensitive (~86%)	pure-r, safe-compiled	primary R only	generic contrib/
sensitive (~14%)	risky	every installed minor	per-minor contrib/<x.y>/

• Primary R = the existing R_VERSION env var in each workflow. Its pass builds the non-sensitive packages (generic slot) and the sensitive packages for its own minor slot.
• Extra minors = every other R version discovered by scanning /opt/R/ at runtime. Each runs a sensitive-only pass. An image with a single R version degrades cleanly to just the primary pass.
• The loop over minors always lives at the shell/script layer (one /opt/R/<ver>/bin/R invocation per minor), never inside a single build_binary_package() call.

Classification granularity: classify once per package (its release version) and apply the resulting flag to all archived versions of that package. Tier rarely changes across recent versions; this avoids multiplying source downloads.

Full build — build-all-versions-* + local/

install-deps step (local/packages-to-build.R)

After computing pkgs_to_build, add an r_minor_sensitive logical column:

1. Pull NeedsCompilation and LinkingTo from tools::CRAN_package_db() (already loaded).
2. Resolve cheaply, no download:
   • NeedsCompilation != "yes" → not sensitive (rule 1, pure-r).
   • LinkingTo references any bincraft::abi_risky_linking_deps() entry → sensitive (rule 2).
3. For the remaining compiled, non-LinkingTo-risky packages only: download the source and call bincraft::needs_per_minor_recompile() (rules 3/4).
4. Join the per-package flag onto every (Package, Version) row.

Outputs:

• pkgs_to_build.rds — now carries the r_minor_sensitive column.
• r_minor_sensitive_pkgs.rds — the sensitive subset, for the extra-minor passes.

build step (local/build-all.R + .crow/build-all-versions-{amd64,arm64}.yaml)

• build-all.R gains an optional --sensitive-only mode (or an arg flag). In normal mode it builds the full chunk, passing is_r_minor_sensitive = <row flag> per package. In sensitive-only mode it reads r_minor_sensitive_pkgs.rds, intersects with its chunk, and builds those with is_r_minor_sensitive = TRUE.
• The workflow step keeps the existing matrix split. After the primary Rscript build-all.R invocation, a shell loop discovers non-primary /opt/R/* minors and runs Rscript build-all.R --sensitive-only <split> <index> <ncpus> under each.
• Index upload step: generic index as today, plus a per-minor index for each minor slot that received artifacts (depends on the upload_package_index() enhancement below).

Iterative build — process-updates-*

Driven by the bincraft enhancement below; the single build step becomes:

1. Primary-R pass: process_cran_updates(…, r_minor_detection = "classifier"). Each updated/new package is classified; risky → primary minor slot, rest → generic slot. Removed-package handling stays as-is.
2. Shell loop over non-primary /opt/R/* minors: process_cran_updates(…, r_minor_detection = "classifier", r_minor_sensitive_only = TRUE). Builds only risky updates into their respective minor slots.
3. Index upload extended to cover each touched minor slot in addition to the generic slot.

Required bincraft enhancements (separate PR, coordinated release)

build_binary_package() needs no change — it already accepts a concrete is_r_minor_sensitive logical and routes the slot accordingly. Classification stays at the orchestration layer (mirroring the full-build precompute), which keeps build_binary_package's pre-build S3 skip-check and source clone untouched.

1. process_cran_updates():
   • Add r_minor_detection = c("none", "issue", "classifier") (default "none" to preserve current behavior; "issue" is today's filter_r_minor_sensitive path).
   • Add r_minor_sensitive_only (default FALSE).
   • With "classifier": for each candidate (name, version), clone the source to a temp dir and call bincraft::needs_per_minor_recompile(); pass the resulting concrete logical as is_r_minor_sensitive to build_binary_package(). When r_minor_sensitive_only = TRUE, drop non-risky candidates before building.
   • Implemented via a small internal helper classify_r_minor_sensitive(name, tag, source_org_url, local_clone_dir) returning a logical — the unit-testable seam.
2. upload_package_index():
   • Add an r_minor = NULL argument. When non-NULL (e.g. "4.4"), point the remote dir at …/contrib/<r_minor>/ and write/upload PACKAGES* (and Meta/archive.rds) there, mirroring the generic-slot logic. Extract the remote-dir construction into a pure helper package_index_remote_dir(s3_bucket, arch, codename, r_minor = NULL) — the unit-testable seam.

These two are the only bincraft changes; detection itself (PR #49) is already merged.

Scope and cleanup

• In scope: .crow/process-updates-* (iterative) and .crow/build-all-versions-* + local/build-all.R + local/packages-to-build.R (full), across amd64 and arm64 and all platforms (alpine, ubuntu, redhat). The per-platform workflow files share the same edit.
• Removed: .crow/build-r-minor-sensitive-packages.yaml — superseded by the integrated flow (it was manual, alpine-3.21-only, and issue-list-driven).
• Out of scope (call out, do not change here): weekly-rebuild-missing-*, archive-missed-packages, and the audit workflows. Revisit separately if per-minor rebuilds are wanted there too.

Open risks / notes

• R-version discovery: assumes /opt/R/<version>/bin/R layout and that the primary R_VERSION is one of the installed versions. Parse minor as major.minor from each discovered version; dedupe by minor (build once per minor even if two patch releases coexist).
• Per-minor index correctness: clients resolving bin/<os>/contrib/<x.y>/ require the per-minor PACKAGES to exist; the upload_package_index() enhancement is a hard dependency for the sensitive artifacts to be usable. Verify against a real client install before declaring done.
• Classification cost: bounded by downloading sources only for the compiled, non-LinkingTo-risky subset in install-deps. Worth measuring on a full run; if still too heavy, consider caching classifications keyed by package+version in the metadata DB.
• Double download: install-deps classification downloads some sources that the build step re-downloads. Acceptable for now; the metadata-DB cache above would also remove this.