Two-phase, TDD plan. Phase A (bincraft): patch registry loading/matching, isolated patched-binary builds, local-repo caching, and wiring into the pak install path, gated by a proof-of-mechanism task. Phase B (this repo): the RcppParallel registry entry, a validator + pre-commit hook, and passing patches through the build entry points.
49 KiB
Package Patching Implementation Plan
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Let a curated registry of packages be "patched" (env/configure/Makevars overrides and/or source diffs) before pak installs them, including when they are transitive dependencies, so compiler-/OS-specific failures like RcppParallel stop cascading.
Architecture: The mechanism lives in bincraft. Before pak::local_install_deps() runs, bincraft pre-builds each registry-matched package as a binary (reusing pkgbuild::build(binary=TRUE) + cranlike::add_PACKAGES), caches it, and serves it from a local file:// repo prepended to options("repos") so pak installs the patched binary — direct or transitive — without recompiling. The curated registry (registry.json + diff files) lives in this repo and is passed in via a new patches argument.
Tech Stack: R, pak, pkgbuild, cranlike (fork pat-s/cranlike@s3), pkgsearch, withr, jsonlite, testthat (3e), mockery. Two repos: bincraft at /Users/pjs/git/codefloe.com/rpkgs/bincraftr, and this repo build-cran-binaries.
Global Constraints
- bincraft package name is
bincraft; repo dir is…/rpkgs/bincraftr. Current version4.2.2.9999. - Logging uses bincraft's wrappers only:
log_info,log_warn,log_success,log_debug,log_error,log_header(never baremessage/catin package code). cli-style inline markup is allowed in log messages (e.g.{.pkg %s},{.path %s}); curly braces in dynamic/error text must be escaped as already done ininstall_helpers.R.- Tests:
testthat3rd edition, files attests/testthat/test-<topic>.R, network/build tests guarded withskip_on_cran()/skip_if_offline(); end-to-end build tests guarded behindskip_if_not(nzchar(Sys.getenv("BINCRAFT_PATCH_E2E"))). - New exported functions need roxygen with
@keywords internalfor non-user helpers; rundevtools::document()after adding roxygen. - Patched-binary cache dir default:
/mnt/cache/patched-binaries. - Platform tokens for matching =
c(<platform>, <family>, <arch>), e.g.ubuntu-2604→c("ubuntu-2604","ubuntu","amd64"). - Registry entry required fields:
package,versions,platforms,reason. Optional:env,configure_args,makevars,patch. - One sentence per line in prose/commit messages; do not hard-wrap at 80 columns.
- Use
fj -H codefloe.comfor any PR operations (Forgejo), notgh.
Phase A — bincraft mechanism
All Phase A paths are relative to /Users/pjs/git/codefloe.com/rpkgs/bincraftr.
Task A0: Proof of mechanism — pak installs a patched binary from a prepended file:// repo
This de-risks the core assumption before building anything on top: that pak installs a binary from a local file:// repo in preference to CRAN for an equal version, and does so without recompiling. If this fails, the contingency (documented in Step 4) is to serve patched source and rely on pkgcache build-caching — the rest of the plan changes only inside build_patched_binary().
Files:
- Create:
tools/verify-patch-mechanism.R
Interfaces:
-
Produces: a runnable script proving
pak::pkg_install()resolves a local patched binary over CRAN. No package API. -
Step 1: Write the verification script
# tools/verify-patch-mechanism.R
# Proves pak installs a patched binary from a prepended file:// repo instead of
# CRAN's, without recompiling. Run inside a Linux build-env container:
# Rscript tools/verify-patch-mechanism.R
# Exits 0 on success, 1 on failure.
pkg <- "glue" # small, pure-R CRAN package
sentinel <- "PatchMechanismProof"
work <- tempfile("verify_")
repo <- file.path(work, "repo", "src", "contrib")
lib <- file.path(work, "lib")
dir.create(repo, recursive = TRUE)
dir.create(lib, recursive = TRUE)
# 1. Download CRAN source for the current version.
ap <- available.packages(repos = "https://cloud.r-project.org")
ver <- ap[pkg, "Version"]
src <- file.path(work, sprintf("%s_%s.tar.gz", pkg, ver))
download.file(
sprintf("https://cloud.r-project.org/src/contrib/%s_%s.tar.gz", pkg, ver),
src, mode = "wb"
)
# 2. Unpack, inject a sentinel field into DESCRIPTION, build a binary.
untar(src, exdir = work)
desc <- file.path(work, pkg, "DESCRIPTION")
writeLines(c(readLines(desc), sprintf("%s: yes", sentinel)), desc)
pkgbuild::build(
file.path(work, pkg), binary = TRUE, vignettes = FALSE,
dest_path = repo, quiet = TRUE
)
built <- list.files(repo, pattern = sprintf("^%s_.*\\.tar\\.gz$", pkg), full.names = TRUE)
file.rename(built[1L], file.path(repo, sprintf("%s_%s.tar.gz", pkg, ver)))
cranlike::add_PACKAGES(sprintf("%s_%s.tar.gz", pkg, ver), repo)
# 3. Install with the local repo prepended; assert our patched build won.
withr::with_options(
list(repos = c(patched = sprintf("file://%s", dirname(dirname(repo))),
CRAN = "https://cloud.r-project.org")),
pak::pkg_install(pkg, lib = lib, ask = FALSE, upgrade = FALSE)
)
installed_desc <- file.path(lib, pkg, "DESCRIPTION")
ok <- file.exists(installed_desc) &&
any(grepl(sentinel, readLines(installed_desc)))
if (ok) {
cat("PROOF PASSED: pak installed the patched local binary.\n")
quit(status = 0L)
} else {
cat("PROOF FAILED: pak did not install the patched local binary.\n")
quit(status = 1L)
}
- Step 2: Run the proof in a build-env container
Run (amd64 example; use any supported build-env image):
just build-single ubuntu 2604 amd64 4.5.0 glue 1.0.0 1 || true # warms the env
docker run --rm -v "$PWD":/work -w /work reg.devxy.io/rpkgs/build-env-ubuntu:2604 \
Rscript tools/verify-patch-mechanism.R
Expected: final line PROOF PASSED: pak installed the patched local binary. and exit status 0.
- Step 3: Commit
git add tools/verify-patch-mechanism.R
git commit -m "test(patches): prove pak installs a patched binary from a local file:// repo"
- Step 4: Record the outcome / contingency
If the proof PASSED, proceed to Task A1 unchanged.
If it FAILED (pak recompiled or picked CRAN's), the mechanism switches to serving patched source: in Task A4 build_patched_binary() skips pkgbuild::build() and instead repackages the patched source tree with pkgbuild::build(binary = FALSE); everything else (registry, matching, cache, repo prepend) is unchanged because pak build-caches the compiled result via pkgcache. Note the chosen path in the commit message and continue.
Task A1: Registry loading and normalization
Files:
- Create:
R/patches.R - Test:
tests/testthat/test-patches.R
Interfaces:
-
Produces:
load_patch_registry(patches_dir)→list()of normalized entries; each entry is a named list withpackage,versions,platforms(character vector),env(named list),configure_args(character),makevars(named list),reason, andpatch_path(absolute path orNULL).normalize_patch_entry(entry, patches_dir)→ one normalized entry; errors on missing required field or missing patch file. -
Step 1: Write the failing test
# tests/testthat/test-patches.R
test_that("load_patch_registry parses and normalizes entries", {
dir <- withr::local_tempdir()
writeLines("--- a patch ---", file.path(dir, "fix.patch"))
jsonlite::write_json(
list(list(
package = "RcppParallel", versions = "*",
platforms = list("alpine", "ubuntu-2604"),
env = list(RCPP_PARALLEL_USE_TBB = "0"),
patch = "fix.patch", reason = "bundled TBB fails"
)),
file.path(dir, "registry.json"), auto_unbox = TRUE
)
reg <- load_patch_registry(dir)
expect_length(reg, 1L)
expect_identical(reg[[1L]]$package, "RcppParallel")
expect_identical(reg[[1L]]$platforms, c("alpine", "ubuntu-2604"))
expect_identical(reg[[1L]]$env$RCPP_PARALLEL_USE_TBB, "0")
expect_identical(reg[[1L]]$configure_args, character(0L))
expect_true(file.exists(reg[[1L]]$patch_path))
})
test_that("load_patch_registry returns empty list when no registry", {
expect_identical(load_patch_registry(NULL), list())
expect_identical(load_patch_registry(withr::local_tempdir()), list())
})
test_that("normalize_patch_entry errors on missing required field", {
expect_error(
normalize_patch_entry(list(package = "x"), tempdir()),
"missing required field"
)
})
test_that("normalize_patch_entry errors on missing patch file", {
expect_error(
normalize_patch_entry(
list(package = "x", versions = "*", platforms = "alpine",
reason = "r", patch = "nope.patch"),
tempdir()
),
"does not exist"
)
})
- Step 2: Run test to verify it fails
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: FAIL with "could not find function load_patch_registry".
- Step 3: Write minimal implementation
# R/patches.R
#' Load and validate the patch registry
#'
#' Reads `registry.json` from `patches_dir` and returns normalized entries.
#'
#' @param patches_dir Directory containing `registry.json` and any patch files,
#' or `NULL` to disable patching.
#' @return A list of normalized patch entries (possibly empty).
#' @keywords internal
load_patch_registry <- function(patches_dir) {
if (is.null(patches_dir)) {
return(list())
}
registry_file <- file.path(patches_dir, "registry.json")
if (!file.exists(registry_file)) {
log_warn(sprintf(
"Patch directory {.path %s} has no registry.json; patching disabled.",
patches_dir
))
return(list())
}
raw <- jsonlite::fromJSON(registry_file, simplifyVector = FALSE)
lapply(raw, normalize_patch_entry, patches_dir = patches_dir)
}
#' Normalize and validate a single patch registry entry
#'
#' @param entry A list parsed from `registry.json`.
#' @param patches_dir Directory used to resolve a relative `patch` path.
#' @return The entry with defaults filled and `patch_path` resolved.
#' @keywords internal
normalize_patch_entry <- function(entry, patches_dir) {
required <- c("package", "versions", "platforms", "reason")
missing <- setdiff(required, names(entry))
if (length(missing) > 0L) {
stop(
sprintf("Patch entry is missing required field(s): %s", toString(missing)),
call. = FALSE
)
}
entry$platforms <- as.character(unlist(entry$platforms))
entry$env <- if (is.null(entry$env)) list() else entry$env
entry$configure_args <- if (is.null(entry$configure_args)) {
character(0L)
} else {
as.character(unlist(entry$configure_args))
}
entry$makevars <- if (is.null(entry$makevars)) list() else entry$makevars
if (!is.null(entry$patch)) {
patch_path <- file.path(patches_dir, entry$patch)
if (!file.exists(patch_path)) {
stop(
sprintf(
"Patch file '%s' for package '%s' does not exist.",
patch_path, entry$package
),
call. = FALSE
)
}
entry$patch_path <- patch_path
} else {
entry$patch_path <- NULL
}
entry
}
- Step 4: Run test to verify it passes
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: PASS (4 tests).
- Step 5: Commit
git add R/patches.R tests/testthat/test-patches.R
git commit -m "feat(patches): load and validate the patch registry"
Task A2: Platform matching and version-constraint satisfaction
Files:
- Modify:
R/patches.R - Test:
tests/testthat/test-patches.R
Interfaces:
-
Consumes: normalized entries from Task A1.
-
Produces:
build_platform_tokens(platform, arch)→ character vector;entry_matches_platform(entry, tokens)→ logical;match_patch_entries(registry, platform, arch)→ filtered list;version_satisfies(version, constraint)→ logical (constraint forms:"*"handled by caller,"x.y.z"exact,">=x","<=x",">x","<x","==x"). -
Step 1: Write the failing test
test_that("build_platform_tokens expands family and arch", {
expect_setequal(
build_platform_tokens("ubuntu-2604", "amd64"),
c("ubuntu-2604", "ubuntu", "amd64")
)
expect_setequal(
build_platform_tokens("alpine-324", "arm64"),
c("alpine-324", "alpine", "arm64")
)
})
test_that("match_patch_entries matches family, codename, arch, and wildcard", {
reg <- list(
list(package = "A", versions = "*", platforms = "alpine", reason = "r"),
list(package = "B", versions = "*", platforms = "ubuntu-2604", reason = "r"),
list(package = "C", versions = "*", platforms = "*", reason = "r"),
list(package = "D", versions = "*", platforms = "redhat", reason = "r")
)
got <- vapply(
match_patch_entries(reg, "ubuntu-2604", "amd64"),
function(e) e$package, character(1L)
)
expect_setequal(got, c("B", "C"))
})
test_that("version_satisfies handles operators and hyphenated versions", {
expect_true(version_satisfies("5.1.11-2", "5.1.11-2"))
expect_true(version_satisfies("5.1.11-2", "==5.1.11-2"))
expect_true(version_satisfies("5.1.12", ">=5.1.0"))
expect_false(version_satisfies("5.0.0", ">=5.1.0"))
expect_true(version_satisfies("5.1.11-2", "<=5.1.11-2"))
expect_false(version_satisfies("5.1.12", "<=5.1.11-2"))
})
- Step 2: Run test to verify it fails
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: FAIL with "could not find function build_platform_tokens".
- Step 3: Write minimal implementation (append to
R/patches.R)
#' Build platform tokens for patch matching
#' @keywords internal
build_platform_tokens <- function(platform, arch) {
family <- sub("-.*$", "", platform)
unique(c(platform, family, arch))
}
#' Does a patch entry apply to the current platform tokens?
#' @keywords internal
entry_matches_platform <- function(entry, tokens) {
any(entry$platforms == "*") ||
length(intersect(entry$platforms, tokens)) > 0L
}
#' Filter registry entries applicable to the current build
#' @keywords internal
match_patch_entries <- function(registry, platform, arch) {
if (length(registry) == 0L) {
return(list())
}
tokens <- build_platform_tokens(platform, arch)
Filter(function(e) entry_matches_platform(e, tokens), registry)
}
#' Test whether a version satisfies a single constraint
#'
#' @param version A version string (CRAN style, may contain `-`).
#' @param constraint One of `"x.y.z"`, `"==x"`, `">=x"`, `"<=x"`, `">x"`, `"<x"`.
#' @keywords internal
version_satisfies <- function(version, constraint) {
constraint <- trimws(constraint)
parts <- regmatches(
constraint,
regexec("^(>=|<=|==|>|<)?\\s*(.+)$", constraint)
)[[1L]]
op <- parts[2L]
target <- parts[3L]
v <- package_version(version)
t <- package_version(target)
if (op == "" || op == "==") {
return(v == t)
}
switch(
op,
">=" = v >= t,
"<=" = v <= t,
">" = v > t,
"<" = v < t,
FALSE
)
}
- Step 4: Run test to verify it passes
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: PASS.
- Step 5: Commit
git add R/patches.R tests/testthat/test-patches.R
git commit -m "feat(patches): platform matching and version-constraint checks"
Task A3: Cache key and version resolution
Files:
- Modify:
R/patches.R - Test:
tests/testthat/test-patches.R
Interfaces:
-
Consumes: normalized entries.
-
Produces:
patch_cache_key(entry, version, platform, arch, r_minor)→ string"<pkg>_<ver>_<platform>_<arch>_<rminor>_<hash12>", wherehash12coversenv/configure_args/makevars/patch bytes;resolve_patch_version(entry)→ latest CRAN version satisfyingentry$versions, orNA_character_;describe_patch(entry)→ short human label. -
Step 1: Write the failing test
test_that("patch_cache_key is stable and sensitive to env/patch changes", {
e1 <- list(package = "P", env = list(A = "1"),
configure_args = character(0L), makevars = list(),
patch_path = NULL)
e2 <- e1; e2$env <- list(A = "2")
k1 <- patch_cache_key(e1, "1.0", "alpine-324", "amd64", "4.5")
expect_identical(k1, patch_cache_key(e1, "1.0", "alpine-324", "amd64", "4.5"))
expect_false(identical(
k1, patch_cache_key(e2, "1.0", "alpine-324", "amd64", "4.5")
))
expect_match(k1, "^P_1.0_alpine-324_amd64_4.5_[0-9a-f]{12}$")
})
test_that("resolve_patch_version returns latest for wildcard, NA when unmet", {
local_mocked_bindings(
cran_package = function(pkg) list(Version = "5.1.12"),
.package = "pkgsearch"
)
expect_identical(
resolve_patch_version(list(package = "RcppParallel", versions = "*")),
"5.1.12"
)
expect_identical(
resolve_patch_version(list(package = "RcppParallel", versions = ">=9.0")),
NA_character_
)
})
test_that("describe_patch summarizes the active overrides", {
expect_match(
describe_patch(list(env = list(RCPP_PARALLEL_USE_TBB = "0"),
configure_args = character(0L), makevars = list(),
patch_path = NULL)),
"env: RCPP_PARALLEL_USE_TBB=0"
)
expect_match(
describe_patch(list(env = list(), configure_args = character(0L),
makevars = list(), patch_path = "/x/fix.patch")),
"source patch"
)
})
- Step 2: Run test to verify it fails
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: FAIL with "could not find function patch_cache_key".
- Step 3: Write minimal implementation (append to
R/patches.R)
#' Compute the cache key for a patched binary
#' @keywords internal
patch_cache_key <- function(entry, version, platform, arch, r_minor) {
payload <- list(
env = entry$env,
configure_args = entry$configure_args,
makevars = entry$makevars,
patch = if (!is.null(entry$patch_path)) {
readBin(entry$patch_path, "raw", file.size(entry$patch_path))
} else {
raw(0L)
}
)
tmp <- tempfile()
on.exit(unlink(tmp), add = TRUE)
saveRDS(payload, tmp)
hash <- substr(unname(tools::md5sum(tmp)), 1L, 12L)
sprintf(
"%s_%s_%s_%s_%s_%s",
entry$package, version, platform, arch, r_minor, hash
)
}
#' Resolve the CRAN version to build for a patch entry
#'
#' Returns the latest CRAN version satisfying the entry's `versions` constraint,
#' or `NA_character_` when CRAN's latest does not satisfy it or lookup fails.
#' @keywords internal
resolve_patch_version <- function(entry) {
latest <- tryCatch(
pkgsearch::cran_package(entry$package)$Version,
error = function(e) NA_character_
)
if (is.na(latest)) {
return(NA_character_)
}
if (identical(entry$versions, "*") || version_satisfies(latest, entry$versions)) {
return(latest)
}
NA_character_
}
#' Short human label describing a patch entry's overrides
#' @keywords internal
describe_patch <- function(entry) {
bits <- character(0L)
if (length(entry$env) > 0L) {
bits <- c(bits, sprintf(
"env: %s",
paste(
names(entry$env),
unlist(entry$env),
sep = "=", collapse = ","
)
))
}
if (length(entry$configure_args) > 0L) {
bits <- c(bits, sprintf("configure: %s", toString(entry$configure_args)))
}
if (length(entry$makevars) > 0L) {
bits <- c(bits, "makevars")
}
if (!is.null(entry$patch_path)) {
bits <- c(bits, "source patch")
}
if (length(bits) == 0L) "no-op" else paste(bits, collapse = "; ")
}
- Step 4: Run test to verify it passes
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: PASS. (Note: local_mocked_bindings requires testthat >= 3.1.7; bincraft uses 3e.)
- Step 5: Commit
git add R/patches.R tests/testthat/test-patches.R
git commit -m "feat(patches): cache key, version resolution, and patch description"
Task A4: Build a patched binary in isolation
Files:
- Modify:
R/patches.R - Test:
tests/testthat/test-patches.R
Interfaces:
-
Consumes: a normalized entry, a resolved
version, adest_dir. -
Produces:
download_cran_source(package, version, dest_dir, cran)→ path orNULL;apply_source_patch(patch_path, pkg_src)→ logical;configure_args_to_build_args(configure_args)→ character;build_patched_binary(entry, version, dest_dir)→ path to built binary tarball orNULL. -
Step 1: Write the failing test
test_that("configure_args_to_build_args formats configure args", {
expect_identical(configure_args_to_build_args(character(0L)), character(0L))
expect_identical(
configure_args_to_build_args(c("--with-foo", "--no-bar")),
"--configure-args=--with-foo --no-bar"
)
})
test_that("apply_source_patch returns FALSE when patch does not apply", {
src <- withr::local_tempdir()
writeLines("unrelated content", file.path(src, "file.txt"))
bad_patch <- tempfile(fileext = ".patch")
writeLines(c(
"--- a/missing.txt", "+++ b/missing.txt",
"@@ -1 +1 @@", "-nope", "+nope2"
), bad_patch)
expect_false(apply_source_patch(bad_patch, src))
})
test_that("build_patched_binary returns NULL when download fails", {
local_mocked_bindings(download_cran_source = function(...) NULL)
expect_null(
build_patched_binary(
list(package = "P", env = list(), configure_args = character(0L),
makevars = list(), patch_path = NULL),
"1.0", withr::local_tempdir()
)
)
})
- Step 2: Run test to verify it fails
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: FAIL with "could not find function configure_args_to_build_args".
- Step 3: Write minimal implementation (append to
R/patches.R)
#' Download a CRAN source tarball for an exact version
#' @keywords internal
download_cran_source <- function(
package,
version,
dest_dir,
cran = "https://cloud.r-project.org"
) {
fname <- sprintf("%s_%s.tar.gz", package, version)
urls <- c(
sprintf("%s/src/contrib/%s", cran, fname),
sprintf("%s/src/contrib/Archive/%s/%s", cran, package, fname)
)
dest <- file.path(dest_dir, fname)
for (u in urls) {
ok <- tryCatch(
{
utils::download.file(u, dest, mode = "wb", quiet = TRUE)
file.exists(dest) && file.size(dest) > 0L
},
error = function(e) FALSE
)
if (isTRUE(ok)) {
return(dest)
}
}
log_warn(sprintf(
"Could not download CRAN source for {.pkg %s} %s.",
package, version
))
NULL
}
#' Apply a unified diff to an unpacked source tree
#'
#' Uses `patch -p1 --forward` so an already-applied or non-applying patch fails
#' cleanly (returns FALSE) instead of corrupting the tree.
#' @keywords internal
apply_source_patch <- function(patch_path, pkg_src) {
status <- system2(
"patch",
args = c(
"-p1", "--forward", "--batch",
"-d", shQuote(pkg_src),
"-i", shQuote(patch_path)
),
stdout = FALSE, stderr = FALSE
)
identical(status, 0L)
}
#' Format configure args for `pkgbuild::build(args = ...)`
#' @keywords internal
configure_args_to_build_args <- function(configure_args) {
if (length(configure_args) == 0L) {
return(character(0L))
}
sprintf("--configure-args=%s", paste(configure_args, collapse = " "))
}
#' Build a patched binary for one registry entry, in isolation
#'
#' Downloads CRAN source for `version`, applies the source patch (if any), and
#' builds a binary with the entry's env / configure / Makevars overrides scoped
#' to this build only. Returns the built tarball path, or `NULL` on any failure.
#' @keywords internal
build_patched_binary <- function(entry, version, dest_dir) {
workdir <- tempfile("patch_build_")
dir.create(workdir, recursive = TRUE, showWarnings = FALSE)
on.exit(unlink(workdir, recursive = TRUE, force = TRUE), add = TRUE)
src_tarball <- download_cran_source(entry$package, version, workdir)
if (is.null(src_tarball)) {
return(NULL)
}
utils::untar(src_tarball, exdir = workdir)
pkg_src <- file.path(workdir, entry$package)
if (!is.null(entry$patch_path)) {
if (!apply_source_patch(entry$patch_path, pkg_src)) {
log_warn(sprintf(
"Patch for {.pkg %s} %s did not apply cleanly; skipping patched build.",
entry$package, version
))
return(NULL)
}
}
build_env <- entry$env
if (length(entry$makevars) > 0L) {
mk <- tempfile(fileext = ".mk")
writeLines(
vapply(
names(entry$makevars),
function(k) sprintf("%s=%s", k, entry$makevars[[k]]),
character(1L)
),
mk
)
build_env$R_MAKEVARS_USER <- mk
}
tryCatch(
withr::with_envvar(build_env, {
pkgbuild::build(
path = pkg_src,
binary = TRUE,
vignettes = FALSE,
dest_path = dest_dir,
args = configure_args_to_build_args(entry$configure_args),
quiet = TRUE
)
}),
error = function(e) {
log_warn(sprintf(
"Isolated patched build of {.pkg %s} %s failed: %s",
entry$package, version, conditionMessage(e)
))
NULL
}
)
}
- Step 4: Run test to verify it passes
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: PASS.
- Step 5: Commit
git add R/patches.R tests/testthat/test-patches.R
git commit -m "feat(patches): build a patched binary in isolation from CRAN source"
Task A5: Orchestrate the local patched repo (cache + index)
Files:
- Modify:
R/patches.R - Test:
tests/testthat/test-patches.R
Interfaces:
-
Consumes: all helpers above.
-
Produces:
prepare_patched_repo(patches_dir, platform, arch, r_minor, cache_dir, repo_dir)→ path to asrc/contrib-style dir containing patched binaries + aPACKAGESindex, orNULLwhen nothing matched/built. On a cache hit it copies the cached tarball intorepo_dir; on a miss it builds, then writes the result intocache_dir. -
Step 1: Write the failing test
test_that("prepare_patched_repo returns NULL when no entries match", {
dir <- withr::local_tempdir()
jsonlite::write_json(
list(list(package = "A", versions = "*", platforms = list("redhat"),
reason = "r")),
file.path(dir, "registry.json"), auto_unbox = TRUE
)
expect_null(
prepare_patched_repo(dir, "ubuntu-2604", "amd64", "4.5",
cache_dir = withr::local_tempdir(),
repo_dir = withr::local_tempdir())
)
})
test_that("prepare_patched_repo serves a cached binary and writes an index", {
dir <- withr::local_tempdir()
jsonlite::write_json(
list(list(package = "glue", versions = "*", platforms = list("*"),
env = list(A = "1"), reason = "r")),
file.path(dir, "registry.json"), auto_unbox = TRUE
)
cache <- withr::local_tempdir()
repo <- withr::local_tempdir()
local_mocked_bindings(
resolve_patch_version = function(entry) "1.0.0",
build_patched_binary = function(entry, version, dest_dir) {
f <- file.path(dest_dir, sprintf("%s_%s.tar.gz", entry$package, version))
writeLines("fake binary", f)
f
}
)
out <- prepare_patched_repo(dir, "ubuntu-2604", "amd64", "4.5",
cache_dir = cache, repo_dir = repo)
expect_identical(out, repo)
expect_true(file.exists(file.path(repo, "glue_1.0.0.tar.gz")))
expect_true(file.exists(file.path(repo, "PACKAGES")))
# The build result was cached under the key.
expect_length(list.files(cache, pattern = "^glue_1.0.0_.*\\.tar\\.gz$"), 1L)
})
- Step 2: Run test to verify it fails
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: FAIL with "could not find function prepare_patched_repo".
- Step 3: Write minimal implementation (append to
R/patches.R)
#' Prepare a local repo of patched binaries for the current build
#'
#' For each registry entry matching the current platform, ensures a patched
#' binary is present in `repo_dir` (from `cache_dir` if available, else built
#' and then cached) and writes a `PACKAGES` index over them.
#'
#' @param patches_dir Directory with `registry.json`, or `NULL`.
#' @param platform Build platform, e.g. `"ubuntu-2604"`.
#' @param arch Build arch, e.g. `"amd64"`.
#' @param r_minor R `"major.minor"` string, e.g. `"4.5"`.
#' @param cache_dir Persistent cache for patched binaries.
#' @param repo_dir Directory to assemble the local repo in.
#' @return `repo_dir` if at least one patched binary was produced, else `NULL`.
#' @keywords internal
prepare_patched_repo <- function(
patches_dir,
platform,
arch,
r_minor,
cache_dir = file.path("/mnt", "cache", "patched-binaries"),
repo_dir = tempfile("patched_repo_")
) {
entries <- match_patch_entries(
load_patch_registry(patches_dir), platform, arch
)
if (length(entries) == 0L) {
return(NULL)
}
dir.create(repo_dir, recursive = TRUE, showWarnings = FALSE)
dir.create(cache_dir, recursive = TRUE, showWarnings = FALSE)
produced <- 0L
for (entry in entries) {
version <- resolve_patch_version(entry)
if (is.na(version)) {
log_warn(sprintf(
"No CRAN version of {.pkg %s} satisfies '%s'; patch skipped.",
entry$package, entry$versions
))
next
}
key <- patch_cache_key(entry, version, platform, arch, r_minor)
cached <- file.path(cache_dir, sprintf("%s.tar.gz", key))
target <- file.path(
repo_dir, sprintf("%s_%s.tar.gz", entry$package, version)
)
if (file.exists(cached)) {
log_info(sprintf(
"Using cached patched binary for {.pkg %s} %s.",
entry$package, version
))
file.copy(cached, target, overwrite = TRUE)
} else {
log_info(sprintf(
"Applying patch to {.pkg %s} %s [%s]: %s",
entry$package, version, describe_patch(entry), entry$reason
))
built <- build_patched_binary(entry, version, repo_dir)
if (is.null(built)) {
next
}
if (!identical(normalizePath(built), normalizePath(target))) {
file.copy(built, target, overwrite = TRUE)
}
file.copy(target, cached, overwrite = TRUE)
}
produced <- produced + 1L
}
if (produced == 0L) {
return(NULL)
}
cranlike::add_PACKAGES(
list.files(repo_dir, pattern = "\\.tar\\.gz$"),
repo_dir
)
repo_dir
}
- Step 4: Run test to verify it passes
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: PASS.
- Step 5: Update imports and document
Add to DESCRIPTION Imports: (alphabetical) jsonlite if not present, and ensure pkgsearch, pkgbuild, cranlike, withr are listed (they are). Then:
Rscript -e 'devtools::document()'
git add R/patches.R tests/testthat/test-patches.R DESCRIPTION NAMESPACE
git commit -m "feat(patches): orchestrate local patched-binary repo with caching"
Task A6: Wire patched repo into the pak install path
Files:
- Modify:
R/install_helpers.R:333-389(run_pak_install_with_mutex) - Modify:
R/install-deps.R:23-75(install_pkg_sys_deps) - Test:
tests/testthat/test-patches.R
Interfaces:
-
Consumes:
prepare_patched_repo(). -
Produces:
run_pak_install_with_mutex(local_clone_dir_single, env_vars, patched_repo = NULL)— prependsfile://<patched_repo>tooptions("repos")for the install;install_pkg_sys_deps(package_name, tag, local_clone_dir, platform, aggressive_cleanup = FALSE, patches = NULL, arch = NULL)— builds the patched repo before installing. -
Step 1: Write the failing test
test_that("run_pak_install_with_mutex prepends the patched repo to repos", {
seen <- NULL
local_mocked_bindings(
acquire_pak_mutex = function(...) tempfile(),
release_pak_mutex = function(...) invisible(NULL),
retry_with_backoff = function(func, ...) func()
)
local_mocked_bindings(
local_install_deps = function(...) {
seen <<- getOption("repos")
invisible(TRUE)
},
.package = "pak"
)
run_pak_install_with_mutex(
tempfile(), list(), patched_repo = "/tmp/patched"
)
expect_true(any(grepl("file:///tmp/patched", seen)))
})
- Step 2: Run test to verify it fails
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: FAIL — patched_repo argument not yet accepted / repos not prepended.
- Step 3: Edit
run_pak_install_with_mutexinR/install_helpers.R
Change the signature line:
run_pak_install_with_mutex <- function(local_clone_dir_single, env_vars) {
to:
run_pak_install_with_mutex <- function(
local_clone_dir_single,
env_vars,
patched_repo = NULL
) {
Replace the inner retry_with_backoff(...) block (the one wrapping pak::local_install_deps) with:
retry_with_backoff(function() {
withr::with_envvar(env_vars, {
repos <- getOption("repos")
if (!is.null(patched_repo)) {
repos <- c(
patched = sprintf("file://%s", patched_repo),
repos
)
}
withr::with_options(list(repos = repos), {
# Default to non-verbose (suppressed messages)
suppressMessages(pak::local_install_deps(sprintf(
"%s",
local_clone_dir_single
)))
})
})
})
- Step 4: Edit
install_pkg_sys_depsinR/install-deps.R
Change the signature to add patches and arch:
install_pkg_sys_deps <- function(
package_name,
tag,
local_clone_dir,
platform = platform,
aggressive_cleanup = FALSE,
patches = NULL,
arch = NULL
) {
Immediately before the run_pak_install_with_mutex(...) call, insert:
# Build a local repo of patched binaries (if any apply) and serve it to pak.
r_minor <- paste(
R.version$major,
strsplit(R.version$minor, ".", fixed = TRUE)[[1L]][1L],
sep = "."
)
patched_repo <- tryCatch(
prepare_patched_repo(patches, platform, arch, r_minor),
error = function(e) {
log_warn(sprintf("Patch preparation failed: %s", conditionMessage(e)))
NULL
}
)
and change the call from:
run_pak_install_with_mutex(
local_clone_dir_single,
env_vars
)
to:
run_pak_install_with_mutex(
local_clone_dir_single,
env_vars,
patched_repo = patched_repo
)
- Step 5: Run test to verify it passes
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: PASS.
- Step 6: Commit
git add R/install_helpers.R R/install-deps.R tests/testthat/test-patches.R
git commit -m "feat(patches): serve patched binaries to pak during dep install"
Task A7: Thread patches through the public build API
Files:
- Modify:
R/build_binaries.R(build_binary_package,execute_package_builds,build_single_tag,handle_system_dependencies) - Create:
man-roxygen/param-patches.R - Test:
tests/testthat/test-patches.R
Interfaces:
-
Produces:
build_binary_package(..., patches = NULL)and the internal chain each carrypatchesdown toinstall_pkg_sys_deps().handle_system_dependencies(..., patches = NULL)passespatchesandarchthrough. -
Step 1: Write the failing test
test_that("handle_system_dependencies forwards patches and arch", {
captured <- list()
local_mocked_bindings(
install_pkg_sys_deps = function(package_name, tag, local_clone_dir_single,
platform, patches = NULL, arch = NULL) {
captured <<- list(patches = patches, arch = arch)
invisible(TRUE)
}
)
handle_system_dependencies(
"RcppParallel", "5.1.11-2", "ubuntu-2604", tempfile(), "amd64",
NULL, NULL, NULL, NULL, NULL, NULL, NULL,
patches = "local/patches"
)
expect_identical(captured$patches, "local/patches")
expect_identical(captured$arch, "amd64")
})
- Step 2: Run test to verify it fails
Run: Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: FAIL — handle_system_dependencies has no patches argument.
- Step 3: Create the roxygen template
# man-roxygen/param-patches.R
#' @param patches Optional path to a patch registry directory containing a
#' `registry.json` (and any referenced diff files). When set, matching
#' packages are pre-built as patched binaries and served to `pak` during
#' dependency installation. Defaults to `NULL` (no patching).
- Step 4: Edit the four functions in
R/build_binaries.R
In build_single_tag()'s call to handle_system_dependencies(...), add patches = patches as the final argument, and add patches = NULL to build_single_tag's own signature plus #' @template param-patches to its roxygen block.
Change handle_system_dependencies signature to end with metadata_db_sslmode, then add patches = NULL, and change its inner install_pkg_sys_deps(...) call from:
install_pkg_sys_deps(
package_name,
tag,
local_clone_dir_single,
platform
)
to:
install_pkg_sys_deps(
package_name,
tag,
local_clone_dir_single,
platform,
patches = patches,
arch = arch
)
In execute_package_builds(), add patches = NULL to the signature and pass patches = patches into its build_single_tag(...) call inside worker_function.
In build_binary_package(), add patches = NULL to the signature (after s3_package_cache), add #' @template param-patches to its roxygen, and pass patches = patches into the execute_package_builds(...) call.
- Step 5: Document, test, and run package check
Run:
Rscript -e 'devtools::document()'
Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: PASS for the new test; man/build_binary_package.Rd etc. regenerated.
- Step 6: Commit
git add R/build_binaries.R man-roxygen/param-patches.R man NAMESPACE tests/testthat/test-patches.R
git commit -m "feat(patches): thread patches argument through build_binary_package"
Task A8: End-to-end patch test (guarded) and version bump
Files:
- Modify:
tests/testthat/test-patches.R - Modify:
DESCRIPTION(version),NEWS.md
Interfaces:
-
Produces: a guarded e2e test proving a dependent package builds when its failing dependency is patched.
-
Step 1: Add the guarded e2e test
test_that("a patched dependency unblocks a dependent build (e2e)", {
skip_if_not(nzchar(Sys.getenv("BINCRAFT_PATCH_E2E")))
skip_if_offline()
patches_dir <- withr::local_tempdir()
jsonlite::write_json(
list(list(
package = "RcppParallel", versions = "*", platforms = list("*"),
env = list(RCPP_PARALLEL_USE_TBB = "0"),
reason = "bundled TBB fails on this toolchain"
)),
file.path(patches_dir, "registry.json"), auto_unbox = TRUE
)
out <- withr::local_tempdir()
result <- build_binary_package(
"rts2", tag = "latest", local_output_dir_root = out,
upload = FALSE, archive = FALSE, patches = patches_dir
)
expect_true(isTRUE(result) || identical(result, "skipped"))
})
- Step 2: Run the e2e test in a container
Run:
docker run --rm -e BINCRAFT_PATCH_E2E=1 -v "$PWD":/work -w /work \
reg.devxy.io/rpkgs/build-env-ubuntu:2604 \
Rscript -e 'devtools::load_all("."); testthat::test_file("tests/testthat/test-patches.R")'
Expected: the e2e test runs (not skipped) and PASSES; build log shows Applying patch to RcppParallel.
- Step 3: Bump version and changelog
In DESCRIPTION, bump Version: to 4.3.0.9999. Prepend to NEWS.md:
# bincraft 4.3.0
* `build_binary_package()` gains a `patches` argument: a registry of
per-package env / configure / Makevars overrides and source diffs that are
pre-built into patched binaries and served to pak, fixing compiler- and
OS-specific failures (e.g. RcppParallel) including for transitive deps.
- Step 4: Commit
git add tests/testthat/test-patches.R DESCRIPTION NEWS.md
git commit -m "test(patches): guarded end-to-end test; bump to 4.3.0.9999"
Phase B — build-cran-binaries registry and wiring
All Phase B paths are relative to this repo (build-cran-binaries). Phase B depends on a bincraft build that includes Phase A (install the updated bincraft in the build containers, or bump the pinned version in .crow/build-all-versions-install-deps.yaml).
Task B1: Create the patch registry with the RcppParallel entry
Files:
- Create:
local/patches/registry.json - Create:
local/patches/README.md
Interfaces:
-
Produces: the curated registry consumed by bincraft's
patchesargument. -
Step 1: Write the registry
[
{
"package": "RcppParallel",
"versions": "*",
"platforms": ["alpine", "ubuntu-2604"],
"env": { "RCPP_PARALLEL_USE_TBB": "0" },
"configure_args": [],
"makevars": {},
"patch": null,
"reason": "bundled Intel TBB fails to build on musl and on newer toolchains (e.g. g++ 15 on ubuntu-2604); disabling TBB falls back to TinyThread"
}
]
- Step 2: Write the README
local/patches/README.md documents the schema (copy the field table from specs/2026-06-30-package-patching-design.md), how to add an entry, and that source diffs go in local/patches/<pkg>/<file>.patch referenced by the patch field.
- Step 3: Verify it parses
Run: Rscript -e 'x <- jsonlite::fromJSON("local/patches/registry.json", simplifyVector = FALSE); stopifnot(length(x) == 1L, x[[1]]$package == "RcppParallel"); cat("ok\n")'
Expected: ok.
- Step 4: Commit
git add local/patches/registry.json local/patches/README.md
git commit -m "feat(patches): add patch registry with RcppParallel TBB workaround"
Task B2: Registry validator script
Files:
- Create:
local/validate-patches.R
Interfaces:
-
Consumes:
local/patches/registry.json. -
Produces: a script that exits non-zero on schema violations, missing patch files, or ambiguous overlapping entries.
-
Step 1: Write the validator
#!/usr/bin/env Rscript
# Validate local/patches/registry.json: schema, referenced patch files, and
# ambiguous overlaps. Exits 1 on any problem. Used by pre-commit and CI.
dir <- "local/patches"
registry_file <- file.path(dir, "registry.json")
if (!file.exists(registry_file)) {
cat("No registry.json found; nothing to validate.\n")
quit(status = 0L)
}
reg <- jsonlite::fromJSON(registry_file, simplifyVector = FALSE)
required <- c("package", "versions", "platforms", "reason")
errs <- character(0L)
for (i in seq_along(reg)) {
e <- reg[[i]]
missing <- setdiff(required, names(e))
if (length(missing) > 0L) {
errs <- c(errs, sprintf(
"entry %d (%s): missing %s", i,
if (is.null(e$package)) "?" else e$package, toString(missing)
))
}
if (!is.null(e$patch)) {
p <- file.path(dir, e$patch)
if (!file.exists(p)) {
errs <- c(errs, sprintf("entry %d (%s): patch file '%s' missing",
i, e$package, p))
}
}
}
# Ambiguous overlap: two entries for the same package with identical platforms
# and versions.
keys <- vapply(reg, function(e) {
sprintf("%s|%s|%s", e$package,
paste(sort(as.character(unlist(e$platforms))), collapse = ","),
e$versions)
}, character(1L))
dups <- keys[duplicated(keys)]
if (length(dups) > 0L) {
errs <- c(errs, sprintf("ambiguous duplicate entries: %s", toString(unique(dups))))
}
if (length(errs) > 0L) {
cat("Patch registry validation FAILED:\n")
cat(paste0(" - ", errs, "\n"))
quit(status = 1L)
}
cat(sprintf("Patch registry OK (%d entrie(s)).\n", length(reg)))
- Step 2: Run it (expect success on the B1 registry)
Run: Rscript local/validate-patches.R
Expected: Patch registry OK (1 entrie(s)). and exit 0.
- Step 3: Run it against a broken registry (expect failure)
Run:
cp local/patches/registry.json /tmp/reg.bak
Rscript -e 'writeLines("[{\"package\":\"X\"}]", "local/patches/registry.json")'
Rscript local/validate-patches.R; echo "exit=$?"
cp /tmp/reg.bak local/patches/registry.json
Expected: prints validation FAILED with a missing-field message and exit=1.
- Step 4: Commit
git add local/validate-patches.R
git commit -m "feat(patches): add registry validator script"
Task B3: Hook the validator into pre-commit
Files:
- Modify:
.pre-commit-config.yaml
Interfaces:
-
Produces: a local hook that runs
local/validate-patches.Rwhen the registry or patch files change. -
Step 1: Add the hook
Add a repo: local hook entry to .pre-commit-config.yaml:
- repo: local
hooks:
- id: validate-patches
name: validate patch registry
entry: Rscript local/validate-patches.R
language: system
files: ^local/patches/
pass_filenames: false
- Step 2: Verify the hook runs
Run: pre-commit run validate-patches --all-files
Expected: hook passes (Patch registry OK).
- Step 3: Commit
git add .pre-commit-config.yaml
git commit -m "ci(patches): validate patch registry in pre-commit"
Task B4: Pass patches through the build entry points
Files:
- Modify:
local/build-one.R:99-119(thebuild_binary_packagecall) - Modify:
local/build-all.R:125-145(thebuild_binary_packagecall)
Interfaces:
-
Consumes: the new bincraft
patchesargument (Phase A) andlocal/patches/. -
Produces: both entry points pass
patches = "local/patches". -
Step 1: Edit
local/build-one.R
In the bincraft::build_binary_package( call, add as a new argument (e.g. after archive = TRUE,):
patches = "local/patches",
- Step 2: Edit
local/build-all.R
In the bincraft::build_binary_package( call, add:
patches = "local/patches",
- Step 3: Verify the scripts still parse
Run: Rscript -e 'invisible(parse("local/build-one.R")); invisible(parse("local/build-all.R")); cat("parse ok\n")'
Expected: parse ok.
- Step 4: Commit
git add local/build-one.R local/build-all.R
git commit -m "feat(patches): pass patch registry to bincraft build calls"
Task B5: Document the feature in the README
Files:
- Modify:
README.md
Interfaces:
-
Produces: a short "Patching packages" section explaining the registry and linking the design spec.
-
Step 1: Add a README section
Add a ## Patching packages section after the "Build Process" section describing: why patching exists (compiler/OS-specific failures cascading via shared deps like RcppParallel), where the registry lives (local/patches/registry.json), the two tiers (env/configure/Makevars overrides vs source diffs), and that bincraft pre-builds patched binaries served to pak. Link specs/2026-06-30-package-patching-design.md.
- Step 2: Commit
git add README.md
git commit -m "docs(patches): document the package patching workflow"
Self-Review
Spec coverage:
- Registry in this repo, passed to bincraft → Task A7 (
patchesarg), B1 (registry), B4 (wiring). ✓ - Mechanism in bincraft (resolve → pre-build → prepend repo → install) → A4–A6. ✓
- Both tiers, env-first → A4 (
env,configure_args,makevars,patchall applied; env is the cheap default and the RcppParallel entry uses only env). ✓ - Pre-built patched binary served from a prepended local repo → A0 proof + A5/A6. ✓
- Cache keyed by pkg/version/platform/arch/rminor/patchhash → A3 (
patch_cache_key) + A5 (cache use). ✓ - S3 upload of patched binaries → partial: A5 caches locally only. Cross-machine S3 reuse is deferred (see note below) to keep the first cut shippable; local
/mnt/cachereuse already removes per-dependent rebuilds within a container. Recorded as out-of-first-cut, not dropped. - Error handling (diff fails → skip+warn; build fails → skip+warn; no match → skip; ambiguous overlap → validation error) → A4 (
apply_source_patchFALSE path, build tryCatch), A5 (resolve_patch_versionNA path), B2 (overlap validation). ✓ - Observability (log line per applied patch) → A5 (
log_infowithdescribe_patch). DB metadata recording of applied patches is deferred with S3 (same note). - Testing (registry parse/match, cache key, integration, failure path) → A1–A5 unit tests, A8 e2e, A4 patch-fail test. ✓
Deferred from spec (call out to user): S3 upload/reuse of patched binaries and recording applied patches in the Postgres build-metadata row. The local /mnt/cache/patched-binaries cache already prevents repeated rebuilds within a container; S3 reuse across CI jobs is a follow-up. If you want it in the first cut, add a Task A5b (upload cached tarball to a …/patched/ S3 slot and check there before building) and a metadata column — say so and I'll insert them.
Placeholder scan: No TBD/TODO; every code step shows complete code. README/registry-README prose steps describe exact content to write (acceptable for docs).
Type consistency: prepare_patched_repo signature is identical across A5 (definition) and A6 (call site, via defaults). patch_cache_key(entry, version, platform, arch, r_minor) argument order matches between A3 and A5. run_pak_install_with_mutex(..., patched_repo) matches between A6 definition and the A6 test. handle_system_dependencies(..., patches) matches between A7 edit and A7 test. build_patched_binary(entry, version, dest_dir) matches A4 and A5.
Open risk: Task A0 gates the whole approach; if it fails, the documented source-serving contingency keeps every later task valid with a localized change in A4.