fix(abi): inspect flat objects of packages CRAN has removed
The backfill decided which objects were worth inspecting by looking the package up in CRAN's current index. CRAN lists only what it currently ships, so a package it has archived is absent, is never recognised as compiled, and is never inspected. No verdict is ever recorded for it. `union_index_records()` meanwhile reads the slot's *own* `NeedsCompilation`, finds `yes`, and drops the record when the source classifier flags the package and no safe verdict exists to clear it. The repair cannot rescue it either, because it works from recorded verdicts and there is no row. The record is therefore dropped from every per-minor index on every re-index, permanently. `mixedMem` and `vcpen`, both removed from CRAN, were stuck out of all three rhel8 per-minor indexes on both architectures that way, surviving a backfill, two re-indexes and a repair. Read `NeedsCompilation` from the slot's own flat index instead, which describes exactly the objects being inspected. It is fetched from the bucket rather than over HTTPS because this runs inside R, where the edge would route the request to a per-minor index. Keyed on `<package>_<version>` rather than the package name: a slot can carry several versions and only some of them compile.
This commit is contained in:
parent
03e007069a
commit
767dd9ee5d
10 changed files with 133 additions and 60 deletions
|
|
@ -8,8 +8,8 @@ s3fs::s3_file_system(
|
|||
refresh = T
|
||||
)
|
||||
|
||||
arch = "arm64"
|
||||
os = "alpine321"
|
||||
arch <- "arm64"
|
||||
os <- "alpine321"
|
||||
|
||||
files <- s3fs::s3_dir_ls(sprintf(
|
||||
"devxy-r-package-binaries-hel1/%s/%s/latest/src/contrib",
|
||||
|
|
@ -17,11 +17,11 @@ files <- s3fs::s3_dir_ls(sprintf(
|
|||
os
|
||||
))
|
||||
length(unique(sapply(strsplit(basename(files), "_"), function(x) x[1])))
|
||||
non_archived = unique(sapply(strsplit(basename(files), "_"), function(x) {
|
||||
non_archived <- unique(sapply(strsplit(basename(files), "_"), function(x) {
|
||||
x[1]
|
||||
})[duplicated(sapply(strsplit(basename(files), "_"), function(x) x[1]))])
|
||||
# RInno is not built because it only exists for Windows
|
||||
non_archived = setdiff(non_archived, "RInno")
|
||||
non_archived <- setdiff(non_archived, "RInno")
|
||||
# future::plan("sequential", workers = 8)
|
||||
future::plan("sequential", workers = 1)
|
||||
# future.apply::future_lapply(non_archived, function(x) archive_package(x, codename = os, arch = arch))
|
||||
|
|
|
|||
|
|
@ -75,19 +75,47 @@ flat_pkg <- sub("_.*$", "", flat_files)
|
|||
flat_ver <- sub("\\.tar\\.gz$", "", sub("^[^_]*_", "", flat_files))
|
||||
|
||||
# Only compiled packages can fail at dyn.load(), so the rest are not worth
|
||||
# downloading. CRAN's own index says which; it is not behind our edge.
|
||||
cran <- read.dcf(
|
||||
gzcon(url(
|
||||
"https://cloud.r-project.org/src/contrib/PACKAGES.gz",
|
||||
open = "rb"
|
||||
)),
|
||||
fields = c("Package", "NeedsCompilation")
|
||||
# downloading. Ask the slot's own index rather than CRAN's: CRAN lists only the
|
||||
# versions it currently ships, so a package it has archived is absent, is never
|
||||
# recognised as compiled, and is never inspected. No verdict is ever recorded
|
||||
# for it, while `union_index_records()` reads the slot's own `NeedsCompilation`
|
||||
# and happily drops it -- with nothing able to clear the flag, because the
|
||||
# repair works from recorded verdicts too. `mixedMem` and `vcpen`, both removed
|
||||
# from CRAN, were stuck out of every rhel8 per-minor index that way.
|
||||
#
|
||||
# Read it from the bucket, not over HTTPS: this runs inside R, so the edge would
|
||||
# route the request to a per-minor index and the flat objects would not all be
|
||||
# there. Same reason the object listing above comes from S3.
|
||||
index_file <- file.path(tempdir(), "flat-PACKAGES.gz")
|
||||
s3fs::s3_file_download(
|
||||
sprintf("%s/PACKAGES.gz", contrib),
|
||||
index_file,
|
||||
overwrite = TRUE
|
||||
)
|
||||
compiled_names <- cran[
|
||||
!is.na(cran[, "NeedsCompilation"]) & cran[, "NeedsCompilation"] == "yes",
|
||||
"Package"
|
||||
index <- read.dcf(
|
||||
gzfile(index_file),
|
||||
fields = c("Package", "Version", "NeedsCompilation")
|
||||
)
|
||||
if (nrow(index) == 0L) {
|
||||
stop(
|
||||
sprintf(
|
||||
"Read no records from %s/PACKAGES.gz; refusing to guess what is compiled.",
|
||||
contrib
|
||||
),
|
||||
call. = FALSE
|
||||
)
|
||||
}
|
||||
# Key on the object, not the package: a slot can carry several versions and
|
||||
# only some of them compile.
|
||||
compiled_objects <- paste(
|
||||
index[, "Package"],
|
||||
index[, "Version"],
|
||||
sep = "_"
|
||||
)[
|
||||
!is.na(index[, "NeedsCompilation"]) &
|
||||
tolower(index[, "NeedsCompilation"]) == "yes"
|
||||
]
|
||||
keep <- flat_pkg %in% compiled_names
|
||||
keep <- paste(flat_pkg, flat_ver, sep = "_") %in% compiled_objects
|
||||
compiled <- data.frame(
|
||||
Package = flat_pkg[keep],
|
||||
Version = flat_ver[keep],
|
||||
|
|
|
|||
|
|
@ -1,22 +1,30 @@
|
|||
### Scope: Check whether there are any incomplete entries in PACKAGES.rds. These will result in NA when calling available.packages()
|
||||
foo = available.packages("https://cran.devxy.io/amd64/jammy/latest/src/contrib")
|
||||
foo <- available.packages(
|
||||
"https://cran.devxy.io/amd64/jammy/latest/src/contrib"
|
||||
)
|
||||
sum((is.na(foo[, "Version"])))
|
||||
which((is.na(foo[, "Version"])))
|
||||
|
||||
foo = available.packages("https://cran.devxy.io/amd64/noble/latest/src/contrib")
|
||||
foo <- available.packages(
|
||||
"https://cran.devxy.io/amd64/noble/latest/src/contrib"
|
||||
)
|
||||
sum((is.na(foo[, "Version"])))
|
||||
which((is.na(foo[, "Version"])))
|
||||
|
||||
|
||||
foo = available.packages("https://cran.devxy.io/amd64/rhel8/latest/src/contrib")
|
||||
foo <- available.packages(
|
||||
"https://cran.devxy.io/amd64/rhel8/latest/src/contrib"
|
||||
)
|
||||
sum((is.na(foo[, "Version"])))
|
||||
which((is.na(foo[, "Version"])))
|
||||
|
||||
foo = available.packages("https://cran.devxy.io/amd64/rhel9/latest/src/contrib")
|
||||
foo <- available.packages(
|
||||
"https://cran.devxy.io/amd64/rhel9/latest/src/contrib"
|
||||
)
|
||||
sum((is.na(foo[, "Version"])))
|
||||
which((is.na(foo[, "Version"])))
|
||||
|
||||
foo = available.packages(
|
||||
foo <- available.packages(
|
||||
"https://cran.devxy.io/amd64/alpine320/latest/src/contrib"
|
||||
)
|
||||
sum((is.na(foo[, "Version"])))
|
||||
|
|
@ -24,24 +32,32 @@ which((is.na(foo[, "Version"])))
|
|||
|
||||
# arm64
|
||||
|
||||
foo = available.packages("https://cran.devxy.io/arm64/jammy/latest/src/contrib")
|
||||
foo <- available.packages(
|
||||
"https://cran.devxy.io/arm64/jammy/latest/src/contrib"
|
||||
)
|
||||
sum((is.na(foo[, "Version"])))
|
||||
which((is.na(foo[, "Version"])))
|
||||
|
||||
foo = available.packages("https://cran.devxy.io/arm64/noble/latest/src/contrib")
|
||||
foo <- available.packages(
|
||||
"https://cran.devxy.io/arm64/noble/latest/src/contrib"
|
||||
)
|
||||
sum((is.na(foo[, "Version"])))
|
||||
which((is.na(foo[, "Version"])))
|
||||
|
||||
|
||||
foo = available.packages("https://cran.devxy.io/arm64/rhel8/latest/src/contrib")
|
||||
foo <- available.packages(
|
||||
"https://cran.devxy.io/arm64/rhel8/latest/src/contrib"
|
||||
)
|
||||
sum((is.na(foo[, "Version"])))
|
||||
which((is.na(foo[, "Version"])))
|
||||
|
||||
foo = available.packages("https://cran.devxy.io/arm64/rhel9/latest/src/contrib")
|
||||
foo <- available.packages(
|
||||
"https://cran.devxy.io/arm64/rhel9/latest/src/contrib"
|
||||
)
|
||||
sum((is.na(foo[, "Version"])))
|
||||
which((is.na(foo[, "Version"])))
|
||||
|
||||
foo = available.packages(
|
||||
foo <- available.packages(
|
||||
"https://cran.devxy.io/arm64/alpine320/latest/src/contrib"
|
||||
)
|
||||
sum((is.na(foo[, "Version"])))
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
library(future)
|
||||
future::plan(multisession)
|
||||
future::plan(sequential)
|
||||
time = Sys.time()
|
||||
time <- Sys.time()
|
||||
files <- s3fs::s3_dir_ls(
|
||||
sprintf(
|
||||
"devxy-r-package-binaries-hel1/arm64/rhel9/latest/src/contrib/Archive"
|
||||
|
|
@ -54,8 +54,8 @@ curl::curl_download(
|
|||
"https://cran.devxy.io/amd64/rhel9/latest/src/contrib/PACKAGES.db",
|
||||
"PACKAGES.db"
|
||||
)
|
||||
con = DBI::dbConnect(RSQLite::SQLite(), "PACKAGES.db")
|
||||
df = DBI::dbReadTable(con, "packages")
|
||||
con <- DBI::dbConnect(RSQLite::SQLite(), "PACKAGES.db")
|
||||
df <- DBI::dbReadTable(con, "packages")
|
||||
|
||||
cranlike:::write_packages_files("/tmp", "PACKAGES.db")
|
||||
|
||||
|
|
@ -69,11 +69,11 @@ system("cat /tmp/PACKAGES | grep '^Package: digest' | wc -l")
|
|||
curl::curl_fetch_memory(
|
||||
"https://cloud.r-project.org/src/contrib/Meta/archive.rds"
|
||||
)
|
||||
remotes = readRDS(url(
|
||||
remotes <- readRDS(url(
|
||||
"https://cloud.r-project.org/src/contrib/Meta/archive.rds",
|
||||
"rb"
|
||||
))
|
||||
packages = readRDS(url(
|
||||
packages <- readRDS(url(
|
||||
"https://cran.devxy.io/arm64/noble/latest/src/contrib/PACKAGES.rds",
|
||||
"rb"
|
||||
))
|
||||
|
|
|
|||
|
|
@ -45,12 +45,12 @@ data <- dbGetQuery(
|
|||
"SELECT name FROM single_builds WHERE platform = $1 AND arch = $2 and removed = FALSE;",
|
||||
params = list(platform, arch)
|
||||
)
|
||||
removed_pkgs = get_removed_cran_packages(lubridate::interval(
|
||||
removed_pkgs <- get_removed_cran_packages(lubridate::interval(
|
||||
lubridate::today(),
|
||||
lubridate::today() - 2
|
||||
))$name
|
||||
pkgs_db <- unique(data$name)
|
||||
pkgs_db = setdiff(pkgs_db, removed_pkgs)
|
||||
pkgs_db <- setdiff(pkgs_db, removed_pkgs)
|
||||
|
||||
pkgs <- setdiff(cran_pkgs, pkgs_db)
|
||||
formatted_pkgs <- gsub(
|
||||
|
|
|
|||
|
|
@ -42,14 +42,14 @@ files <- s3fs::s3_dir_ls(sprintf(
|
|||
"devxy-r-package-binaries-hel1/arm64/alpine320/latest/src/contrib"
|
||||
))
|
||||
|
||||
files_b = basename(files)
|
||||
files_b_split = sapply(strsplit(basename(files_b), "_"), function(x) x[1])
|
||||
files_b <- basename(files)
|
||||
files_b_split <- sapply(strsplit(basename(files_b), "_"), function(x) x[1])
|
||||
|
||||
cran_pkgs <- tools::CRAN_package_db() |>
|
||||
filter(`Date/Publication` <= "2025-01-25") |>
|
||||
pull(Package)
|
||||
|
||||
pkgs = setdiff(cran_pkgs, files_b_split)
|
||||
pkgs <- setdiff(cran_pkgs, files_b_split)
|
||||
formatted_pkgs <- gsub(
|
||||
'"',
|
||||
"'",
|
||||
|
|
|
|||
|
|
@ -39,14 +39,14 @@ files <- s3fs::s3_dir_ls(sprintf(
|
|||
"devxy-r-package-binaries-hel1/arm64/alpine320/latest/src/contrib"
|
||||
))
|
||||
|
||||
files_b = basename(files)
|
||||
files_b_split = sapply(strsplit(basename(files_b), "_"), function(x) x[1])
|
||||
files_b_split = setdiff(
|
||||
files_b <- basename(files)
|
||||
files_b_split <- sapply(strsplit(basename(files_b), "_"), function(x) x[1])
|
||||
files_b_split <- setdiff(
|
||||
files_b_split,
|
||||
c("PACKAGES", "PACKAGES.gz", "PACKAGES.rds", "PACKAGES.db", "Archive")
|
||||
)
|
||||
# windows-only
|
||||
files_b_split = setdiff(
|
||||
files_b_split <- setdiff(
|
||||
files_b_split,
|
||||
c(
|
||||
"PACKAGES",
|
||||
|
|
@ -67,12 +67,12 @@ files_b_split = setdiff(
|
|||
"R2wd"
|
||||
)
|
||||
)
|
||||
pkgs_index = available.packages(
|
||||
pkgs_index <- available.packages(
|
||||
"https://cran.devxy.io/amd64/jammy/latest/src/contrib"
|
||||
)[, "Package"]
|
||||
pkgs_index_b = basename(pkgs_index)
|
||||
pkgs_index_b <- basename(pkgs_index)
|
||||
if (length(files_b_split) != length(pkgs_index_b)) {
|
||||
pkgs_missing = setdiff(files_b_split, pkgs_index_b)
|
||||
pkgs_missing <- setdiff(files_b_split, pkgs_index_b)
|
||||
message("Packages missing in index but present in S3:")
|
||||
pkgs_missing
|
||||
formatted_pkgs <- gsub(
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ pkgs <- data |>
|
|||
distinct(name) |>
|
||||
pull(name)
|
||||
|
||||
pkgs = setdiff(cran_pkgs, pkgs)
|
||||
pkgs <- setdiff(cran_pkgs, pkgs)
|
||||
# Format, remove line breaks, and print as a single line
|
||||
formatted_pkgs <- gsub(
|
||||
'"',
|
||||
|
|
|
|||
|
|
@ -29,12 +29,14 @@ All Phase A paths are relative to `/Users/pjs/git/codefloe.com/rpkgs/bincraftr`.
|
|||
|
||||
### Task A0: Proof of mechanism — pak installs a patched binary from a prepended `file://` repo
|
||||
|
||||
This de-risks the core assumption before building anything on top: that `pak` installs a binary from a local `file://` repo in preference to CRAN for an equal version, and does so without recompiling. If this fails, the contingency (documented in Step 4) is to serve patched *source* and rely on `pkgcache` build-caching — the rest of the plan changes only inside `build_patched_binary()`.
|
||||
This de-risks the core assumption before building anything on top: that `pak` installs a binary from a local `file://` repo in preference to CRAN for an equal version, and does so without recompiling. If this fails, the contingency (documented in Step 4) is to serve patched _source_ and rely on `pkgcache` build-caching — the rest of the plan changes only inside `build_patched_binary()`.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `tools/verify-patch-mechanism.R`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Produces: a runnable script proving `pak::pkg_install()` resolves a local patched binary over CRAN. No package API.
|
||||
|
||||
- [ ] **Step 1: Write the verification script**
|
||||
|
|
@ -118,17 +120,19 @@ git commit -m "test(patches): prove pak installs a patched binary from a local f
|
|||
- [ ] **Step 4: Record the outcome / contingency**
|
||||
|
||||
If the proof PASSED, proceed to Task A1 unchanged.
|
||||
If it FAILED (pak recompiled or picked CRAN's), the mechanism switches to serving patched *source*: in Task A4 `build_patched_binary()` skips `pkgbuild::build()` and instead repackages the patched source tree with `pkgbuild::build(binary = FALSE)`; everything else (registry, matching, cache, repo prepend) is unchanged because `pak` build-caches the compiled result via `pkgcache`. Note the chosen path in the commit message and continue.
|
||||
If it FAILED (pak recompiled or picked CRAN's), the mechanism switches to serving patched _source_: in Task A4 `build_patched_binary()` skips `pkgbuild::build()` and instead repackages the patched source tree with `pkgbuild::build(binary = FALSE)`; everything else (registry, matching, cache, repo prepend) is unchanged because `pak` build-caches the compiled result via `pkgcache`. Note the chosen path in the commit message and continue.
|
||||
|
||||
---
|
||||
|
||||
### Task A1: Registry loading and normalization
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `R/patches.R`
|
||||
- Test: `tests/testthat/test-patches.R`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Produces: `load_patch_registry(patches_dir)` → `list()` of normalized entries; each entry is a named list with `package`, `versions`, `platforms` (character vector), `env` (named list), `configure_args` (character), `makevars` (named list), `reason`, and `patch_path` (absolute path or `NULL`). `normalize_patch_entry(entry, patches_dir)` → one normalized entry; errors on missing required field or missing patch file.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
|
@ -275,10 +279,12 @@ git commit -m "feat(patches): load and validate the patch registry"
|
|||
### Task A2: Platform matching and version-constraint satisfaction
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `R/patches.R`
|
||||
- Test: `tests/testthat/test-patches.R`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Consumes: normalized entries from Task A1.
|
||||
- Produces: `build_platform_tokens(platform, arch)` → character vector; `entry_matches_platform(entry, tokens)` → logical; `match_patch_entries(registry, platform, arch)` → filtered list; `version_satisfies(version, constraint)` → logical (constraint forms: `"*"` handled by caller, `"x.y.z"` exact, `">=x"`, `"<=x"`, `">x"`, `"<x"`, `"==x"`).
|
||||
|
||||
|
|
@ -398,10 +404,12 @@ git commit -m "feat(patches): platform matching and version-constraint checks"
|
|||
### Task A3: Cache key and version resolution
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `R/patches.R`
|
||||
- Test: `tests/testthat/test-patches.R`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Consumes: normalized entries.
|
||||
- Produces: `patch_cache_key(entry, version, platform, arch, r_minor)` → string `"<pkg>_<ver>_<platform>_<arch>_<rminor>_<hash12>"`, where `hash12` covers `env`/`configure_args`/`makevars`/patch bytes; `resolve_patch_version(entry)` → latest CRAN version satisfying `entry$versions`, or `NA_character_`; `describe_patch(entry)` → short human label.
|
||||
|
||||
|
|
@ -546,10 +554,12 @@ git commit -m "feat(patches): cache key, version resolution, and patch descripti
|
|||
### Task A4: Build a patched binary in isolation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `R/patches.R`
|
||||
- Test: `tests/testthat/test-patches.R`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Consumes: a normalized entry, a resolved `version`, a `dest_dir`.
|
||||
- Produces: `download_cran_source(package, version, dest_dir, cran)` → path or `NULL`; `apply_source_patch(patch_path, pkg_src)` → logical; `configure_args_to_build_args(configure_args)` → character; `build_patched_binary(entry, version, dest_dir)` → path to built binary tarball or `NULL`.
|
||||
|
||||
|
|
@ -737,10 +747,12 @@ git commit -m "feat(patches): build a patched binary in isolation from CRAN sour
|
|||
### Task A5: Orchestrate the local patched repo (cache + index)
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `R/patches.R`
|
||||
- Test: `tests/testthat/test-patches.R`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Consumes: all helpers above.
|
||||
- Produces: `prepare_patched_repo(patches_dir, platform, arch, r_minor, cache_dir, repo_dir)` → path to a `src/contrib`-style dir containing patched binaries + a `PACKAGES` index, or `NULL` when nothing matched/built. On a cache hit it copies the cached tarball into `repo_dir`; on a miss it builds, then writes the result into `cache_dir`.
|
||||
|
||||
|
|
@ -902,11 +914,13 @@ git commit -m "feat(patches): orchestrate local patched-binary repo with caching
|
|||
### Task A6: Wire patched repo into the pak install path
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `R/install_helpers.R:333-389` (`run_pak_install_with_mutex`)
|
||||
- Modify: `R/install-deps.R:23-75` (`install_pkg_sys_deps`)
|
||||
- Test: `tests/testthat/test-patches.R`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Consumes: `prepare_patched_repo()`.
|
||||
- Produces: `run_pak_install_with_mutex(local_clone_dir_single, env_vars, patched_repo = NULL)` — prepends `file://<patched_repo>` to `options("repos")` for the install; `install_pkg_sys_deps(package_name, tag, local_clone_dir, platform, aggressive_cleanup = FALSE, patches = NULL, arch = NULL)` — builds the patched repo before installing.
|
||||
|
||||
|
|
@ -1052,11 +1066,13 @@ git commit -m "feat(patches): serve patched binaries to pak during dep install"
|
|||
### Task A7: Thread `patches` through the public build API
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `R/build_binaries.R` (`build_binary_package`, `execute_package_builds`, `build_single_tag`, `handle_system_dependencies`)
|
||||
- Create: `man-roxygen/param-patches.R`
|
||||
- Test: `tests/testthat/test-patches.R`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Produces: `build_binary_package(..., patches = NULL)` and the internal chain each carry `patches` down to `install_pkg_sys_deps()`. `handle_system_dependencies(..., patches = NULL)` passes `patches` and `arch` through.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
|
@ -1151,10 +1167,12 @@ git commit -m "feat(patches): thread patches argument through build_binary_packa
|
|||
### Task A8: End-to-end patch test (guarded) and version bump
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `tests/testthat/test-patches.R`
|
||||
- Modify: `DESCRIPTION` (version), `NEWS.md`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Produces: a guarded e2e test proving a dependent package builds when its failing dependency is patched.
|
||||
|
||||
- [ ] **Step 1: Add the guarded e2e test**
|
||||
|
|
@ -1202,7 +1220,7 @@ In `DESCRIPTION`, bump `Version:` to `4.3.0.9999`. Prepend to `NEWS.md`:
|
|||
```markdown
|
||||
# bincraft 4.3.0
|
||||
|
||||
* `build_binary_package()` gains a `patches` argument: a registry of
|
||||
- `build_binary_package()` gains a `patches` argument: a registry of
|
||||
per-package env / configure / Makevars overrides and source diffs that are
|
||||
pre-built into patched binaries and served to pak, fixing compiler- and
|
||||
OS-specific failures (e.g. RcppParallel) including for transitive deps.
|
||||
|
|
@ -1224,10 +1242,12 @@ All Phase B paths are relative to this repo (`build-cran-binaries`). Phase B dep
|
|||
### Task B1: Create the patch registry with the RcppParallel entry
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `local/patches/registry.json`
|
||||
- Create: `local/patches/README.md`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Produces: the curated registry consumed by bincraft's `patches` argument.
|
||||
|
||||
- [ ] **Step 1: Write the registry**
|
||||
|
|
@ -1268,9 +1288,11 @@ git commit -m "feat(patches): add patch registry with RcppParallel TBB workaroun
|
|||
### Task B2: Registry validator script
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `local/validate-patches.R`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Consumes: `local/patches/registry.json`.
|
||||
- Produces: a script that exits non-zero on schema violations, missing patch files, or ambiguous overlapping entries.
|
||||
|
||||
|
|
@ -1360,9 +1382,11 @@ git commit -m "feat(patches): add registry validator script"
|
|||
### Task B3: Hook the validator into pre-commit
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `.pre-commit-config.yaml`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Produces: a local hook that runs `local/validate-patches.R` when the registry or patch files change.
|
||||
|
||||
- [ ] **Step 1: Add the hook**
|
||||
|
|
@ -1397,10 +1421,12 @@ git commit -m "ci(patches): validate patch registry in pre-commit"
|
|||
### Task B4: Pass `patches` through the build entry points
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `local/build-one.R:99-119` (the `build_binary_package` call)
|
||||
- Modify: `local/build-all.R:125-145` (the `build_binary_package` call)
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Consumes: the new bincraft `patches` argument (Phase A) and `local/patches/`.
|
||||
- Produces: both entry points pass `patches = "local/patches"`.
|
||||
|
||||
|
|
@ -1437,9 +1463,11 @@ git commit -m "feat(patches): pass patch registry to bincraft build calls"
|
|||
### Task B5: Document the feature in the README
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `README.md`
|
||||
|
||||
**Interfaces:**
|
||||
|
||||
- Produces: a short "Patching packages" section explaining the registry and linking the design spec.
|
||||
|
||||
- [ ] **Step 1: Add a README section**
|
||||
|
|
@ -1458,6 +1486,7 @@ git commit -m "docs(patches): document the package patching workflow"
|
|||
## Self-Review
|
||||
|
||||
**Spec coverage:**
|
||||
|
||||
- Registry in this repo, passed to bincraft → Task A7 (`patches` arg), B1 (registry), B4 (wiring). ✓
|
||||
- Mechanism in bincraft (resolve → pre-build → prepend repo → install) → A4–A6. ✓
|
||||
- Both tiers, env-first → A4 (`env`, `configure_args`, `makevars`, `patch` all applied; env is the cheap default and the RcppParallel entry uses only env). ✓
|
||||
|
|
|
|||
|
|
@ -33,13 +33,13 @@ For a fix to reach a dependency-of-a-dependency, the fixed package must be visib
|
|||
Decisions taken during brainstorming:
|
||||
|
||||
- **Mechanism lives in `bincraft`** (the engine), because only there can transitive deps be influenced.
|
||||
- **Patch tiers: both, env-overrides first.** Support cheap per-package env vars / configure args / Makevars (version-independent) *and* true source diffs (version-pinned), preferring the lightweight override.
|
||||
- **Patch tiers: both, env-overrides first.** Support cheap per-package env vars / configure args / Makevars (version-independent) _and_ true source diffs (version-pinned), preferring the lightweight override.
|
||||
- **Registry data lives in this repo** (`build-cran-binaries`) and is passed into `bincraft`, keeping `bincraft` as pure mechanism and the frequently-changing policy data with operational config.
|
||||
|
||||
## Approaches considered
|
||||
|
||||
| Approach | How pak sees the fix | Verdict |
|
||||
|---|---|---|
|
||||
| ---------------------------------------------------------------------------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| A. Patched **source** repo — drop patched `.tar.gz` source into a local repo, prepend it | pak recompiles from your source | Simple, but env-tier overrides leak globally (one subprocess builds everything) and the dep recompiles on every dependent build |
|
||||
| **B. Pre-built patched binary repo (chosen)** | pak installs a ready binary by repo priority | Per-package scoping is free; no recompile; the binary is a cacheable/uploadable artifact that fits the existing system |
|
||||
| C. pkgdepends per-build hook | intercept each build | No clean per-package pre-compile hook exists; fragile |
|
||||
|
|
|
|||
Loading…
Reference in a new issue