feat(local): auto-apply registry patches with a build-env trial-build gate (#124)
All checks were successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful

Closes the classifier loop (issue #115, step 3): turns the auto-proposable candidates into an actual PR, gated by a real trial build **in our own build-env images**.

Chosen model (from the design discussion): **autonomous PR**, **PR-first with a CI trial-build gate**, **bounded top-N batch** per run.

## Creating the patch PR

- `propose-patches.R` gains:
  - `--limit N` -- act on the top-N candidates by failure volume; the rest defer to the next run (logged).
  - `--open-pr` -- write the entries onto the reused `auto/registry-patch-proposals` branch, push (with `REPO_RW_TOKEN`), and open/update **one** PR via the Forgejo API (so re-runs update the same PR instead of piling up).
- `.crow/auto-apply-patches.yaml` -- a single job that runs `--open-pr --limit` on a cron/manual trigger. Needs `FORGEJO_TOKEN` + a write-scoped `REPO_RW_TOKEN`.

Novel source diffs and unknown signatures are still never proposed; nothing merges.

## The merge gate (our build-env images)

- `.crow/trial-build-registry.yaml` -- matrixed over the real `OS/IMG` build-env matrix (alpine:3.24, redhat:8/9/10, ubuntu:jammy/noble/**resolute** for ubuntu-2604). Each platform runs `local/trial-build-registry.R`, which diffs the branch registry against `main` and trial-builds **only the entries the branch adds** that apply to that platform, inside `reg.devxy.io/rpkgs/build-env-*`. Green only if every new entry builds; a platform with no new entries is a fast no-op. Nothing is uploaded/archived/recorded.
- The base-registry read **fails loud** if it can't read `registry.json` at `main`, rather than silently treating the base as empty and trial-building the whole registry.

## Notes / follow-up

- The repo uses **no `pull_request` triggers**, so the gate runs manually or on a cron against the branch (`--var patch_branch=...`). Wiring it to fire automatically on the PR needs `event: pull_request` enabled on the Forgejo webhook -- a one-line addition once that's on.
- Two new crons to register in the crow UI: `auto-apply-patches` and `trial-build-registry`. New secret needed: `REPO_RW_TOKEN` (write scope) for the push.

## Verification

- New pure helpers `entry_applies_to_os()` / `new_registry_packages()` covered by tests (platform codename/family/wildcard matching; added-vs-unchanged entry detection; per-platform filtering).
- `--limit` smoke (stubbed DB): top-2 by volume proposed, 3 deferred, candidate registry validates.
- Full suite: 105 tests pass; all pre-commit hooks pass (air, prettier, markdownlint, yamllint, validate-patches).

Reviewed-on: #124
This commit is contained in:
Patrick Schratz 2026-07-15 08:28:15 +00:00 committed by Patrick Schratz
commit 4bca17e4ac

View file

@ -15,10 +15,17 @@
#
# Usage:
# PGPASS=... Rscript local/propose-patches.R [--platform P] [--arch A] [--min N]
# [--limit N]
# (default) print candidates + validation, take no action
# --write append candidates to local/patches/registry.json and
# the proposals ledger (commit + open a PR yourself)
# --open-issue post/update a Forgejo tracking issue (needs FORGEJO_TOKEN)
# --open-pr write the entries, push the `auto/registry-patch-proposals`
# branch, and open/update a PR autonomously (needs
# FORGEJO_TOKEN, and REPO_RW_TOKEN to push in CI). The
# `trial-build-registry` pipeline is the merge gate.
# --limit N only act on the top-N candidates by failure volume
# (bounded batch; the rest are picked up on the next run)
# --json PATH also write the machine-readable candidate list to PATH
options(error = function() {
@ -58,6 +65,8 @@ if (is.na(min_count)) {
}
do_write <- "--write" %in% args
do_issue <- "--open-issue" %in% args
do_pr <- "--open-pr" %in% args
limit <- suppressWarnings(as.integer(opt_val("--limit", NA_character_)))
if (nchar(Sys.getenv("PGPASS")) == 0L) {
stop("PGPASS env var is not set; a DB password is required.")
@ -65,6 +74,8 @@ if (nchar(Sys.getenv("PGPASS")) == 0L) {
registry_file <- file.path(script_dir, "patches", "registry.json")
ledger_file <- file.path(script_dir, "patches", "proposals-log.json")
# Branch the autonomous PR reuses, so re-runs update one PR instead of piling up.
pr_branch <- "auto/registry-patch-proposals"
# ---------------------------------------------------------------------------
# Query failing builds (same shape as failing-builds-report.R)
@ -118,7 +129,8 @@ registered_pkgs <- unique(vapply(
report <- build_triage_report(failures, registered_pkgs)
report <- Filter(function(r) r$build_count >= min_count, report)
# Flatten auto-proposable groups into candidate records.
# Flatten auto-proposable groups into candidate records, carrying the group's
# build volume so we can prioritise the highest-impact fixes.
candidates <- list()
for (r in report) {
if (is.null(r$proposed_entries)) {
@ -130,6 +142,7 @@ for (r in report) {
signature = r$signature,
confidence = r$confidence,
tier = r$tier,
build_count = r$build_count,
entry = r$proposed_entries[[pkg]]
)
}
@ -173,6 +186,22 @@ if (length(candidates) == 0L) {
q(status = 0)
}
# Prioritise by failure volume, then apply --limit so one run tackles a bounded
# batch (the rest are picked up on the next run).
candidates <- candidates[order(
-vapply(candidates, function(c) c$build_count %||% 0L, integer(1L))
)]
deferred <- 0L
if (!is.na(limit) && limit >= 0L && length(candidates) > limit) {
deferred <- length(candidates) - limit
candidates <- utils::head(candidates, limit)
cat(sprintf(
"\nLimiting to top %d candidate(s) by failure volume; %d deferred to a later run.\n",
limit,
deferred
))
}
cat(sprintf(
"\n%d candidate registry %s:\n",
length(candidates),
@ -386,8 +415,163 @@ if (do_write) {
cat(sprintf("Opened tracking issue #%d.\n", created$number))
}
save_ledger(merge_ledger(load_ledger(), new_ledger_records))
} else if (do_pr) {
forgejo_token <- Sys.getenv("FORGEJO_TOKEN")
if (nchar(forgejo_token) == 0L) {
stop("--open-pr requires FORGEJO_TOKEN (to open the PR).")
}
suppressPackageStartupMessages(library(httr2, quietly = TRUE))
forgejo_base <- "https://git.devxy.io/api/v1"
repo <- "devxy/build-cran-binaries"
# Write the entries + ledger, then commit them onto the reused auto branch.
jsonlite::write_json(
merged,
registry_file,
auto_unbox = TRUE,
pretty = TRUE,
null = "null"
)
save_ledger(merge_ledger(load_ledger(), new_ledger_records))
git <- function(...) {
st <- system2("git", c(...), stdout = TRUE, stderr = TRUE)
if (!identical(attr(st, "status"), NULL)) {
stop(sprintf(
"git %s failed:\n%s",
paste(..., collapse = " "),
paste(st, collapse = "\n")
))
}
invisible(st)
}
git("config", "user.name", Sys.getenv("GIT_USER", "devxy-bot"))
git(
"config",
"user.email",
Sys.getenv("GIT_EMAIL", "bot@devxy.io")
)
git("checkout", "-B", pr_branch)
git(
"add",
file.path(script_dir, "patches", "registry.json"),
ledger_file
)
git(
"commit",
"-m",
sprintf(
"feat(patches): auto-propose %d registry %s from classified failures",
length(candidate_entries),
if (length(candidate_entries) == 1L) "entry" else "entries"
)
)
# Push with a write token when provided (CI); otherwise rely on origin creds.
rw_token <- Sys.getenv("REPO_RW_TOKEN")
push_target <- if (nchar(rw_token) > 0L) {
sprintf(
"https://%s:%s@git.devxy.io/%s.git",
Sys.getenv("GIT_USER", "devxy-bot"),
rw_token,
repo
)
} else {
"origin"
}
git("push", "-f", push_target, sprintf("HEAD:refs/heads/%s", pr_branch))
pr_title <- sprintf(
"feat(patches): auto-proposed registry patches (%s)",
now
)
body_lines <- c(
sprintf(
"_Auto-generated %s by `local/propose-patches.R --open-pr` from classified `single_builds` failures._",
now
),
"",
sprintf(
"Adds %d known-lever registry %s (top by failure volume%s).",
length(candidate_entries),
if (length(candidate_entries) == 1L) "entry" else "entries",
if (deferred > 0L) {
sprintf("; %d deferred to a later run", deferred)
} else {
""
}
),
"",
"**Merge gate:** the `trial-build-registry` pipeline builds each new entry in its target build-env image; merge only once it is green.",
"Novel source diffs and unknown signatures are never auto-proposed.",
"",
"| package | signature | platforms |",
"| --- | --- | --- |"
)
for (c in candidates) {
body_lines <- c(
body_lines,
sprintf(
"| %s | %s | %s |",
c$package,
c$signature,
toString(as.character(c$entry$platforms))
)
)
}
new_body <- paste(body_lines, collapse = "\n")
# One PR per reused branch: update if open, else create.
pulls_url <- sprintf(
"%s/repos/%s/pulls?state=open&limit=50",
forgejo_base,
repo
)
open_pulls <- httr2::request(pulls_url) |>
httr2::req_headers(Authorization = paste("token", forgejo_token)) |>
httr2::req_perform() |>
httr2::resp_body_json(simplifyVector = FALSE)
match_idx <- which(vapply(
open_pulls,
function(p) identical(p$head$ref, pr_branch),
logical(1L)
))
if (length(match_idx) > 0L) {
num <- open_pulls[[match_idx[1]]]$number
httr2::request(sprintf("%s/repos/%s/pulls/%d", forgejo_base, repo, num)) |>
httr2::req_headers(
Authorization = paste("token", forgejo_token),
`Content-Type` = "application/json"
) |>
httr2::req_body_json(list(title = pr_title, body = new_body)) |>
httr2::req_method("PATCH") |>
httr2::req_perform()
cat(sprintf("\nUpdated auto-patch PR #%d (branch %s).\n", num, pr_branch))
} else {
created <- httr2::request(sprintf(
"%s/repos/%s/pulls",
forgejo_base,
repo
)) |>
httr2::req_headers(
Authorization = paste("token", forgejo_token),
`Content-Type` = "application/json"
) |>
httr2::req_body_json(list(
title = pr_title,
head = pr_branch,
base = "main",
body = new_body
)) |>
httr2::req_perform() |>
httr2::resp_body_json()
cat(sprintf(
"\nOpened auto-patch PR #%d (branch %s).\n",
created$number,
pr_branch
))
}
} else {
cat(
"\nDry run: no changes made. Re-run with --write or --open-issue to act.\n"
"\nDry run: no changes made. Re-run with --write, --open-issue, or --open-pr to act.\n"
)
}