Design: classify failing binary builds and auto-propose registry patches #115
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Turn recurring build failures into triaged patch suggestions instead of hand-scraping Crow logs.
The
single_buildsmetadata table already records every failure with its error text, so we can classify those errors against known signatures and open PRs that pre-filllocal/patches/registry.jsonentries, keeping the existing human review gate.Motivation
Adding a patch today is fully manual: a build fails, someone reads the Crow log, recognises the failure class (musl + bundled native dep, a removed TBB header, a system-lib link leak, ...), picks the right fix tier, and hand-writes a
registry.jsonentry.The recent entries show the recurring shapes:
RcppParallelbundled Intel TBB fails on musl / g++ 15 (disable-tbb.patch).fslinks system libuv and ships a binary withNEEDED libuv.so.1(force-vendored-libuv.patch).rstan/ StanHeaders includes the removedtbb/tbb_stddef.h(-DTBB_INTERFACE_NEW, #114).These are a small, repeating set of fingerprints, which is exactly what a classifier is good at.
What already exists (foundation is in place)
bincraft::store_build_metadata()writeserror_occurredanderror_text(plusname,tag,platform,arch,r_version,timestamp) into thesingle_buildsPostgres table (r-binaries.devxy.io:15432/build_metadata).local/validate-patches.R), and the bincraft ingestion/isolated-build path are all live.patchhashalready auto-invalidates cached patched binaries when an entry or its diff changes, so a proposed entry can be trial-built in isolation without polluting the real cache.So the missing piece is not capture; it is classification + proposal.
Proposed design (staged, cheapest first)
1. Failure view (query-only, no schema change)
A helper (e.g.
local/failing-builds-report.R) that queriessingle_builds WHERE error_occurredand groups by a normalised error fingerprint, so we can see "N packages, M platforms, same root error" at a glance.Normalisation strips version numbers, temp paths (
/tmp/Rtmp...), and package-specific tokens fromerror_textso the same root cause collapses to one bucket.2. Signature classifier (rules first, LLM for the long tail)
Map a normalised
error_textto a suggested fix tier. Seed rules from the existing entries:tbb/tbb_stddef.h: No such fileCPPFLAGS += -DTBB_INTERFACE_NEWNEEDED libuv.so.1/ dyn.load of*.sofails on missing runtime libUSE_TBB ... is not supported/ bundled TBB build hang on musldisable-tbbstyle overrideEach rule carries a confidence and the tier (
env/configure_args/makevars/patch).Unknown signatures are surfaced for a human, never guessed at.
3. Propose, do not apply (human gate stays)
When a failing build matches a known signature and the package has no current registry entry, open a PR (or issue) with the pre-filled entry +
reason, run the validator, and let an isolated trial build confirm before merge.Acceptance criteria = validator passes + the trial patched build succeeds + the previously-failing dependent installs.
4. Feedback loop
Track proposed-vs-merged and signature hit rate so the rule set improves, and so a signature that stops matching (upstream fixed it) can retire its entry.
Explicit non-goals / guardrails
.patchthat silences a compile error is exactly how we ship a broken-but-installable binary (thefs/libuv failure mode). That stays human-reviewed.src/contribindex.single_builds.error_text.Suggested first step
Build 1 + 2 (failure view + rule-based classifier over
single_builds) as a read-only reporting script.That immediately replaces "user pastes a log" with a triaged suggestion, and is a safe place to validate the signature set before wiring up automated PR creation in step 3.