feat(local): report unclassified and dependency-blocked failures for discovery #122

Merged

pat-s merged 1 commit from t3code/report-unclassified-failures into main

2026-07-15 07:46:29 +00:00

pat-s commented

2026-07-15 07:34:24 +00:00

Owner

Why

Issue #120 (the auto-proposed-patches issue) only lists auto-proposable fixes -- currently just the TBB signatures. So a reasonable read of it was "TBB is our only build failure", when in fact three whole categories are simply not shown there:

Unclassified failures -- anything that doesn't match a seeded signature is routed to human triage and never appears (we've only seeded TBB and libuv signatures).
Dependency-blocked failures -- the ~800 RcppParallel dependents (post #121) are still failing; they only show as a log line.
Human-only signatures (libuv).

These blind spots are exactly where the next signatures should come from, so they deserve the same visibility as the proposals.

What

Extend the feedback-loop tracker to surface the classifier's blind spots:

unclassified_summary() -- groups every unknown-signature failure by normalised fingerprint, ranked by build count, capped with an explicit dropped_groups count (no silent truncation), each with example packages + platforms. These are the candidates for new build_signatures() rules.
blocked_summary() -- lists each dependency (e.g. RcppParallel) and how many dependents wait on it.
proposal-tracking.R prints both sections, and a new --open-issue mode posts/updates a "Unclassified build failures (needs signatures) (#115)" Forgejo issue.
The weekly crow pipeline now runs the tracker with --open-issue, so it maintains a second tracking issue alongside the proposals one. Read-only on the DB; the only writes are the two issues.

Verification

New tests cover unclassified_summary (ranking + both caps) and blocked_summary.
Tracker smoke with a stubbed DB (proposable + blocked + unclassified mix) prints the hit rate, Blocked on a dependency: RcppParallel: 2 dependent(s), and Unclassified failures ... [2 builds | 2 pkgs] ld: undefined reference ....
Full suite: 95 tests pass; all pre-commit hooks pass (air, prettier, markdownlint, yamllint, validate-patches).

## Why Issue #120 (the auto-proposed-patches issue) only lists **auto-proposable** fixes -- currently just the TBB signatures. So a reasonable read of it was "TBB is our only build failure", when in fact three whole categories are simply not shown there: - **Unclassified failures** -- anything that doesn't match a seeded signature is routed to human triage and never appears (we've only seeded TBB and libuv signatures). - **Dependency-blocked failures** -- the ~800 RcppParallel dependents (post #121) are still failing; they only show as a log line. - Human-only signatures (libuv). These blind spots are exactly where the *next* signatures should come from, so they deserve the same visibility as the proposals. ## What Extend the feedback-loop tracker to surface the classifier's blind spots: - **`unclassified_summary()`** -- groups every unknown-signature failure by normalised fingerprint, ranked by build count, capped with an explicit `dropped_groups` count (no silent truncation), each with example packages + platforms. These are the candidates for new `build_signatures()` rules. - **`blocked_summary()`** -- lists each dependency (e.g. RcppParallel) and how many dependents wait on it. - `proposal-tracking.R` prints both sections, and a new **`--open-issue`** mode posts/updates a *"Unclassified build failures (needs signatures) (#115)"* Forgejo issue. - The weekly crow pipeline now runs the tracker with `--open-issue`, so it maintains a second tracking issue alongside the proposals one. Read-only on the DB; the only writes are the two issues. ## Verification - New tests cover `unclassified_summary` (ranking + both caps) and `blocked_summary`. - Tracker smoke with a stubbed DB (proposable + blocked + unclassified mix) prints the hit rate, `Blocked on a dependency: RcppParallel: 2 dependent(s)`, and `Unclassified failures ... [2 builds | 2 pkgs] ld: undefined reference ...`. - Full suite: 95 tests pass; all pre-commit hooks pass (air, prettier, markdownlint, yamllint, validate-patches).

pat-s added 1 commit

2026-07-15 07:34:25 +00:00

feat(local): report unclassified and dependency-blocked failures for discovery 6ed17f3fb1

The proposer issue only lists auto-proposable fixes (currently the TBB
signatures), so every failure without a signature -- and every group blocked on
a dependency build -- was invisible unless someone read the run log. That hid
the classifier's blind spots and, with them, which new signatures are worth
adding next.

Extend the tracker to surface those blind spots and give them the same
visibility as the proposals:

- add pure `unclassified_summary()` (unknown-signature groups ranked by build
  count, capped with an explicit dropped count) and `blocked_summary()` helpers
- print both sections in proposal-tracking.R and add a `--open-issue` mode that
  posts/updates a "Unclassified build failures (needs signatures)" Forgejo issue
- run the tracker with --open-issue in the weekly crow pipeline (second issue)
- cover the new helpers with tests and document the flow in the patches README

pat-s merged commit 21a2fe9e6c into main

2026-07-15 07:46:29 +00:00

pat-s deleted branch t3code/report-unclassified-failures

2026-07-15 07:46:30 +00:00

pat-s referenced this pull request from a commit

2026-07-15 07:46:30 +00:00

feat(local): report unclassified and dependency-blocked failures for discovery (#122)