feat(local): report unclassified and dependency-blocked failures for discovery (#122)
All checks were successful
ci/crow/manual/weekly-patch-proposals Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful

## Why

Issue #120 (the auto-proposed-patches issue) only lists **auto-proposable** fixes -- currently just the TBB signatures. So a reasonable read of it was "TBB is our only build failure", when in fact three whole categories are simply not shown there:

- **Unclassified failures** -- anything that doesn't match a seeded signature is routed to human triage and never appears (we've only seeded TBB and libuv signatures).
- **Dependency-blocked failures** -- the ~800 RcppParallel dependents (post #121) are still failing; they only show as a log line.
- Human-only signatures (libuv).

These blind spots are exactly where the *next* signatures should come from, so they deserve the same visibility as the proposals.

## What

Extend the feedback-loop tracker to surface the classifier's blind spots:

- **`unclassified_summary()`** -- groups every unknown-signature failure by normalised fingerprint, ranked by build count, capped with an explicit `dropped_groups` count (no silent truncation), each with example packages + platforms. These are the candidates for new `build_signatures()` rules.
- **`blocked_summary()`** -- lists each dependency (e.g. RcppParallel) and how many dependents wait on it.
- `proposal-tracking.R` prints both sections, and a new **`--open-issue`** mode posts/updates a *"Unclassified build failures (needs signatures) (#115)"* Forgejo issue.
- The weekly crow pipeline now runs the tracker with `--open-issue`, so it maintains a second tracking issue alongside the proposals one. Read-only on the DB; the only writes are the two issues.

## Verification

- New tests cover `unclassified_summary` (ranking + both caps) and `blocked_summary`.
- Tracker smoke with a stubbed DB (proposable + blocked + unclassified mix) prints the hit rate, `Blocked on a dependency: RcppParallel: 2 dependent(s)`, and `Unclassified failures ... [2 builds | 2 pkgs] ld: undefined reference ...`.
- Full suite: 95 tests pass; all pre-commit hooks pass (air, prettier, markdownlint, yamllint, validate-patches).

Reviewed-on: #122
This commit is contained in:
Patrick Schratz 2026-07-15 07:46:29 +00:00 committed by Patrick Schratz
commit 21a2fe9e6c

View file

@ -86,15 +86,21 @@ Rscript local/trial-build-patch.R <package>
### Feedback loop (step 4)
`local/proposal-tracking.R` reports the signature hit rate, proposed-vs-merged status (a proposal counts as merged once its package appears in the registry), and retirement candidates (registry entries whose package no longer appears in any current failure, so the upstream cause was likely fixed).
It is read-only.
It also surfaces the classifier's **blind spots**: the unclassified failures (candidates for a new signature) and the groups blocked on a dependency build, so the unknown buckets get the same visibility as the proposals.
It is read-only on the DB, with an optional Forgejo issue as the only write.
```bash
# Print the metrics + blind spots.
PGPASS=... Rscript local/proposal-tracking.R --json metrics.json
# Post/update a "needs signatures" tracking issue with the unclassified failures.
PGPASS=... FORGEJO_TOKEN=... Rscript local/proposal-tracking.R --open-issue
```
The unclassified groups are the natural place to discover which new signatures are worth adding to `build_signatures()`.
The pure metric/ledger helpers live in `local/proposal-tracking-lib.R` and are covered by `local/tests/test-proposal-tracking-lib.R`.
### Scheduled run
`.crow/weekly-patch-proposals.yaml` runs both steps weekly (register the `weekly-patch-proposals` cron in the crow UI): it posts/updates a Forgejo tracking issue with the auto-proposable entries and logs the feedback-loop metrics.
It clones read-only; the only write is the tracking issue.
`.crow/weekly-patch-proposals.yaml` runs both steps weekly (register the `weekly-patch-proposals` cron in the crow UI).
It posts/updates two Forgejo issues -- one with the auto-proposable entries, one with the unclassified/blocked failures -- and logs the feedback-loop metrics.
It clones read-only; the only writes are the two tracking issues.