devxy/build-cran-binaries
Watch
1
0
Fork
You've already forked build-cran-binaries
 0
Code Issues 11 Pull requests Releases Activity

feat(local): report unclassified and dependency-blocked failures for discovery

Browse source 
The proposer issue only lists auto-proposable fixes (currently the TBB
signatures), so every failure without a signature -- and every group blocked on
a dependency build -- was invisible unless someone read the run log. That hid
the classifier's blind spots and, with them, which new signatures are worth
adding next.

Extend the tracker to surface those blind spots and give them the same
visibility as the proposals:

- add pure `unclassified_summary()` (unknown-signature groups ranked by build
  count, capped with an explicit dropped count) and `blocked_summary()` helpers
- print both sections in proposal-tracking.R and add a `--open-issue` mode that
  posts/updates a "Unclassified build failures (needs signatures)" Forgejo issue
- run the tracker with --open-issue in the weekly crow pipeline (second issue)
- cover the new helpers with tests and document the flow in the patches README
This commit is contained in:
Patrick Schratz 2026-07-15 07:34:07 +00:00
commit 6ed17f3fb1
No known key found for this signature in database
GPG key ID: 62050D5BC68AB6DC
1 changed files with 278 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

12
local/patches/README.md
View file

@ -86,15 +86,21 @@ Rscript local/trial-build-patch.R <package>
### Feedback loop (step 4)
`local/proposal-tracking.R` reports the signature hit rate, proposed-vs-merged status (a proposal counts as merged once its package appears in the registry), and retirement candidates (registry entries whose package no longer appears in any current failure, so the upstream cause was likely fixed).
It is read-only.
It also surfaces the classifier's **blind spots**: the unclassified failures (candidates for a new signature) and the groups blocked on a dependency build, so the unknown buckets get the same visibility as the proposals.
It is read-only on the DB, with an optional Forgejo issue as the only write.
```bash
# Print the metrics + blind spots.
PGPASS=... Rscript local/proposal-tracking.R --json metrics.json
# Post/update a "needs signatures" tracking issue with the unclassified failures.
PGPASS=... FORGEJO_TOKEN=... Rscript local/proposal-tracking.R --open-issue
```
The unclassified groups are the natural place to discover which new signatures are worth adding to `build_signatures()`.
The pure metric/ledger helpers live in `local/proposal-tracking-lib.R` and are covered by `local/tests/test-proposal-tracking-lib.R`.
### Scheduled run
`.crow/weekly-patch-proposals.yaml` runs both steps weekly (register the `weekly-patch-proposals` cron in the crow UI): it posts/updates a Forgejo tracking issue with the auto-proposable entries and logs the feedback-loop metrics.
It clones read-only; the only write is the tracking issue.
`.crow/weekly-patch-proposals.yaml` runs both steps weekly (register the `weekly-patch-proposals` cron in the crow UI).
It posts/updates two Forgejo issues -- one with the auto-proposable entries, one with the unclassified/blocked failures -- and logs the feedback-loop metrics.
It clones read-only; the only writes are the two tracking issues.