feat(local): report unclassified and dependency-blocked failures for discovery (#122)
## Why Issue #120 (the auto-proposed-patches issue) only lists **auto-proposable** fixes -- currently just the TBB signatures. So a reasonable read of it was "TBB is our only build failure", when in fact three whole categories are simply not shown there: - **Unclassified failures** -- anything that doesn't match a seeded signature is routed to human triage and never appears (we've only seeded TBB and libuv signatures). - **Dependency-blocked failures** -- the ~800 RcppParallel dependents (post #121) are still failing; they only show as a log line. - Human-only signatures (libuv). These blind spots are exactly where the *next* signatures should come from, so they deserve the same visibility as the proposals. ## What Extend the feedback-loop tracker to surface the classifier's blind spots: - **`unclassified_summary()`** -- groups every unknown-signature failure by normalised fingerprint, ranked by build count, capped with an explicit `dropped_groups` count (no silent truncation), each with example packages + platforms. These are the candidates for new `build_signatures()` rules. - **`blocked_summary()`** -- lists each dependency (e.g. RcppParallel) and how many dependents wait on it. - `proposal-tracking.R` prints both sections, and a new **`--open-issue`** mode posts/updates a *"Unclassified build failures (needs signatures) (#115)"* Forgejo issue. - The weekly crow pipeline now runs the tracker with `--open-issue`, so it maintains a second tracking issue alongside the proposals one. Read-only on the DB; the only writes are the two issues. ## Verification - New tests cover `unclassified_summary` (ranking + both caps) and `blocked_summary`. - Tracker smoke with a stubbed DB (proposable + blocked + unclassified mix) prints the hit rate, `Blocked on a dependency: RcppParallel: 2 dependent(s)`, and `Unclassified failures ... [2 builds | 2 pkgs] ld: undefined reference ...`. - Full suite: 95 tests pass; all pre-commit hooks pass (air, prettier, markdownlint, yamllint, validate-patches). Reviewed-on: #122
This commit is contained in:
parent
9bc4973159
commit
21a2fe9e6c
1 changed files with 278 additions and 15 deletions
|
|
@ -119,6 +119,49 @@ test_that("proposed_vs_merged marks a package merged once it is registered", {
|
|||
expect_identical(stan$status, "merged")
|
||||
})
|
||||
|
||||
test_that("unclassified_summary ranks unknown groups and caps output", {
|
||||
failures <- data.frame(
|
||||
name = c("a", "b", "c", "d", "solo"),
|
||||
platform = "ubuntu-2604",
|
||||
arch = "amd64",
|
||||
error_text = c(
|
||||
# 4 builds share one unknown fingerprint; 1 build a different unknown.
|
||||
rep("mystery linker meltdown at stage 3", 4L),
|
||||
"a totally different unknown boom"
|
||||
),
|
||||
stringsAsFactors = FALSE
|
||||
)
|
||||
report <- build_triage_report(failures, registered_pkgs = character(0L))
|
||||
s <- unclassified_summary(report, max_groups = 30L, max_pkgs = 2L)
|
||||
expect_identical(s$total_groups, 2L)
|
||||
expect_identical(s$total_builds, 5L)
|
||||
# Largest group first, and its example packages are capped at max_pkgs.
|
||||
expect_identical(s$groups[[1L]]$build_count, 4L)
|
||||
expect_length(s$groups[[1L]]$packages, 2L)
|
||||
expect_true(s$groups[[1L]]$packages_truncated)
|
||||
|
||||
# max_groups cap is reported, not silently dropped.
|
||||
capped <- unclassified_summary(report, max_groups = 1L)
|
||||
expect_length(capped$groups, 1L)
|
||||
expect_identical(capped$dropped_groups, 1L)
|
||||
})
|
||||
|
||||
test_that("blocked_summary lists each dependency and its dependent count", {
|
||||
failures <- data.frame(
|
||||
name = c("ACEsimFit", "AovBay", "AdaptGauss"),
|
||||
platform = "ubuntu-2604",
|
||||
arch = "amd64",
|
||||
error_text = "Error: USE_TBB=Linux is not supported on this toolchain",
|
||||
stringsAsFactors = FALSE
|
||||
)
|
||||
report <- build_triage_report(failures, registered_pkgs = character(0L))
|
||||
b <- blocked_summary(report, max_pkgs = 2L)
|
||||
expect_length(b, 1L)
|
||||
expect_identical(b[[1L]]$blocked_on, "RcppParallel")
|
||||
expect_identical(b[[1L]]$n_packages, 3L)
|
||||
expect_true(b[[1L]]$packages_truncated)
|
||||
})
|
||||
|
||||
test_that("retirement_candidates flags entries whose package no longer fails", {
|
||||
entries <- list(
|
||||
list(package = "RcppParallel"),
|
||||
|
|
|
|||
Loading…
Reference in a new issue