feat(local): detect dependency-cascade failures generally, not just RcppParallel #128

Merged
pat-s merged 1 commit from t3code/detect-dependency-cascades into main 2026-07-16 08:23:55 +00:00
Owner

Why (from the #127 trial-build gate)

The gate did its job: 0/3 passed, merge blocked. The log showed why -- BFpack, BayesERtools, GMLTM all fail while building their shared dependency rstan, not in their own code:

Failed to build source package rstan.
  .../StanHeaders/include/stan/math/prim/core/init_threadpool_tbb.hpp:9:10:
  fatal error: tbb/tbb_stddef.h: No such file or directory

So the per-package -DTBB_INTERFACE_NEW makevars entries the classifier proposed are useless for these packages -- they're blocked on rstan (which already has a registry entry). This is the same dependency cascade the RcppParallel applies_to guard catches, but tbb-stddef-removed is a generic signature with no such pin, so ~73 Stan packages kept getting proposed.

What

Generalise cascade detection beyond the RcppParallel special case:

  • failing_dependency(error_text, package) -- when the log names a different package as the one that failed to compile (Failed to build source package X, compilation failed for package 'X', dependency 'X' ... not available), that package is the real cause.
  • build_triage_report() now blocks any package whose every failing build is such a cascade: reported as blocked_on that dependency, never proposed a bogus per-package entry. A package that fails in its own compilation is still proposed.
  • The applies_to (RcppParallel) and data-driven (rstan) cases are unified into one blocked_packages / blocked_on model; the report, proposer, and blocked_summary count the actually-blocked packages, and the blocked note shows even when a group also has genuine proposals.

Effect

Next auto-apply run will stop proposing the rstan-blocked Stan packages (and any future dependency cascade) and surface them as "blocked on rstan" instead. Fixing rstan once clears the whole cluster.

Verification

  • New tests: failing_dependency (cascade vs own-compile vs none), and an end-to-end split where BFpack/GMLTM (blocked on rstan) are not proposed while an own-compile package still is.
  • Full suite: 112 tests pass; all pre-commit hooks pass.

Refs #120, #127. (Separate follow-ups: fixing rstan's build itself, and quieting the gate's metadata-DB retry storm -- both root-caused to bincraft.)

## Why (from the #127 trial-build gate) The gate did its job: 0/3 passed, merge blocked. The log showed *why* -- BFpack, BayesERtools, GMLTM all fail while building their shared dependency **`rstan`**, not in their own code: ``` Failed to build source package rstan. .../StanHeaders/include/stan/math/prim/core/init_threadpool_tbb.hpp:9:10: fatal error: tbb/tbb_stddef.h: No such file or directory ``` So the per-package `-DTBB_INTERFACE_NEW` makevars entries the classifier proposed are useless for these packages -- they're blocked on `rstan` (which already has a registry entry). This is the **same dependency cascade** the RcppParallel `applies_to` guard catches, but `tbb-stddef-removed` is a generic signature with no such pin, so ~73 Stan packages kept getting proposed. ## What Generalise cascade detection beyond the RcppParallel special case: - `failing_dependency(error_text, package)` -- when the log names a **different** package as the one that failed to compile (`Failed to build source package X`, `compilation failed for package 'X'`, `dependency 'X' ... not available`), that package is the real cause. - `build_triage_report()` now blocks any package whose **every** failing build is such a cascade: reported as `blocked_on` that dependency, never proposed a bogus per-package entry. A package that fails in its **own** compilation is still proposed. - The `applies_to` (RcppParallel) and data-driven (rstan) cases are unified into one `blocked_packages` / `blocked_on` model; the report, proposer, and `blocked_summary` count the actually-blocked packages, and the blocked note shows even when a group also has genuine proposals. ## Effect Next auto-apply run will stop proposing the rstan-blocked Stan packages (and any future dependency cascade) and surface them as "blocked on rstan" instead. Fixing `rstan` once clears the whole cluster. ## Verification - New tests: `failing_dependency` (cascade vs own-compile vs none), and an end-to-end split where BFpack/GMLTM (blocked on rstan) are not proposed while an own-compile package still is. - Full suite: 112 tests pass; all pre-commit hooks pass. Refs #120, #127. (Separate follow-ups: fixing rstan's build itself, and quieting the gate's metadata-DB retry storm -- both root-caused to bincraft.)
The first trial-build gate run correctly went red: the 10 auto-proposed
tbb-stddef packages (BFpack, BayesERtools, GMLTM, ...) all fail while building
their shared `rstan` dependency, not in their own code -- the `tbb/tbb_stddef.h`
error is in rstan's Module.cpp. So a per-package makevars entry is useless for
them; they are blocked on rstan (which already has an entry). This is the same
cascade the RcppParallel `applies_to` guard catches, but the tbb-stddef
signature is generic and had no such pin.

Add `failing_dependency(error_text, package)`: when the log shows a DIFFERENT
package failed to compile ("Failed to build source package X", "compilation
failed for package 'X'", ...), that package is the real cause. build_triage_report
now blocks any package whose every failing build is such a cascade -- reported
as blocked_on the dependency, never proposed a bogus entry. A package that fails
in its OWN compilation is still proposed. This unifies the applies_to and
data-driven cases into one `blocked_packages` / `blocked_on` model.

- report/proposer/blocked_summary now count the actually-blocked packages
- blocked note shows even when a group also has genuine proposals
- tests cover the rstan-cascade vs own-compile split (with the real log strings)
pat-s merged commit f9d399fac0 into main 2026-07-16 08:23:55 +00:00
pat-s deleted branch t3code/detect-dependency-cascades 2026-07-16 08:23:56 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
devxy/build-cran-binaries!128
No description provided.