fix(audit): replace arch subsection in place instead of appending duplicates #138

Merged

pat-s merged 1 commit from t3code/audit-id-bug into main

2026-07-21 08:51:23 +00:00

pat-s commented

2026-07-21 08:45:45 +00:00

Owner

Problem

A fresh weekly-audit-missing run followed by weekly-rebuild-missing reported that almost every package "already exists in the remote bucket", even though the audit had just flagged them as missing.

Root cause: the audit's Forgejo-issue update matched the existing ### <arch> subsection by its bare header (### arm64) while writing headers with a (N missing, M to rebuild) suffix.
The equality test never matched, so every run appended a new block instead of replacing the old one.
Issue #63 had accumulated 38 arch subsections under ## alpine-323 where 2 are expected (75 total across the alpine platforms; body ~77k lines).

fetch-rebuild-packages-from-issue.R reads the first matching block, which was the oldest snapshot.
So the rebuild kept re-checking a months-old list (180 packages, mostly already built), while the genuinely-missing packages, ~1921 for alpine-323/arm64 in the freshest block, were never fed to the rebuild and the backlog grew silently.

Changes

local/weekly-missing-binaries-audit.R: match arch subsections by prefix (^### <arch>( |$)) and remove all blocks for that arch before writing one fresh block. Accumulation now self-heals on every run.
local/dedupe-audit-issue.R (new): one-off cleanup that collapses each ## platform section to the freshest block per arch across the three OS-family issues. Supports DRY_RUN=1.
local/fetch-rebuild-packages-from-issue.R: prefer the audit's freshly-written RDS (overwritten each run, immune to issue-body drift), falling back to issue parsing when absent.

Validation

Simulated the dedupe logic against the live #63 body: 75 → 8 arch subsections, body 77k → 28k lines, and the kept alpine-323/arm64 block correctly resolves to GARCH.X (3.0) (the stale first block held 2.0).

Follow-ups (not in this PR)

alpine-321 is audited but has no row in the rebuild matrix (7,230 missing, never rebuilt).
An alpine-324 section exists in the issue but is in neither matrix.

## Problem A fresh `weekly-audit-missing` run followed by `weekly-rebuild-missing` reported that almost every package "already exists in the remote bucket", even though the audit had just flagged them as missing. Root cause: the audit's Forgejo-issue update matched the existing `### <arch>` subsection by its **bare** header (`### arm64`) while writing headers with a ` (N missing, M to rebuild)` suffix. The equality test never matched, so every run **appended** a new block instead of replacing the old one. Issue #63 had accumulated 38 arch subsections under `## alpine-323` where 2 are expected (75 total across the alpine platforms; body ~77k lines). `fetch-rebuild-packages-from-issue.R` reads the **first** matching block, which was the oldest snapshot. So the rebuild kept re-checking a months-old list (180 packages, mostly already built), while the genuinely-missing packages, ~1921 for alpine-323/arm64 in the freshest block, were never fed to the rebuild and the backlog grew silently. ## Changes - **`local/weekly-missing-binaries-audit.R`**: match arch subsections by prefix (`^### <arch>( |$)`) and remove **all** blocks for that arch before writing one fresh block. Accumulation now self-heals on every run. - **`local/dedupe-audit-issue.R`** (new): one-off cleanup that collapses each `## platform` section to the freshest block per arch across the three OS-family issues. Supports `DRY_RUN=1`. - **`local/fetch-rebuild-packages-from-issue.R`**: prefer the audit's freshly-written RDS (overwritten each run, immune to issue-body drift), falling back to issue parsing when absent. ## Validation Simulated the dedupe logic against the live #63 body: **75 → 8** arch subsections, body 77k → 28k lines, and the kept alpine-323/arm64 block correctly resolves to `GARCH.X (3.0)` (the stale first block held `2.0`). ## Follow-ups (not in this PR) - `alpine-321` is audited but has no row in the rebuild matrix (7,230 missing, never rebuilt). - An `alpine-324` section exists in the issue but is in neither matrix.

pat-s added 1 commit

2026-07-21 08:45:45 +00:00

fix(audit): replace arch subsection in place instead of appending duplicates db7afa6319

The weekly audit matched the existing "### <arch>" issue subsection by its
bare header ("### arm64") while writing headers with a
" (N missing, M to rebuild)" suffix. The equality test never matched, so every
run appended a fresh block instead of replacing the old one. Issue bodies
accumulated dozens of stale subsections (alpine-323 had 38 where 2 are
expected), and fetch-rebuild-packages-from-issue.R read the *first* block,
feeding the rebuild a months-old list while the genuinely-missing packages were
never rebuilt.

- Match arch subsections by prefix and drop all blocks for the arch before
  writing one fresh block, so accumulation self-heals each run.
- Add local/dedupe-audit-issue.R to collapse the already-accumulated
  duplicates in the existing issues (keeps the freshest block per arch).
- Make fetch-rebuild-packages-from-issue.R prefer the audit's freshly-written
  RDS, falling back to issue parsing when it is absent.

pat-s merged commit 71f3f0c601 into main

2026-07-21 08:51:23 +00:00

pat-s deleted branch t3code/audit-id-bug

2026-07-21 08:51:24 +00:00

pat-s referenced this pull request from a commit

2026-07-21 08:51:24 +00:00

fix(audit): replace arch subsection in place instead of appending duplicates (#138)