docs: add design spec for weekly missing binaries audit & rebuild workflow
This commit is contained in:
parent
14950a6bfc
commit
154a517cfc
GPG key ID:
3C6318841EF78925
1 changed files with 246 additions and 0 deletions
|
|
@ -0,0 +1,246 @@
|
|||
# Weekly Missing Binaries Audit & Rebuild
|
||||
|
||||
## Goal
|
||||
|
||||
A weekly CI workflow that identifies CRAN packages whose latest release version has no binary available, reports them in Forgejo issues grouped by OS family and arch, and rebuilds those that have no prior build failure recorded in the database.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
Two independent workflow sets, each with one file per platform/arch combo (14 files each), plus a shared R script and an excluded-packages config file.
|
||||
|
||||
```
|
||||
weekly-audit-missing-*-*.yaml (14 files)
|
||||
│
|
||||
▼
|
||||
local/weekly-missing-binaries-audit.R
|
||||
│
|
||||
├── Updates Forgejo issues (3 issues, one per OS family)
|
||||
└── Writes RDS files to /mnt/cache/packages/
|
||||
|
||||
weekly-rebuild-missing-*-*.yaml (14 files)
|
||||
│
|
||||
▼
|
||||
Reads RDS, builds missing packages via bincraft::build_binary_package()
|
||||
```
|
||||
|
||||
The audit and build workflows are fully independent.
|
||||
Either can be triggered on its own via cron or manually.
|
||||
|
||||
## Platforms
|
||||
|
||||
All current platform/arch combinations:
|
||||
|
||||
| Platform | Arch | Image | Codename |
|
||||
|-------------|-------|-------------------------------------------------|------------|
|
||||
| ubuntu-2204 | amd64 | reg.devxy.io/rpkgs/build-env-ubuntu:jammy-4.4.3 | jammy |
|
||||
| ubuntu-2204 | arm64 | reg.devxy.io/rpkgs/build-env-ubuntu:jammy-4.4.3 | jammy |
|
||||
| ubuntu-2404 | amd64 | reg.devxy.io/rpkgs/build-env-ubuntu:noble-4.4.3 | noble |
|
||||
| ubuntu-2404 | arm64 | reg.devxy.io/rpkgs/build-env-ubuntu:noble-4.4.3 | noble |
|
||||
| alpine-321 | amd64 | reg.devxy.io/rpkgs/build-env-alpine:3.21-4.5 | alpine321 |
|
||||
| alpine-321 | arm64 | reg.devxy.io/rpkgs/build-env-alpine:3.21-4.5 | alpine321 |
|
||||
| alpine-322 | amd64 | reg.devxy.io/rpkgs/build-env-alpine:3.22-4.5 | alpine322 |
|
||||
| alpine-322 | arm64 | reg.devxy.io/rpkgs/build-env-alpine:3.22-4.5 | alpine322 |
|
||||
| alpine-323 | amd64 | reg.devxy.io/rpkgs/build-env-alpine:3.23-4.5 | alpine323 |
|
||||
| alpine-323 | arm64 | reg.devxy.io/rpkgs/build-env-alpine:3.23-4.5 | alpine323 |
|
||||
| redhat-8 | amd64 | reg.devxy.io/rpkgs/build-env-redhat:8-4.4.3 | rhel8 |
|
||||
| redhat-8 | arm64 | reg.devxy.io/rpkgs/build-env-redhat:8-4.4.3 | rhel8 |
|
||||
| redhat-9 | amd64 | reg.devxy.io/rpkgs/build-env-redhat:9-4.4.3 | rhel9 |
|
||||
| redhat-9 | arm64 | reg.devxy.io/rpkgs/build-env-redhat:9-4.4.3 | rhel9 |
|
||||
|
||||
## Component 1: Excluded Packages Config
|
||||
|
||||
**File:** `local/excluded-packages.json`
|
||||
|
||||
A JSON array of objects with `package` and `reason` fields:
|
||||
|
||||
```json
|
||||
[
|
||||
{"package": "RInno", "reason": "windows-only"},
|
||||
{"package": "KeyboardSimulator", "reason": "windows-only"},
|
||||
{"package": "doBy", "reason": "hang"},
|
||||
{"package": "frailtypack", "reason": "hang"},
|
||||
...
|
||||
]
|
||||
```
|
||||
|
||||
This file is the single source of truth for packages that should be skipped.
|
||||
Both the audit script and rebuild workflows read from it.
|
||||
|
||||
The existing `build-all-versions-*.yaml` workflows retain their inline lists for now (migration is out of scope).
|
||||
|
||||
## Component 2: Audit R Script
|
||||
|
||||
**File:** `local/weekly-missing-binaries-audit.R`
|
||||
|
||||
**Environment variables consumed:**
|
||||
- `PLATFORM` — e.g. `ubuntu-2204`, `alpine-321`, `redhat-9`
|
||||
- `ARCH` — `amd64` or `arm64`
|
||||
- `B2_S3_ACCESS_KEY`, `B2_S3_SECRET_KEY` — S3 credentials
|
||||
- `PGPASS` — PostgreSQL password
|
||||
- `FORGEJO_TOKEN` — API token for issue updates
|
||||
|
||||
The workflow commands (not the R script) also use:
|
||||
- `REPO_RO_TOKEN` — for `git clone` in the workflow commands
|
||||
|
||||
**Logic:**
|
||||
|
||||
1. Parse `PLATFORM` to derive OS family (`Ubuntu`, `Alpine`, `Red Hat`) and S3 codename (e.g. `ubuntu-2204` -> `jammy`, `redhat-9` -> `rhel9`).
|
||||
2. Fetch CRAN release packages via `tools::CRAN_package_db()` — extract `Package` and `Version`.
|
||||
3. List S3 tarballs at `devxy-rpkgs-binaries/{arch}/{codename}/latest/src/contrib/` and parse `{name}_{version}.tar.gz`.
|
||||
4. Find packages where the CRAN release version is missing from S3.
|
||||
5. Read `local/excluded-packages.json` and remove those packages from the missing list.
|
||||
6. Query the `single_builds` DB table: for each missing package+version+platform+arch, check if `error_occurred = TRUE`. Split into:
|
||||
- **Rebuildable:** missing, not excluded, no prior failure for this version
|
||||
- **Known failures:** missing, not excluded, but has a recorded failure for this version
|
||||
7. Write the rebuildable package list (names only) to `/mnt/cache/packages/weekly_rebuild_{platform}_{arch}.rds`.
|
||||
8. Update the Forgejo issue for this OS family.
|
||||
|
||||
**Issue update logic:**
|
||||
|
||||
- Issue title: `Missing package binaries for latest version (<OS family>)` where OS family is `Ubuntu`, `Alpine`, or `Red Hat`.
|
||||
- Search for existing open issue via `GET /api/v1/repos/devxy/build-cran-binaries/issues?type=issues&state=open&q=<url-encoded title>`. Match by exact title.
|
||||
- If found, read the existing body, replace the section for this platform/arch, and `PATCH` the issue.
|
||||
- If not found, `POST` a new issue with just this platform/arch section.
|
||||
|
||||
**Issue body format:**
|
||||
|
||||
```markdown
|
||||
_Last updated: 2026-04-11_
|
||||
|
||||
## ubuntu-2204
|
||||
|
||||
### amd64 (12 missing, 8 to rebuild)
|
||||
- ggplot2 (3.5.2)
|
||||
- dplyr (1.1.5)
|
||||
- ...
|
||||
|
||||
#### Known build failures
|
||||
- somepkg (1.0.0)
|
||||
|
||||
### arm64 (5 missing, 5 to rebuild)
|
||||
- ...
|
||||
|
||||
## ubuntu-2404
|
||||
|
||||
### amd64 (3 missing, 3 to rebuild)
|
||||
- ...
|
||||
|
||||
### arm64 (0 missing)
|
||||
All binaries available.
|
||||
|
||||
---
|
||||
|
||||
## Excluded packages
|
||||
doBy (hang), frailtypack (hang), RInno (windows-only), ...
|
||||
```
|
||||
|
||||
Each audit workflow run updates only its own platform/arch section within the issue.
|
||||
The "Excluded packages" section and "Last updated" timestamp are rewritten on every run.
|
||||
|
||||
**Section replacement strategy:**
|
||||
The script parses the existing issue body as markdown, finds the `## {platform}` + `### {arch}` section, replaces it, and writes back the full body.
|
||||
If the section doesn't exist yet, it's appended under the correct `## {platform}` header (or a new one is created).
|
||||
|
||||
## Component 3: Audit Workflows
|
||||
|
||||
**Files:** 14 files, named `weekly-audit-missing-{platform}-{arch}.yaml`
|
||||
|
||||
Example: `.crow/weekly-audit-missing-ubuntu-2204-amd64.yaml`
|
||||
|
||||
**Trigger:**
|
||||
```yaml
|
||||
when:
|
||||
- event: cron
|
||||
cron: weekly-audit-missing-ubuntu-2204-amd64
|
||||
- event: manual
|
||||
evaluate: 'task == "weekly-audit-missing-ubuntu-2204-amd64"'
|
||||
```
|
||||
|
||||
**Step:** Lightweight — clones repo, installs bincraft + dependencies, runs the audit R script.
|
||||
|
||||
**Container:** Uses the platform-appropriate build image (needed for correct platform identification), but with minimal resource requests since no building happens.
|
||||
|
||||
**Resources:** ~2Gi memory, 2 CPUs.
|
||||
|
||||
## Component 4: Rebuild Workflows
|
||||
|
||||
**Files:** 14 files, named `weekly-rebuild-missing-{platform}-{arch}.yaml`
|
||||
|
||||
Example: `.crow/weekly-rebuild-missing-ubuntu-2204-amd64.yaml`
|
||||
|
||||
**Trigger:**
|
||||
```yaml
|
||||
when:
|
||||
- event: cron
|
||||
cron: weekly-rebuild-missing-ubuntu-2204-amd64
|
||||
- event: manual
|
||||
evaluate: 'task == "weekly-rebuild-missing-ubuntu-2204-amd64"'
|
||||
```
|
||||
|
||||
**Step:**
|
||||
|
||||
1. Clone repo, install bincraft.
|
||||
2. Read `local/excluded-packages.json` as a safety net.
|
||||
3. Read `/mnt/cache/packages/weekly_rebuild_{platform}_{arch}.rds`. If missing or empty, exit 0.
|
||||
4. Filter out excluded packages (double-check).
|
||||
5. Iterate and call `bincraft::build_binary_package()` for each package with `tag_limit = 1L`.
|
||||
Same S3/DB parameters as existing build workflows.
|
||||
6. Uses `xvfb-run` / `xwfb-run` for graphical packages (same pattern as existing builds).
|
||||
|
||||
**Resources:** Same as `process-updates` workflows — 5Gi request, 18Gi limit, 3 CPUs.
|
||||
|
||||
**Cache volume:** Maps `${ARCH}-binaries-r-dep-cache-${PLATFORM}:/mnt/cache` (same volumes as existing builds, so the RDS files written by audit are visible).
|
||||
|
||||
## Secrets Required
|
||||
|
||||
All existing secrets are reused:
|
||||
- `B2_S3_ACCESS_KEY`, `B2_S3_SECRET_KEY` — S3 access
|
||||
- `PGPASS` — PostgreSQL
|
||||
- `REPO_RO_TOKEN` — Git clone
|
||||
- `GITHUB_PAT` — For bincraft GitHub mirror access
|
||||
|
||||
New secret needed:
|
||||
- `FORGEJO_TOKEN` — API token for creating/updating issues on git.devxy.io
|
||||
|
||||
## Cron Schedule
|
||||
|
||||
The audit and rebuild workflows each get their own cron names.
|
||||
The cron schedule itself is configured in the Crow/Woodpecker server, not in the YAML.
|
||||
Intended cadence: once per week (e.g. Sunday morning).
|
||||
|
||||
## File Inventory
|
||||
|
||||
| File | Type | Description |
|
||||
|------|------|-------------|
|
||||
| `local/excluded-packages.json` | Config | Excluded packages with reasons |
|
||||
| `local/weekly-missing-binaries-audit.R` | R script | Audit logic, parameterized by env vars |
|
||||
| `.crow/weekly-audit-missing-ubuntu-2204-amd64.yaml` | Workflow | Audit for ubuntu-2204/amd64 |
|
||||
| `.crow/weekly-audit-missing-ubuntu-2204-arm64.yaml` | Workflow | Audit for ubuntu-2204/arm64 |
|
||||
| `.crow/weekly-audit-missing-ubuntu-2404-amd64.yaml` | Workflow | Audit for ubuntu-2404/amd64 |
|
||||
| `.crow/weekly-audit-missing-ubuntu-2404-arm64.yaml` | Workflow | Audit for ubuntu-2404/arm64 |
|
||||
| `.crow/weekly-audit-missing-alpine-321-amd64.yaml` | Workflow | Audit for alpine-321/amd64 |
|
||||
| `.crow/weekly-audit-missing-alpine-321-arm64.yaml` | Workflow | Audit for alpine-321/arm64 |
|
||||
| `.crow/weekly-audit-missing-alpine-322-amd64.yaml` | Workflow | Audit for alpine-322/amd64 |
|
||||
| `.crow/weekly-audit-missing-alpine-322-arm64.yaml` | Workflow | Audit for alpine-322/arm64 |
|
||||
| `.crow/weekly-audit-missing-alpine-323-amd64.yaml` | Workflow | Audit for alpine-323/amd64 |
|
||||
| `.crow/weekly-audit-missing-alpine-323-arm64.yaml` | Workflow | Audit for alpine-323/arm64 |
|
||||
| `.crow/weekly-audit-missing-redhat-8-amd64.yaml` | Workflow | Audit for redhat-8/amd64 |
|
||||
| `.crow/weekly-audit-missing-redhat-8-arm64.yaml` | Workflow | Audit for redhat-8/arm64 |
|
||||
| `.crow/weekly-audit-missing-redhat-9-amd64.yaml` | Workflow | Audit for redhat-9/amd64 |
|
||||
| `.crow/weekly-audit-missing-redhat-9-arm64.yaml` | Workflow | Audit for redhat-9/arm64 |
|
||||
| `.crow/weekly-rebuild-missing-ubuntu-2204-amd64.yaml` | Workflow | Rebuild for ubuntu-2204/amd64 |
|
||||
| `.crow/weekly-rebuild-missing-ubuntu-2204-arm64.yaml` | Workflow | Rebuild for ubuntu-2204/arm64 |
|
||||
| `.crow/weekly-rebuild-missing-ubuntu-2404-amd64.yaml` | Workflow | Rebuild for ubuntu-2404/amd64 |
|
||||
| `.crow/weekly-rebuild-missing-ubuntu-2404-arm64.yaml` | Workflow | Rebuild for ubuntu-2404/arm64 |
|
||||
| `.crow/weekly-rebuild-missing-alpine-321-amd64.yaml` | Workflow | Rebuild for alpine-321/amd64 |
|
||||
| `.crow/weekly-rebuild-missing-alpine-321-arm64.yaml` | Workflow | Rebuild for alpine-321/arm64 |
|
||||
| `.crow/weekly-rebuild-missing-alpine-322-amd64.yaml` | Workflow | Rebuild for alpine-322/amd64 |
|
||||
| `.crow/weekly-rebuild-missing-alpine-322-arm64.yaml` | Workflow | Rebuild for alpine-322/arm64 |
|
||||
| `.crow/weekly-rebuild-missing-alpine-323-amd64.yaml` | Workflow | Rebuild for alpine-323/amd64 |
|
||||
| `.crow/weekly-rebuild-missing-alpine-323-arm64.yaml` | Workflow | Rebuild for alpine-323/arm64 |
|
||||
| `.crow/weekly-rebuild-missing-redhat-8-amd64.yaml` | Workflow | Rebuild for redhat-8/amd64 |
|
||||
| `.crow/weekly-rebuild-missing-redhat-8-arm64.yaml` | Workflow | Rebuild for redhat-8/arm64 |
|
||||
| `.crow/weekly-rebuild-missing-redhat-9-amd64.yaml` | Workflow | Rebuild for redhat-9/amd64 |
|
||||
| `.crow/weekly-rebuild-missing-redhat-9-arm64.yaml` | Workflow | Rebuild for redhat-9/arm64 |
|
||||
|
||||
**Total: 29 new files** (1 JSON config + 1 R script + 14 audit workflows + 14 rebuild workflows + the design doc itself)
|
||||
Loading…
Reference in a new issue