Commit graph build-cran-binaries/cdn.tf
Author SHA1 Message Date
9bded261ee fix(cdn): restore Alliance pull-zone hostname (#166)
All checks were successful
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
## Motivation

Applying #165 recreated the Alliance SwissPass pull zone without its custom hostname because the hostname association was not represented in OpenTofu.
The recreated zone also received a new numeric ID, making the weekly purge configuration stale.

## Changes

- Manage `cran.allianceswisspass.devxy.io` as a pull-zone hostname with TLS and forced HTTPS.
- Resolve the Alliance pull-zone ID from its hostname before purging instead of persisting a replaceable numeric ID.
- Install `jq` in the purge step for the Bunny API lookup.

## Verification

- Targeted `prek` hooks pass.
- `tofu validate` passes.
- `crow lint .crow/` passes.
- `just edge-test` passes all 14 routing steps.
- `bash -n scripts/purge_cdn_zone.sh` passes.

## Deployment

Run `tofu apply` to restore the Alliance hostname on the recreated pull zone.

Reviewed-on: #166
2026-08-13 14:13:04 +00:00
a1c1f5e78f fix(cdn): align repository routing across pull zones (#165)
## Motivation

`cran.rpkgs.com` and `cran.allianceswisspass.devxy.io` serve the same B2 repository through separate Bunny pull zones, but only the first zone was managed and purged after weekly reindexing.
This allowed the Alliance endpoint to retain stale repository metadata and left locked `renv` restores unable to retrieve versions whose binary archive object was absent.

## Changes

- Adopt the Alliance SwissPass pull zone `3265648` into OpenTofu and configure it with the shared B2 origin and middleware script.
- Purge both Bunny pull zones after the weekly rebuild reindex.
- Preserve the requested public hostname in middleware redirects.
- Redirect missing archived binaries to the corresponding CRAN source package, checking whether the version is archived or still current.
- Cover the existing archived-binary passthrough behavior in the edge routing matrix.

## Verification

- `prek run -a`
- `just edge-test`
- `crow lint .crow/`
- `tofu validate`
- `bash -n scripts/purge_cdn_zone.sh`

## Deployment

Run `tofu apply` to adopt pull zone `3265648`, publish the middleware release, and align both pull zones.
After the apply, rerun the Alliance SwissPass CI restore that requested `cli 3.6.5` and `AzureStor 3.7.1`.

Reviewed-on: #165
2026-08-13 14:08:10 +00:00
0c86b2692d fix(cdn): declare the User-Agent cache vary instead of dropping it (#156)
All checks were successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
## Problem

`tofu plan` after importing the middleware script shows:

```
  ~ resource "bunnynet_pullzone" "cran_rpkgs_com" {
      ~ cache_vary_headers = [
          - "User-Agent",
        ]
```

The zone carries `cache_vary_headers = ["User-Agent"]`, set before this configuration existed. `cdn.tf` never declared it, so the first apply of the managed middleware would remove it — as a side effect of an unrelated change, with no decision recorded anywhere.

## What this changes

Declares the attribute with its current value, so the pull zone is a no-op in that plan.

## Why keep it rather than let it go

On paper the router makes it redundant. The only UA-dependent responses it produces are redirects, and those carry `Cache-Control: no-store`; their targets are concrete per-slot, per-minor URLs whose content depends only on the path. Dropping the vary would also be a genuine win, since otherwise every distinct R version string (`R (4.5.3 x86_64-pc-linux-musl …)`) keys its own copy of every tarball.

It stays anyway, for now:

- it is the second line of defence against the one failure mode that would be quiet and confusing — an R 4.6 client served the 4.5 index
- Bunny honouring `no-store` on an edge-script response has been confirmed for today's redirects (`cdn-cache: BYPASS` on `max-age=0`), but not for the new script in production
- keeping it is the status quo, so it cannot regress anything

Removing it is worth doing on its own, once per-minor routing is confirmed live and the redirects can be observed bypassing cache — not as a side effect of enabling that routing.

## Verification

`tofu validate` passes. Re-planning after this merges should leave `bunnynet_pullzone.cran_rpkgs_com` unchanged, reducing the plan to the script `content` update and the new `UNION_SLOTS` variable.

Reviewed-on: #156
2026-08-09 10:08:40 +00:00
2f732457d2 feat(edge): route PACKAGES requests to the per-R-minor slot (#152)
Some checks failed
ci/crow/cron/process-updates/13 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/4 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/3 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/1 Pipeline was successful
ci/crow/cron/process-updates/15 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/10 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/12 Pipeline was successful
ci/crow/cron/process-updates/11 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/16 Pipeline was successful
ci/crow/cron/process-updates/17 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/14 Pipeline was successful
ci/crow/cron/process-updates/4 Pipeline was successful
ci/crow/cron/process-updates/10 Pipeline was successful
ci/crow/cron/process-updates/14 Pipeline was successful
ci/crow/cron/process-updates/5 Pipeline was successful
ci/crow/cron/process-updates/16 Pipeline was successful
ci/crow/cron/process-updates/18 Pipeline was successful
ci/crow/cron/process-updates/12 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/2 Pipeline was successful
ci/crow/cron/process-updates/6 Pipeline was successful
ci/crow/cron/process-updates/1 Pipeline was successful
ci/crow/cron/process-updates/2 Pipeline was successful
ci/crow/cron/process-updates/7 Pipeline was successful
ci/crow/cron/process-updates/8 Pipeline was successful
ci/crow/cron/process-updates/9 Pipeline was successful
ci/crow/cron/process-updates/3 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/17 Pipeline failed
ci/crow/manual/weekly-rebuild-missing/6 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/5 Pipeline was successful
ci/crow/manual/weekly-rebuild-missing/18 Pipeline was successful
## Problem

`install.packages("curl")` fails in `reg.devxy.io/r/r-alpine:4.5-3.24` with "package 'curl' is not available for this version of R", on both arches.

`curl` is not missing from the repo: it is in `…/latest/src/contrib/4.5/` and `…/4.6/`, the per-minor slots that base R cannot address. The image's repo URL resolves to `…/latest/src/contrib`, whose index does not list it. On `amd64/alpine324` that is 2 886 packages invisible to `install.packages()` (23 on `amd64/noble`) — what issue #63 records as "missing binaries".

Two further findings while investigating:

- The middleware only ever rewrote the bare `cran.rpkgs.com/src/contrib/…` form, and that form was broken for every Linux client on a stock R user agent: `ALPINE_REGEX`/`UBUNTU_REGEX`/`RHEL_REGEX` need a Posit-style UA that carries the distro, so stock R fell through to `extractOs()` and got redirected to `/amd64/linux-musl/latest/…`, a slot that does not exist.
- `PACKAGES*` is served `cdn-cache: BYPASS` (bincraft uploads it `no-store`), so the middleware sees every index request and no purge is needed for routing changes to take effect.

## What this changes

**`edge/rpkgs-router.ts`** — the middleware, now a reviewed file in this repo rather than dashboard state. It routes `PACKAGES`, `PACKAGES.gz` and `PACKAGES.rds` into `…/src/contrib/<x.y>/` for slots listed in `UNION_SLOTS`, and nothing else.

Tarballs are deliberately left alone. R keeps the `contriburl` it *asked for*, not the one the redirect served it, so every tarball URL is resolved against the flat directory and the union index steers the per-minor ones with a `Path: <x.y>` field. Rewriting a tarball request here would send flat-slot packages into a directory that does not hold them.

Also in the script: the phantom `linux-gnu`/`linux-musl` fallback is gone (an unidentifiable distro goes to CRAN, as an unparseable UA already did), and every redirect carries `Cache-Control: no-store` since its target depends on the User-Agent. The macOS branches are unchanged.

**`cdn.tf`** — `bunnynet_compute_script.rpkgs_router` with `content = file("edge/rpkgs-router.ts")`, the `UNION_SLOTS` variable, and `middleware_script` pointing at the resource instead of the literal `29277`.

`UNION_SLOTS` is empty, so merging and applying this changes no client's behaviour. A slot is added only once bincraft has republished its per-minor index as a union (rpkgs/bincraft#97); routing to a raw per-minor index would hide every package it does not carry. Rolling back is a variable edit, not a deploy.

**`specs/`, `plans/`** — the design and the implementation plan, including the two approaches that were rejected (edge-side merge, moving the minor up the path) and why.

## Verification

`just edge-test` runs 13 routing cases against the SDK's local server, so what is tested is the artifact that gets deployed; pass-through cases proxy to the real origin. All pass.

End to end, with the middleware in front of a locally built union index for `amd64/alpine324` (31 507 records), inside the runtime image:

```
curl:     7.1.0  -> …/latest/src/contrib/4.5  -> curl_7.1.0.tar.gz      717 725 B
jsonlite: 2.0.0  -> …/latest/src/contrib      -> jsonlite_2.0.0.tar.gz  1 055 849 B
```

`tofu validate` passes. `tofu plan` has not been run: no `BUNNYNET_API_KEY` available in this environment.

## Before applying

The script pre-dates this configuration, so it must be adopted, not created:

```sh
tofu import bunnynet_compute_script.rpkgs_router 29277
tofu plan
```

The plan should show an in-place `content` update and no replacement of the pull zone. Without the import, tofu creates a second script and repoints the zone at it. Note that `name = "rpkgs-router"` will rename the existing script on apply.

## Not fixed here

`install.packages("curl")` on `alpine324` will now *resolve*, then fail to build: that slot's tarballs are byte-identical CRAN **source** tarballs (no `Meta/`, no `Built:` in DESCRIPTION) which the index nevertheless stamps `Built: R 4.5.3; …-linux-musl`. Sampled: `amd64/alpine324` 3/12 binary, `arm64/alpine324` 13/30, `amd64/noble` 12/12, `amd64/alpine323` 17/20. That slot needs a rebuild, tracked separately.

Reviewed-on: #152
2026-08-07 14:12:22 +00:00
a02bd5c4ff
chore: adjust cdn limits
Some checks are pending
ci/crow/cron/process-updates-redhat-8-amd64 Pipeline was successful
ci/crow/cron/process-updates-redhat-9-amd64 Pipeline was successful
ci/crow/cron/process-updates-redhat-8-arm64 Pipeline was successful
ci/crow/cron/process-updates-alpine-323-amd64 Pipeline was successful
ci/crow/cron/process-updates-alpine-323-arm64 Pipeline was successful
ci/crow/cron/process-updates-redhat-9-arm64 Pipeline was successful
ci/crow/cron/process-updates-ubuntu-2204-amd64 Pipeline was successful
ci/crow/cron/process-updates-ubuntu-2204-arm64 Pipeline was successful
ci/crow/cron/process-updates-ubuntu-2404-amd64 Pipeline was successful
ci/crow/cron/process-updates-ubuntu-2404-arm64 Pipeline was successful
ci/crow/cron/process-updates-redhat-10-amd64 Pipeline was successful
ci/crow/cron/process-updates-alpine-322-amd64 Pipeline was successful
ci/crow/cron/process-updates-alpine-322-arm64 Pipeline is running
ci/crow/cron/process-updates-redhat-10-arm64 Pipeline was successful
2026-06-11 14:42:51 +02:00
44482b18f9
chore: cleanup and adjust cdn definitions
All checks were successful
ci/crow/cron/process-updates-ubuntu-2404-amd64 Pipeline was successful
ci/crow/cron/process-updates-ubuntu-2404-arm64 Pipeline was successful
ci/crow/cron/process-updates-redhat-10-amd64 Pipeline was successful
ci/crow/cron/process-updates-redhat-10-arm64 Pipeline was successful
2026-06-11 11:19:26 +02:00
35dafab737 refactor: hoist CDN block_ips to a variable (#81)
## Summary

`block_ips = ["185.172.53.0"]` was duplicated in both `bunnynet_pullzone` resources in `cdn.tf`. Move it to a typed `cdn_block_ips` variable in `vars.tf` with the existing IP as the default.

- No plan diff on apply — same value, just sourced from `var.cdn_block_ips` instead of a literal.
- Adding/removing IPs is now a one-line var override (or a default change) instead of two edits in the resource bodies.

## Interaction with #80 (for_each refactor)

`#80` collapses the two pullzones to `bunnynet_pullzone.this[for_each]`. Either order works; whichever lands second is a trivial one-line rebase on the surviving `block_ips =` line.

Reviewed-on: #81
2026-06-08 08:29:06 +00:00
7892eb0423
chore: remove unused cdns 2026-04-21 19:31:01 +02:00
e959d9743f
refactor: migrate to backblaze
Some checks failed
ci/crow/cron/process-updates-alpine-323-amd64 Pipeline was successful
ci/crow/cron/update-package-index-alpine-321-amd64 Pipeline was canceled
ci/crow/cron/update-package-index-alpine-323-arm64 Pipeline was canceled
ci/crow/cron/update-package-index-redhat-9-arm64 Pipeline was canceled
ci/crow/cron/update-package-index-alpine-322-arm64 Pipeline was canceled
ci/crow/cron/update-package-index-ubuntu-2204-arm64 Pipeline was canceled
ci/crow/cron/update-package-index-redhat-8-amd64 Pipeline was canceled
ci/crow/cron/update-package-index-alpine-322-amd64 Pipeline was canceled
ci/crow/cron/update-package-index-alpine-323-amd64 Pipeline was canceled
ci/crow/cron/update-package-index-ubuntu-2404-arm64 Pipeline was canceled
ci/crow/cron/update-package-index-ubuntu-2404-amd64 Pipeline failed
ci/crow/cron/update-package-index-alpine-321-arm64 Pipeline was canceled
ci/crow/cron/update-package-index-redhat-8-arm64 Pipeline was canceled
ci/crow/cron/update-package-index-redhat-9-amd64 Pipeline was canceled
ci/crow/cron/update-package-index-ubuntu-2204-amd64 Pipeline was canceled
ci/crow/cron/process-updates-ubuntu-2404-amd64 Pipeline was successful
ci/crow/cron/process-updates-ubuntu-2204-arm64 Pipeline failed
ci/crow/cron/process-updates-ubuntu-2404-arm64 Pipeline was successful
ci/crow/cron/process-updates-redhat-9-arm64 Pipeline was successful
ci/crow/cron/process-updates-alpine-323-arm64 Pipeline was successful
ci/crow/cron/process-updates-redhat-8-arm64 Pipeline was successful
2026-03-22 00:32:03 +01:00
bd43869515
feat: import bunny pullzone, add cran.rpkgs.com pullzone config
Some checks failed
ci/crow/cron/update-package-index-alpine-320-amd64 Pipeline failed
ci/crow/cron/update-package-index-ubuntu-2204-amd64 Pipeline failed
ci/crow/cron/update-package-index-alpine-321-arm64 Pipeline failed
ci/crow/cron/update-package-index-redhat-8-arm64 Pipeline failed
ci/crow/cron/update-package-index-alpine-320-arm64 Pipeline failed
ci/crow/cron/update-package-index-alpine-321-amd64 Pipeline failed
ci/crow/cron/update-package-index-redhat-8-amd64 Pipeline failed
ci/crow/cron/update-package-index-redhat-9-amd64 Pipeline failed
ci/crow/cron/update-package-index-ubuntu-2404-amd64 Pipeline failed
ci/crow/cron/update-package-index-ubuntu-2404-arm64 Pipeline failed
ci/crow/cron/update-package-index-redhat-9-arm64 Pipeline failed
ci/crow/cron/update-package-index-ubuntu-2204-arm64 Pipeline failed
ci/crow/cron/process-updates-alpine-321-arm64 Pipeline failed
ci/crow/cron/process-updates-alpine-320-arm64 Pipeline failed
ci/crow/cron/process-updates-redhat-8-arm64 Pipeline failed
ci/crow/cron/process-updates-redhat-8-amd64 Pipeline failed
ci/crow/cron/process-updates-alpine-320-amd64 Pipeline failed
ci/crow/cron/process-updates-ubuntu-2404-amd64 Pipeline failed
ci/crow/cron/process-updates-redhat-9-arm64 Pipeline failed
ci/crow/cron/process-updates-alpine-321-amd64 Pipeline failed
ci/crow/cron/process-updates-ubuntu-2204-amd64 Pipeline failed
2025-05-25 10:45:13 +02:00
576125d377
feat: add docs website 2025-01-03 13:39:38 +01:00