pat-s
85295a9495
## Motivation
Every reindex reports `failure` at the purge step:
```
Purging BunnyCDN pull zone 3857050
Purged pull zone 3857050 (HTTP 204)
jq: error (at /tmp/tmp.eFPFmO:0): Cannot index array with string "Items"
Could not find BunnyCDN pull zone for hostname cran.allianceswisspass.devxy.io
```
`cran.rpkgs.com` purges fine. The Alliance zone never has, so it is still serving objects that rebuilds replaced, behind a ~370-day `cache_expiration_time`.
## The defect
```sh
jq -r '(.Items // .)[] | ...'
```
This was meant to accept both response shapes. It accepts neither: indexing an array with a string is an **error** in jq, not a null, so `//` never gets the chance to substitute and the whole expression aborts. The listing endpoint answers with a bare array for this account, so the lookup has always failed.
## Change
- Select the array explicitly by type instead of relying on `//` to absorb an error.
- Check the HTTP status of the listing call. It was previously used unconditionally, so an auth or rate-limit failure surfaced as "could not find hostname" — pointing at the wrong thing entirely.
- Fail when a hostname matches multiple zones rather than silently purging whichever jq emitted first.
- Request `perPage=1000`, so a paginated response cannot silently truncate the zone list.
## Verification
Ran the current `main` script and the fixed one against a stubbed `curl` returning an array-shaped listing:
```
=== BEFORE (main) ===
Purged pull zone 3857050 (HTTP 204)
jq: error (at ...): Cannot index array with string ("Items")
Could not find BunnyCDN pull zone for hostname cran.allianceswisspass.devxy.io
=== AFTER ===
Purged pull zone 3857050 (HTTP 204)
Purging BunnyCDN pull zone 222
Purged pull zone 222 (HTTP 204)
```
The jq expression was also checked against both an array-shaped and an object-shaped (`.Items`) response; the old one fails the array case, the new one handles both. `shellcheck` clean.
Reviewed-on: #178
112 lines
3.5 KiB
Shell
Executable file
112 lines
3.5 KiB
Shell
Executable file
#!/usr/bin/env bash
|
|
#
|
|
# Purge the entire BunnyCDN pull zone.
|
|
#
|
|
# `purge_cdn_cache.sh` purges the five index files by URL, which is right after
|
|
# a normal update: new packages arrive at new URLs, so only the index is stale.
|
|
#
|
|
# A rebuild is different. It replaces an object *in place*: a package whose
|
|
# build failed was published as its CRAN source, and the rebuilt binary takes
|
|
# exactly the same URL. The zone caches tarballs for ~370 days
|
|
# (`cache_expiration_time` in cdn.tf), so without a purge every client keeps
|
|
# receiving the source tarball for up to a year, and nothing about it looks
|
|
# wrong from the outside.
|
|
#
|
|
# Purging per URL would mean one API call per replaced package -- ~13.5k per
|
|
# arch against a rate-limited endpoint, where a single missed call leaves a
|
|
# silently stale package. One zone purge is a single call regardless of how many
|
|
# objects were replaced. The cost is a cold cache for everything else, which is
|
|
# why this is not used by the daily update path.
|
|
#
|
|
# The public hostnames currently use separate pull zones, so callers must pass
|
|
# every zone that serves the repository. A zone can be identified by its
|
|
# numeric ID or by one of its hostnames; hostname lookup avoids persisting IDs
|
|
# that change when a zone is recreated.
|
|
#
|
|
# Usage:
|
|
# purge_cdn_zone.sh <BUNNYNET_API_KEY> <pull_zone> [<pull_zone>...]
|
|
#
|
|
set -euo pipefail
|
|
|
|
if (($# < 2)); then
|
|
echo "usage: $0 <api_key> <pull_zone> [<pull_zone>...]" >&2
|
|
exit 2
|
|
fi
|
|
|
|
api_key="$1"
|
|
shift
|
|
|
|
resolve_zone_id() {
|
|
local zone="$1"
|
|
local response_file
|
|
local zone_id
|
|
|
|
if [[ "${zone}" =~ ^[0-9]+$ ]]; then
|
|
echo "${zone}"
|
|
return
|
|
fi
|
|
|
|
response_file=$(mktemp)
|
|
local status
|
|
status=$(
|
|
curl -sS -o "${response_file}" -w '%{http_code}' \
|
|
-H "AccessKey: ${api_key}" \
|
|
"https://api.bunny.net/pullzone?perPage=1000"
|
|
)
|
|
|
|
if [[ "${status}" != "200" ]]; then
|
|
echo "Listing BunnyCDN pull zones failed with HTTP ${status}:" >&2
|
|
head -c 500 "${response_file}" >&2
|
|
echo >&2
|
|
rm -f "${response_file}"
|
|
exit 1
|
|
fi
|
|
|
|
# The endpoint answers with a bare array on some accounts and a paginated
|
|
# object on others. `.Items // .` looks like it covers both but does not:
|
|
# indexing an array with a string is an *error*, and `//` only substitutes
|
|
# for null, so the array case aborted with
|
|
# "Cannot index array with string" and the zone was never purged.
|
|
zone_id=$(
|
|
jq -r --arg hostname "${zone}" \
|
|
'(if type == "object" then (.Items // []) else . end)[]
|
|
| select(any(.Hostnames[]?; .Value == $hostname))
|
|
| .Id' \
|
|
"${response_file}"
|
|
)
|
|
rm -f "${response_file}"
|
|
|
|
if [[ -z "${zone_id}" ]]; then
|
|
echo "Could not find BunnyCDN pull zone for hostname ${zone}" >&2
|
|
exit 1
|
|
fi
|
|
|
|
# Two zones sharing a hostname would purge only whichever jq emitted first.
|
|
if [[ $(wc -l <<<"${zone_id}") -gt 1 ]]; then
|
|
echo "Hostname ${zone} matched multiple pull zones: ${zone_id//$'\n'/ }" >&2
|
|
exit 1
|
|
fi
|
|
|
|
echo "${zone_id}"
|
|
}
|
|
|
|
for zone in "$@"; do
|
|
zone_id=$(resolve_zone_id "${zone}")
|
|
echo "Purging BunnyCDN pull zone ${zone_id}"
|
|
|
|
response_file="/tmp/purge_zone_response_${zone_id}.txt"
|
|
status=$(
|
|
curl -sS -o "${response_file}" -w '%{http_code}' -X POST \
|
|
-H "AccessKey: ${api_key}" \
|
|
-H "Content-Length: 0" \
|
|
"https://api.bunny.net/pullzone/${zone_id}/purgeCache"
|
|
)
|
|
|
|
if [[ "${status}" != "200" && "${status}" != "204" ]]; then
|
|
echo "Purge of pull zone ${zone_id} failed with HTTP ${status}:" >&2
|
|
cat "${response_file}" >&2
|
|
exit 1
|
|
fi
|
|
|
|
echo "Purged pull zone ${zone_id} (HTTP ${status})"
|
|
done
|