fix(ci): authenticate B2 list-bucket request in existing-version check (#7)
Some checks failed
ci/crow/cron/build/1 Pipeline was successful
ci/crow/cron/build/2 Pipeline was successful
ci/crow/cron/build/4 Pipeline was successful
ci/crow/cron/build/5 Pipeline was successful
ci/crow/cron/build/6 Pipeline was successful
ci/crow/cron/build/8 Pipeline was successful
ci/crow/cron/build/10 Pipeline was successful
ci/crow/cron/build/11 Pipeline was successful
ci/crow/cron/build/12 Pipeline was successful
ci/crow/cron/build/13 Pipeline was successful
ci/crow/cron/build/3 Pipeline was successful
ci/crow/cron/build/18 Pipeline was successful
ci/crow/cron/build/16 Pipeline was successful
ci/crow/cron/build/14 Pipeline was successful
ci/crow/cron/build/9 Pipeline was successful
ci/crow/cron/build/17 Pipeline was canceled
ci/crow/cron/build/15 Pipeline was canceled
ci/crow/cron/build/7 Pipeline was canceled

Backblaze B2 requires authentication for the list-bucket API; anonymous
access only works for individual public-read objects. The unauthenticated
curl returned AccessDenied, so r-version-s3.txt was always empty and every
R version was rebuilt despite already existing in s3.

Sign the listing request with --aws-sigv4 using the same B2 credentials as
the upload step. Also fix the broken version extraction: the old
substr($NF, 3, 5) ran against the prefixed key and yielded garbage, so the
Build step's grep could never match. Print the actual version on a match
and use a literal index() instead of a regex comparison.

Reviewed-on: #7
This commit is contained in:
Patrick Schratz 2026-06-29 11:02:04 +00:00 committed by Patrick Schratz
commit 1e278cdb3a

View file

@ -158,11 +158,23 @@ steps:
- name: Check which R versions already exist in s3
image: reg.devxy.io/docker.io/library/alpine:3.23
privileged: true
environment:
AWS_ACCESS_KEY_ID:
from_secret: B2_S3_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY:
from_secret: B2_S3_SECRET_KEY
commands:
- ip link set dev eth0 mtu 1280 2>/dev/null || true
- for i in 1 2 3 4 5; do apk add -q --no-cache curl && break; sleep 5; done
- |
LISTING=$(curl -s "https://s3.eu-central-003.backblazeb2.com/devxy-r-builds?prefix=${PLATFORM_ID}/" | \
# Backblaze B2 requires authentication for the list-bucket API (anonymous
# GET works only for individual public-read objects), so the request must be
# SigV4-signed with the same credentials used for the upload step. Without
# this the listing returns AccessDenied, r-versions-existing.txt stays empty,
# and every version is rebuilt even though it already exists.
LISTING=$(curl -s --aws-sigv4 "aws:amz:eu-central-003:s3" \
--user "${AWS_ACCESS_KEY_ID}:${AWS_SECRET_ACCESS_KEY}" \
"https://s3.eu-central-003.backblazeb2.com/devxy-r-builds?prefix=${PLATFORM_ID}/" | \
grep -oE '<Key>[^<]+</Key>' | sed 's/<[^>]*>//g')
: > r-versions-existing.txt
for VERSION in $(cat r-versions-to-build.txt); do