devxy/build-cran-binaries
Watch
1
0
Fork
You've already forked build-cran-binaries
0

feat(stats): collect CDN access logs and publish download statistics

Add an hourly download-stats pipeline that reads the cran-rpkgs Bunny
access logs into a download_stats schema in the build metadata database
and publishes aggregated counts to /stats/downloads.json for the
rpkgs.com /stats page.

- Classify each request (download, index, redirect reason, probe, noise)
  mirroring the edge router, and split cloud/CI networks from the rest.
- De-duplicate downloads per client, package version and day, and key
  clients on a salted HMAC so no IP is stored or published.
- Exclude our own CI runners and record run windows to show gaps.
- Declare the zone's logging settings and allow CORS for JSON.
This commit is contained in:
Patrick Schratz 2026-11-14 13:02:53 +01:00
commit 2614d40bd9

View file

@ -1,7 +1,7 @@
library(future)
future::plan(multisession)
future::plan(sequential)
time = Sys.time()
time <- Sys.time()
files <- s3fs::s3_dir_ls(
sprintf(
"devxy-r-package-binaries-hel1/arm64/rhel9/latest/src/contrib/Archive"
@ -54,8 +54,8 @@ curl::curl_download(
"https://cran.devxy.io/amd64/rhel9/latest/src/contrib/PACKAGES.db",
"PACKAGES.db"
)
con = DBI::dbConnect(RSQLite::SQLite(), "PACKAGES.db")
df = DBI::dbReadTable(con, "packages")
con <- DBI::dbConnect(RSQLite::SQLite(), "PACKAGES.db")
df <- DBI::dbReadTable(con, "packages")
cranlike:::write_packages_files("/tmp", "PACKAGES.db")
@ -69,11 +69,11 @@ system("cat /tmp/PACKAGES | grep '^Package: digest' | wc -l")
curl::curl_fetch_memory(
"https://cloud.r-project.org/src/contrib/Meta/archive.rds"
)
remotes = readRDS(url(
remotes <- readRDS(url(
"https://cloud.r-project.org/src/contrib/Meta/archive.rds",
"rb"
))
packages = readRDS(url(
packages <- readRDS(url(
"https://cran.devxy.io/arm64/noble/latest/src/contrib/PACKAGES.rds",
"rb"
))