fix: recompute package snapshot when missing from cache #95

Merged

pat-s merged 1 commit from fix/recompute-missing-package-snapshot into main

2026-06-17 17:49:05 +00:00

pat-s commented

2026-06-17 17:47:43 +00:00

Owner

Summary

build-all.R reads three snapshot files from /mnt/cache/packages/ that the build-all-versions-install-deps step precomputes: pkgs_to_build.rds, r_minor_sensitive_pkgs.rds, and s3_cache.rds.
That cache volume is per-agent, so a build job scheduled on a different (fresh) agent than the one that ran install-deps finds the snapshot absent and dies at readRDS (cannot open compressed file '/mnt/cache/packages/pkgs_to_build.rds').

This adds a conditional guard at the top of build-all.R: when any of the three files is missing, it sources local/packages-to-build.R (which has all needed creds via PGPASS / B2_S3_* env, already present in the build step) and writes the derived .rds files — exactly mirroring the install-deps command.

The first build job on a fresh agent repopulates the shared cache, so subsequent jobs on that agent reuse it.
Concurrent jobs that also miss simply redo the work (accepted tradeoff vs. slow shared storage like NFS).
Saves use a temp-file + atomic file.rename, so a concurrent reader never sees a half-written .rds.

## Summary `build-all.R` reads three snapshot files from `/mnt/cache/packages/` that the `build-all-versions-install-deps` step precomputes: `pkgs_to_build.rds`, `r_minor_sensitive_pkgs.rds`, and `s3_cache.rds`. That cache volume is **per-agent**, so a build job scheduled on a different (fresh) agent than the one that ran install-deps finds the snapshot absent and dies at `readRDS` (`cannot open compressed file '/mnt/cache/packages/pkgs_to_build.rds'`). This adds a conditional guard at the top of `build-all.R`: when any of the three files is missing, it sources `local/packages-to-build.R` (which has all needed creds via `PGPASS` / `B2_S3_*` env, already present in the build step) and writes the derived `.rds` files — exactly mirroring the install-deps command. - The first build job on a fresh agent repopulates the shared cache, so subsequent jobs on that agent reuse it. - Concurrent jobs that also miss simply redo the work (accepted tradeoff vs. slow shared storage like NFS). - Saves use a temp-file + atomic `file.rename`, so a concurrent reader never sees a half-written `.rds`.

pat-s added 1 commit

2026-06-17 17:47:43 +00:00

fix: recompute package snapshot when missing from cache 26ac988717

The install-deps step precomputes pkgs_to_build.rds, r_minor_sensitive_pkgs.rds
and s3_cache.rds into /mnt/cache, but that cache volume is per-agent.
A build job scheduled on a fresh agent (or racing install-deps) finds the
snapshot absent and fails at readRDS.

Guard the reads in build-all.R: when any snapshot file is missing, source
packages-to-build.R and save the derived files (atomic temp+rename so a
concurrent job never reads a half-written rds). The first job on an agent
repopulates the shared cache for subsequent jobs; jobs that also miss redo
the work.

pat-s merged commit fa955c10bf into main

2026-06-17 17:49:05 +00:00

pat-s deleted branch fix/recompute-missing-package-snapshot

2026-06-17 17:49:05 +00:00

pat-s referenced this pull request from a commit

2026-06-17 17:49:06 +00:00

fix: recompute package snapshot when missing from cache (#95)