diff --git a/.gitignore b/.gitignore index 442134b..64c54dd 100644 --- a/.gitignore +++ b/.gitignore @@ -43,9 +43,6 @@ vignettes/*.pdf # R Environment Variables .Renviron -# pkgdown site -docs/ - # translation temp files po/*~ diff --git a/docs/_brand.yaml b/docs/_brand.yaml index 247a00c..b2b110f 100644 --- a/docs/_brand.yaml +++ b/docs/_brand.yaml @@ -22,13 +22,12 @@ defaults: typography: fonts: - family: Jura - source: google - - family: Inter Variable - source: google - - family: Montserrat Alternates - source: google - base: - family: Inter Variable + source: bunny + - family: Inter + source: bunny + - family: Montserrat + source: bunny + base: Inter headings: Montserrat link: decoration: underline diff --git a/docs/_quarto.yml b/docs/_quarto.yml index 2c5fe3c..cf9f7ba 100644 --- a/docs/_quarto.yml +++ b/docs/_quarto.yml @@ -35,10 +35,10 @@ website: file: r-dev-environments/index.qmd - text: "Containers" file: containers/index.qmd - # - section: "**Technical Background**" - # contents: - # - text: "Infrastructure" - # file: 031-infrastructure.qmd + - section: "**Technical Background**" + contents: + - text: "Infrastructure" + file: infrastructure/index.qmd # - text: "Continuous Updates" # file: 032-continuous-updates.qmd # - section: "**Miscellaneous**" diff --git a/docs/index.qmd b/docs/index.qmd index f8ce941..4aa7744 100644 --- a/docs/index.qmd +++ b/docs/index.qmd @@ -5,7 +5,7 @@ title: "CRAN R Package Binaries" This project provides R package binaries for various Linux distributions and architectures. While it provides binaries for [CRAN packages](https://cran.r-project.org/), it is not affiliated with CRAN in any way. -Binaries are distributed to the community for free with a monthly download cap of **50 Terrabyte**. +Binaries are distributed to the community for free with a monthly download cap of **50 Terabyte**. ::: {.callout-note} If you represent a for-profit company and plan to make extensive use of these binaries, please get in touch. We offer the option to obtain a private mirror with unlimited access. diff --git a/docs/infrastructure/index.qmd b/docs/infrastructure/index.qmd index 6d3f665..6b4c2f4 100644 --- a/docs/infrastructure/index.qmd +++ b/docs/infrastructure/index.qmd @@ -1 +1,61 @@ -WIP \ No newline at end of file +--- +title: Infrastructure +--- + +The infrastructure is composed out of the following main components: + +- A k3s cluster (on Hetzner) +- S3 object storage (on Hetzner) +- A CDN (Bunny CDN) + +As the resource need for this project is quite variable, running on Kubernetes allows the efficient use of (shared) resources. + +## k3s + +Efficiently building on different architectures requires access to native servers for those architectures. +While most cloud providers make this possible, Kubernetes simplifies the process by enabling orchestration of specific builds to designated nodes. +This flexibility ensures that builds are executed on the appropriate hardware, optimizing performance and compatibility across diverse environments. + +### Memory Requirements + +Because the memory requirements for each package are vastly different and range between a few hundred MBs to a ~ 12 GB, these must also be reflected in the resource requirements of each pod. +Using `requests.memory` of 5Gi and `limits.memory` of 14Gi has been proven to work reliable for scheduling and individual resource needs. + +### Matrix builds + +While daily package updates can be handled with a single process per OS/version, building binaries for all CRAN packages requires a different orchestration strategy. +On average, each CRAN package has six versions (calculated by dividing the total number of binaries by the OS/versions built and the number of unique packages), making some level of parallelization necessary. + +Initially, parallelizing at the level of package versions was implemented. +However, this approach led to occasional conflicts when dependencies were installed into a shared package cache. +It also introduced unpredictable memory requirements within workflows: some packages caused memory usage to spike beyond 30 GB, depending on the number of parallel workers. +These spikes not only caused individual processes to crash but also demanded significantly higher overall resource limits. + +A more robust solution was found by processing individual packages sequentially within each matrix job. + +To build all versions of all packages, CRAN packages are divided into subsets, each comprising 1/10 or fewer of the total packages, and these subsets are processed in parallel. +The total time required depends on factors such as the distribution (e.g., distributions with newer C compilers like Alpine tend to be faster) and the number of parallel workers. +This approach typically takes anywhere from a few days to two weeks. + +## Storage: S3 + +Binaries need to be stored somewhere, and what better option than S3? S3 is significantly more cost-effective than traditional cloud disk storage and offers the added benefit of being accessible via a public API. +Beyond AWS, the original provider and inventor of S3, there are numerous alternatives with better price-to-storage ratios and lower transfer costs. + +The timing was perfect when Hetzner introduced their own S3-compatible object storage, coinciding with the start of this project’s build processing. +This solution brings multiple advantages: lower overall storage costs, free internal traffic between Hetzner servers and their S3 storage, and the proximity of storage to the build servers, which minimizes upload latency. + +::: {.callout-note} +While uploading packages to S3 is not complicated, there hasn't been any way to create the required PACKAGES index files for binaries stored there. +This is why forked versions of {cranlike} and {desc} have been created. +::: + +## Content Delivery Network (CDN) + +Storing binaries in S3 works well for distribution, but it’s not inherently very fast. Adding a CDN in front of S3 enables caching and allows assets to be distributed via servers located in various regions worldwide. This significantly reduces download latency, making downloads feel much faster. + +All packages are delivered through a CDN, which includes three dedicated static caches strategically placed in Germany, the USA, and Asia. + +With a CDN in place, downloads are optimized to feel “fast” from virtually anywhere, with only minor variations depending on the user’s location. + +The CDN determines when an asset is added to its permanent cache and how often it is revalidated against the S3 source. Since package binaries are one-time builds that typically remain unchanged unless a forced rebuild occurs, relying heavily on a permanent cache is highly efficient in this context.