devxy/build-cran-binaries
Watch
1
0
Fork
You've already forked build-cran-binaries
 0
Code Issues 10 Pull requests Releases Activity

add problematic pkgs

Browse source
This commit is contained in:
Patrick Schratz 2024-11-25 10:05:23 +01:00
commit 1da194471d
Signed by: pat-s
GPG key ID: 3C6318841EF78925
1 changed files with 27 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

38
README.md
View file

@ -52,9 +52,9 @@ For every package+tag combination:
## Build Environment
Binaries are built on a mixed-architecture Kubernetes cluster using CI.
Dedicated arm64 and amd64 nodes are utilized to efficiently build the binaries.
After all binaries for a specific architecture/OS combination are built, CRON jobs handle the processing of daily change operations.
Binaries are built on a mixed-architecture Kubernetes cluster using CI.
Dedicated arm64 and amd64 nodes are utilized to efficiently build the binaries.
After all binaries for a specific architecture/OS combination are built, CRON jobs handle the processing of daily change operations.
This elastic server architecture offers a robust and performant backend while minimizing costs.
## Technical Details
@ -100,8 +100,8 @@ The build metadata is stored in a PostgreSQL database.
The database has a public endpoint at `r-binaries.devxy.io` and port `15432`.
The database contains one table named `single_builds`, which holds the build metadata for each package:
| package_name | tag | platform | error_occurred | build_timestamp | build_duration | error | size |
| ------------ | --- | -------- | -------------- | --------------- | -------------- | ----- | ---- |
| package_name | tag | platform | error_occurred | build_timestamp | build_duration | error | size | removed |
| ------------ | --- | -------- | -------------- | --------------- | -------------- | ----- | ---- | ------- |
Column types:
@ -113,6 +113,7 @@ Column types:
- `build_duration`: numeric(1000,2)
- `error`: text
- `size`: numeric(1000,2)
- `removed`: boolean
Alternatively, use `\d+ single_builds`.
@ -180,12 +181,13 @@ Technically, no date-based snapshots are planned, so this component from the Pos
## Helpers
Helper scripts are located in `local/`.
Helper scripts are located in `local/`.
These scripts can assist in various situations, such as manually processing packages or filtering specific information from the metadata database.
## Common Errors
Below is a collection of raw errors observed during the build process:
<details>
```
@ -225,17 +227,31 @@ R -q -e 'install.packages("pak", repos = sprintf("https://r-lib.github.io/p/pak/
## Packages Skipped on Purpose
Some packages have been intentionally skipped after multiple build attempts.
The reasons for this vary, and ideally, solutions can be found over the long term.
Some packages have been intentionally skipped after multiple build attempts.
The reasons for this vary, and ideally, solutions can be found over the long term.
Contributions to help resolve these issues are highly welcome!
| Name | Platform | Arch | Reason | Solved via | Date Created | Date Solved |
| ------------- | -------------- | ------- | ------------- | ---------- | ------------ | ----------- |
| Apollonius | redhat-8 | | gmp missing | 2024-11-15 | |
| doBy | alpine-320 | amd | hangs | 2024-11-23 | |
| later | ubuntu 22 & 24 | amd | hangs - xfvb? | 2024-11-23 | |
| CoTiMA | ubuntu 22 & 24 | amd | OOM? | 2024-11-23 | |
| FrF2 | alpine | arm | hangs | 2024-11-23 | |
| FrF2.catlg128 | alpine | arm | hangs | 2024-11-23 | |
| DoE.base | alpine | arm/amd | hangs | 2024-11-23 | |
| eha | alpine | amd | hangs | 2024-11-23 | |
| gRain | alpine | amd | hangs | 2024-11-23 | |
| gRbase | alpine | amd | hangs | 2024-11-24 | |
| IDPmisc | alpine | amd | hangs | 2024-11-24 | |
## CDN Settings
A CDN is used in front of the S3 bucket to efficiently distribute the binaries globally.
A CDN is used in front of the S3 bucket to efficiently distribute the binaries globally.
A "Perma-Cache" is enabled for three different regions around the world (DE, US, Asia).
A "Perma-Cache" is enabled for three different regions around the world (DE, US, Asia).
Once a binary is requested for the first time from a specific location, the asset is copied to the perma-cache and served from there for subsequent requests.
The `CacheControl = "no-cache"` header is set for all PACKAGES* files to ensure users always receive the latest version, as these files change daily.
The `CacheControl = "no-cache"` header is set for all PACKAGES\* files to ensure users always receive the latest version, as these files change daily.
A monthly traffic limit of 50 TB is set on cran.devxy.io to prevent abuse and manage costs.