docs: update README

This commit is contained in:
Patrick Schratz 2025-01-02 13:55:31 +01:00
commit f0ec2e4942
Signed by: pat-s
GPG key ID: 3C6318841EF78925

View file

@ -3,8 +3,7 @@ gitea: none
include_toc: true
---
# bincraftR
# build-binaries
This project offers a framework for creating R package binaries on Linux across various architectures and distributions.
@ -18,7 +17,7 @@ It achieves this through the integration of several components:
## R Package
The R package [`bincraftR`](https://gitlab.com/devxy/r-package-binaries/bincraftr) powers everything.
It provides the following functioanlity:
It provides the following functionality:
- build binaries
- archive packages following the CRAN-like directory structure
@ -26,19 +25,19 @@ It provides the following functioanlity:
- update the package index files (`PACKAGES*`)
- store build metadata, including error logs, in a PostgreSQL database
See the function reference on the pkgdown site for a full overview.
See the function reference on the [pkgdown site](https://devxy.gitlab.io/r-package-binaries/bincraftr/) for a full overview.
The focus of the R package is on usability rather than minimizing dependencies.
The individual containerfiles include the package along with its dependencies.
Bundling more R packages upfront helps reduce the number of additional packages needed when installing the dependencies for building packages.
Bundling more R packages upfront helps reducing the number of additional packages needed when installing the dependencies for the individual packages to be built.
## Containerfiles
The toolchain in the containerfile of each distribution is a very important element for the build success of the packages.
The toolchain in the containerfile of each distribution is a core element for the build success of each package.
The C compiler settings should be close to the recommended settings from CRAN and allow compatibility for most CRAN packages.
Here, especially Alpine is tricky as CRAN does not test R packages for Alpine.
Since Alpine uses a different C library (MUSL instead of GLIBC), many R packages that include C/C++ code encounter errors.
Especially Alpine is tricky as CRAN currently does not test R packages for Alpine compatibility (i.e. working with MUSL instead of GLIBC).
Due to the different C library (MUSL instead of GLIBC), many R packages that include C/C++ code encounter errors.
## Build Process
@ -69,7 +68,7 @@ Currently, the {cranlike} and {desc} packages only work with files on a local fi
This is infeasible if the goal is to store binaries in S3.
Storing binaries permanently on a disk-based file system would incur significantly higher costs, especially when operating in the cloud.
Hence, modified versions of {cranlike} and {desc} were created that are able to handle files in S3 (through {s3fs}).
Hence, [{cranlike}](https://devxy.gitlab.io/r-package-binaries/bincraftr/) and [{desc}](https://github.com/pat-s/desc/tree/description-from-remote) were forked to handle files in S3 (through {s3fs}).
### Resources
@ -108,20 +107,22 @@ It contains one table named `single_builds`, which holds the build metadata for
Column types:
- `package_name`: character varying(255)
- `tag`: character varying(255)
- `platform`: character varying(255)
- `error_occurred`: boolean
- `build_timestamp`: timestamp without time zone
- `build_duration`: numeric(1000,2)
- `error`: text
- `size`: numeric(1000,2)
- `removed`: boolean
| Column Name | Data Type |
| ----------------- | --------------------------- |
| `package_name` | character varying(255) |
| `tag` | character varying(255) |
| `platform` | character varying(255) |
| `error_occurred` | boolean |
| `build_timestamp` | timestamp without time zone |
| `build_duration` | numeric(1000,2) |
| `error` | text |
| `size` | numeric(1000,2) |
| `removed` | boolean |
Alternatively, use `\d+ single_builds`.
**Note**: There is currently no read-only role available to connect to the DB as a viewer.
A Shiny dashboard providing a search functionality of the database and grouped statistics is available at <https://shinyproxy.devxy.io/app/r-package-binaries-dashboard> with the source repo living at <https://gitlab.com/devxy/r-package-binaries/shiny>.
A Shiny dashboard providing a search functionality of the database and grouped statistics is available at <https://app.devxy.io/app/r-package-binaries-dashboard> with the source repo living at <https://gitlab.com/devxy/r-package-binaries/shiny>.
## Support for Archived Versions
@ -131,7 +132,7 @@ Is supported by writing `Meta/archive.rds` during each package index update, lis
### `pak::pak(package@version)`
`pak` searches for `Archive/<package>` and can install all versions it finds.
`pak` searches for `Archive/<package>` and can install all versions it finds.
Ensure to use a clean cache if other repositories have been used previously.
If in doubt or when testing, call `pak::meta_clean(force = TRUE)`.
@ -164,6 +165,7 @@ Platform identifiers have been aligned with those used in <https://github.com/rs
- ubuntu-2204
- ubuntu-2404
- alpine-320
- alpine-321
The final repository URL is structured slightly differently and follows the format of the Posit Packagemanager:
@ -221,8 +223,10 @@ R -q -e 'install.packages("pak", repos = sprintf("https://r-lib.github.io/p/pak/
## Packages with unresolved build failures
Some packages are intentionally skipped right now after multiple failed build attempts without proper resolutions yet.
While for some it be due to exotic external dependencies which require manual installation, the main issue are specific packages on Alpine which just get stuck without any error.
Some packages are intentionally skipped right for "full builds" (i.e. building of all versions) as they result in getting stuck indefinitely.
This applies mostly to very old package versions and the most recent version can usually be built successfully.
For others it might be due to exotic external dependencies which require manual installation.
Help in resolving these issues are highly welcome!