devxy/build-cran-binaries
Watch
1
0
Fork
You've already forked build-cran-binaries
 0
Code Issues 10 Pull requests Releases Activity
Building CRAN binaries for rpkgs.com
135 commits 7 branches 0 tags 3.1 MiB
  • R 77.4%
  • Shell 8.6%
  • TypeScript 7%
  • HCL 3.5%
  • Just 1.8%
  • Other 1.7%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
automation-bot f5ba3229a9 chore: Configure Renovate (#6) 
Welcome to [Renovate](https://github.com/renovatebot/renovate)! This is an onboarding PR to help you understand and configure settings before regular Pull Requests begin.

🚦 To activate Renovate, merge this Pull Request. To disable Renovate, simply close this Pull Request unmerged.

---
### Detected Package Files

 * `docker/Dockerfile-ubi9` (dockerfile)
 * `docker/Dockerfile.rhel8` (dockerfile)
 * `.woodpecker/build.yaml` (woodpecker)
 * `.woodpecker/images.yaml` (woodpecker)

### Configuration Summary

Based on the default config's presets, Renovate will:

  - Start dependency updates only once this onboarding PR is merged
  - Enable Renovate Dependency Dashboard creation.
  - Use semantic commit type `fix` for dependencies and `chore` for all others if semantic commits are in use.
  - Ignore `node_modules`, `bower_components`, `vendor` and various test/tests directories.
  - Group known monorepo packages together.
  - Use curated list of recommended non-monorepo package groupings.
  - Apply crowd-sourced package replacement rules.
  - Apply crowd-sourced workarounds for known problems with packages.
  - Run lock file maintenance (updates) early Monday mornings.
  - Schedule automerge daily.
  - Schedule for weekends.
  - Enable Renovate Dependency Dashboard creation.
  - Use semantic commit type `fix` for dependencies and `chore` for all others if semantic commits are in use.
  - Ignore `node_modules`, `bower_components`, `vendor` and various test/tests directories.
  - Group known monorepo packages together.
  - Use curated list of recommended non-monorepo package groupings.
  - Apply crowd-sourced package replacement rules.
  - Apply crowd-sourced workarounds for known problems with packages.
  - Run lock file maintenance (updates) early Monday mornings.
  - Schedule automerge daily.
  - Schedule for weekends.
  - Run Renovate on following schedule: every weekend

🔡 Do you want to change how Renovate upgrades your dependencies? Add your custom config to `renovate.json` in this branch. Renovate will update the Pull Request description the next time it runs.

---

### What to Expect

It looks like your repository dependencies are already up-to-date and no Pull Requests will be necessary right away.

---

❓ Got questions? Check out Renovate's [Docs](https://docs.renovatebot.com/), particularly the Getting Started section.
If you need any further assistance then you can also [request help here](https://github.com/renovatebot/renovate/discussions).

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

<!--renovate-config-hash:5194d01a36db041d33badf95c91bbd2577f7875d668ecaa620d459efdda0b7ef-->

Reviewed-on: devxy/arm64-r-binaries#6
Co-authored-by: automation-bot <no-reply@devxy.io>
Co-committed-by: automation-bot <no-reply@devxy.io>
2024-07-29 21:21:39 +00:00
.woodpecker brackets issue 2024-07-29 19:40:32 +02:00
docker change url 2024-07-29 01:34:15 +02:00
man cleanup and reorg 2024-07-27 13:33:50 +02:00
R brackets issue 2024-07-29 19:40:32 +02:00
.editorconfig linters 2024-05-29 19:53:38 +02:00
.gitignore gitignore 2024-07-28 10:55:33 +02:00
.markdownlint.yaml linters 2024-05-29 19:53:38 +02:00
.prettierrc.json linters 2024-05-29 19:53:38 +02:00
.Rbuildignore Rbuildignore 2024-07-25 00:28:16 +02:00
04-create-archive-rds.sh save state 2024-05-29 19:52:07 +02:00
05-get-updated-cran-pkgs.R move to R package 2024-06-02 13:05:39 +02:00
DESCRIPTION cleanup and reorg 2024-07-27 13:33:50 +02:00
Justfile start building docker images 2024-06-03 23:51:17 +02:00
LICENSE.md license 2024-07-26 11:23:40 +02:00
NAMESPACE cleanup and reorg 2024-07-27 13:33:50 +02:00
rBinaries.Rproj move to R package 2024-06-02 13:05:39 +02:00
README.md readme 2024-07-26 11:24:08 +02:00
renovate.json chore: Configure Renovate (#6) 2024-07-29 21:21:39 +00:00

README.md

README

Workflow

  1. "Prepare"
    1. Install R interpreter
    2. Add required repos to resolve syslibs
    3. Install pak
    4. Install awscli for remote s3 sync
  2. "Build"
    1. For every package, do
      1. Clone from GitHub CRAN mirror
      2. Install all system dependencies via pak::local_install_dev_deps()
      3. Checkout each available tag
      1. Build binary package
      2. Create Archive/ directory
      3. Move all versions besides the most recent one to Archive/ (for source and binary)
      4. Create PACKAGES metadata file using cranlike package
      5. Store build metadata in sqlite DB
  3. "Upload"
    1. Sync all binaries and sources to S3 bucket
  4. -4-create-archive-rds.sh
    1. Create Meta/archive.rds file for each package so that remotes::install_version() can install archived versions
    2. Upload Meta/archive.rds to S3 bucket

Build metadata

The build metadata is stored in a sqlite database. The following tables are used:

  • metadata contains the build metadata for each package:
package_name tag platform error_occurred build_timestamp
  • metadata_summary contains aggregated build metadata for each package and platform:
package_name platform successful_builds unsuccessful_builds total_builds percentage_successful_builds average_build_time_per_tag

Support for archived versions

remotes::install_version()

remotes::install_version() is searching for a Meta/archive.rds file in the /src/contrib directory. This file must be a list of dataframes containing the information about the archived versions of the package.

Example:


con <- gzcon(url(sprintf("%s/src/contrib/Meta/archive.rds",
                           c("CRAN" = "https://cloud.r-project.org")), "rb"))
foo = readRDS(con)
foo[[1]]

pak::pak(package@version)

pak searches for Archive/<package> and is able to install all versions existing there. Ensure to use a clean cache if other repos have been used before, i.e. call pak::meta_clean(force = TRUE) if in doubt or when testing.

Platforms

The following platforms are supported:

  • RHEL9

Lessons learned

  • Newer R versions have issues compiling C code from older package Versions. Using an older R version to build the binaries is recommended.
  • R 4.0.5 is used to build all binaries for R 4.x
  • Binaries which cannot be built using R 4.0.5 often also require older system library or compiler versions and are out of scope for this project.

Platforms

Helpers

Check the row number of a package by name

pkgs = tools::CRAN_package_db()
which(pkgs$Package == "ABCanalysis")