r-builds

The environment which produces R builds obtainable from: https://cdn.rstudio.com/r/versions.json

Adding a new platform.

Dockerfile

Create a builder/Dockerfile.platform-version (where platform-version is ubuntu-1604 or centos-74, etc.) This file must contain four major tasks:

an OS_IDENTIFIER env with the platform-version.
a step which ensures the R source build dependencies are installed
The awscli, most likely installed via pip for uploading tarballs to S3
COPY and ENTRYPOINT for the build.sh file in builder/.

docker-compose.yml

A new service in the docker-compose file named according to the platform-version and containing the proper entries:

command: ./build.sh
environment:
  - R_VERSION=${R_VERSION} # for testing out R builds locally
  - LOCAL_STORE=/tmp/output # ensures that output tarballs are persisted locally
build:
  context: .
  dockerfile: Dockerfile.debian-9
image: r-builds:debian-9
volumes:
  - ./integration/tmp:/tmp/output  # path to output tarballs

Job definition

IN serverless-resources.yml you'll need to add a job definition that points to the ECR image.

rBuildsBatchJobDefinitionDebian9:
  Type: AWS::Batch::JobDefinition
  Properties:
    Type: container
    ContainerProperties:
      Command:
        - ./build.sh
      Vcpus: 4
      Memory: 4096
      JobRoleArn:
        "Fn::GetAtt": [ rBuildsEcsTaskIamRole, Arn ]
      Image: #{AWS::AccountId}.dkr.ecr.#{AWS::Region}.amazonaws.com/r-builds:debian-9

Environment variables in the serverless.yml functions.

The serverless functions which trigger R builds need to be informed of new platforms.

Add a JOB_DEFINITION_ARN_PlatformVersion env variable with a ref to the Job definition above.
Append the platform-version to SUPPORTED_PLATFORMS.

environment:
  # snip
  JOB_DEFINITION_ARN_debian_9:
    Ref: rBuildsBatchJobDefinitionDebian9
  SUPPORTED_PLATFORMS: ubuntu-1604,ubuntu-1804,debian-9

Makefile

In order for the makefile to push these new platforms to ECR, add them to the PLATFORMS variable near the top of the Makefile

Submit a Pull Request

Once you've followed the steps above, submit a pull request. On successful merge, builds for this platform will begin to be available from the CDN.

"Break Glass"

Periodically, someone with access to these resources may need to re-trigger every R version/platform combination. This quite easy with the serverless tool installed.

serverless invoke stepf -n rBuilds -d '{"force": true}'

2.5 KiB Raw Blame History