81 lines
2.5 KiB
Markdown
81 lines
2.5 KiB
Markdown
# r-builds
|
|
|
|
The environment which produces R builds obtainable from: https://cdn.rstudio.com/r/versions.json
|
|
|
|
## Adding a new platform.
|
|
|
|
### Dockerfile
|
|
|
|
Create a `builder/Dockerfile.platform-version` (where `platform-version` is `ubuntu-1604` or `centos-74`, etc.) This file must contain four major tasks:
|
|
|
|
1. an `OS_IDENTIFIER` env with the `platform-version`.
|
|
2. a step which ensures the R source build dependencies are installed
|
|
3. The `awscli`, most likely installed via `pip` for uploading tarballs to S3
|
|
4. `COPY` and `ENTRYPOINT` for the `build.sh` file in `builder/`.
|
|
|
|
### docker-compose.yml
|
|
|
|
A new service in the docker-compose file named according to the `platform-version` and containing the proper entries:
|
|
|
|
```
|
|
command: ./build.sh
|
|
environment:
|
|
- R_VERSION=${R_VERSION} # for testing out R builds locally
|
|
- LOCAL_STORE=/tmp/output # ensures that output tarballs are persisted locally
|
|
build:
|
|
context: .
|
|
dockerfile: Dockerfile.debian-9
|
|
image: r-builds:debian-9
|
|
volumes:
|
|
- ./integration/tmp:/tmp/output # path to output tarballs
|
|
```
|
|
|
|
### Job definition
|
|
|
|
IN `serverless-resources.yml` you'll need to add a job definition that points to the ECR image.
|
|
|
|
```
|
|
rBuildsBatchJobDefinitionDebian9:
|
|
Type: AWS::Batch::JobDefinition
|
|
Properties:
|
|
Type: container
|
|
ContainerProperties:
|
|
Command:
|
|
- ./build.sh
|
|
Vcpus: 4
|
|
Memory: 4096
|
|
JobRoleArn:
|
|
"Fn::GetAtt": [ rBuildsEcsTaskIamRole, Arn ]
|
|
Image: #{AWS::AccountId}.dkr.ecr.#{AWS::Region}.amazonaws.com/r-builds:debian-9
|
|
```
|
|
|
|
### Environment variables in the serverless.yml functions.
|
|
|
|
The serverless functions which trigger R builds need to be informed of new platforms.
|
|
|
|
1. Add a `JOB_DEFINITION_ARN_PlatformVersion` env variable with a ref to the Job definition above.
|
|
2. Append the `platform-version` to `SUPPORTED_PLATFORMS`.
|
|
|
|
```
|
|
environment:
|
|
# snip
|
|
JOB_DEFINITION_ARN_debian_9:
|
|
Ref: rBuildsBatchJobDefinitionDebian9
|
|
SUPPORTED_PLATFORMS: ubuntu-1604,ubuntu-1804,debian-9
|
|
```
|
|
|
|
### Makefile
|
|
|
|
In order for the makefile to push these new platforms to ECR, add them to the PLATFORMS variable near the top of the Makefile
|
|
|
|
### Submit a Pull Request
|
|
|
|
Once you've followed the steps above, submit a pull request. On successful merge, builds for this platform will begin to be available from the CDN.
|
|
|
|
## "Break Glass"
|
|
|
|
Periodically, someone with access to these resources may need to re-trigger every R version/platform combination. This quite easy with the `serverless` tool installed.
|
|
|
|
```
|
|
serverless invoke stepf -n rBuilds -d '{"force": true}'
|
|
```
|