Rocker and binder and renv, Oh My!

why?

  • reproducibility
  • run script across sites

Docker is a tool for running programs in an "isolated" environment.

https://github.com/jennybc/docker-why

  • test packages in development

Dockerfile

image

container

Docker Hub

file with a series of commands to build an image

package defining a working environment

instance of an image/running image

website/repository for sharing Docker images

FROM rocker/tidyverse:4.0.2
MAINTAINER grixor@gmail.com

RUN Rscript -e "install.packages('pmlbr')"

Dockerfile

docker build --tag [image-name] [directory]

command line:

environment/Dockerfile
pmlb-analysis

images reside in registries

  • build images locally (from Dockerfile)
  • push images to remote registries (DockerHub, GitHub)
  • others: pull images from remote registries
docker run [options] image

  -p [ports]
  -v [volumes]
  -d -i -t
  --name [container]
  --rm

pmlb-analysis

create and run a container 

docker stop [container]
docker pull [user/repo]

docker pull rocker/tidyverse

The Rocker Project

## List Docker images
docker images

## List Docker containers
docker ps 

## List Docker commands
docker
docker container --help

## Display Docker version and info
docker version
docker info

linux users will need sudo in front of all commands

# Make the image from the directory with the Dockerfile
docker build -t pmlb-analysis environment/Dockerfile

# Run the image and connect to the RStudio port
# Then, navigate to http://localhost:8787/
docker run --name pmlb -p 8787:8787 -d pmlb-analysis

# list docker containers
docker ps

# stop the relevant container
docker stop pmlb

# Run the image, but this time attach a volume
docker run -d -p 8787:8787 \
  -v $(pwd):/home/rstudio/work pmlb-analysis

all together now

supporting R package
holepunch

ideally used for final publication of analysis

renv

think "conda environment" but just for R

  1. initialize a new project-local environment with a private R library

  2. Work in the project as normal, installing and removing new R packages as they are needed in the project

  3. Save the state of the project library to the lockfile (called renv.lock)

  4. Continue working on your project, installing and updating R packages as needed.

  5. again to save the state of your project library if your attempts to update R packages were successful

renv::init() 
renv::restore() 
renv::snapshot()

revert to the previous state as encoded in the lockfile if your attempts to update packages introduced some new problems.

renv::snapshot()
{
  "R": {
    "Version": "3.6.3",
    "Repositories": [
      {
        "Name": "CRAN",
        "URL": "https://cran.microsoft.com/snapshot/2020-09-14"
      }
    ]
  },
  "Packages": {
    "BH": {
      "Package": "BH",
      "Version": "1.72.0-3",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "8f9ce74c6417d61f0782cbae5fd2b7b0"
    },
    "GrpString": {
      "Package": "GrpString",
      "Version": "0.3.2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "236333c7d243ac125651ce4205de9019"
    },
    "KernSmooth": {
      "Package": "KernSmooth",
      "Version": "2.23-16",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "997471f25a7ed6c782f0090ce52cc63a"
    },
    "MASS": {
      "Package": "MASS",
      "Version": "7.3-51.5",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "9efe80472b21189ebab1b74169808c26"
    },
    "Matrix": {
      "Package": "Matrix",
      "Version": "1.2-18",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "08588806cba69f04797dab50627428ed"
    },
    "R6": {
      "Package": "R6",
      "Version": "2.4.1",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "292b54f8f4b94669b08f94e5acce6be2"
    },
    "RColorBrewer": {
      "Package": "RColorBrewer",
      "Version": "1.1-2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "e031418365a7f7a766181ab5a41a5716"
    },
    "Rcpp": {
      "Package": "Rcpp",
      "Version": "1.0.4.6",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "e652f23d8b1807cc975c51410d05b72f"
    },
    "RcppArmadillo": {
      "Package": "RcppArmadillo",
      "Version": "0.9.860.2.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "da1503de8d67c2101d181250c41b75e1"
    },
    "SnowballC": {
      "Package": "SnowballC",
      "Version": "0.7.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "bc26e07c0d747fd287c370fe355e7b85"
    },
    "TDAmapper": {
      "Package": "TDAmapper",
      "Version": "1.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "174086f82b2040013ff70575f61c9854"
    },
    "assertthat": {
      "Package": "assertthat",
      "Version": "0.2.1",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "50c838a310445e954bc13f26f26a6ecf"
    },
    "backports": {
      "Package": "backports",
      "Version": "1.1.6",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "3997fd62345a616e59e8161ee0a5816f"
    },
    "broom": {
      "Package": "broom",
      "Version": "0.5.6",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "754fdb4d17d4ae871343ff2909728ca3"
    },
    "callr": {
      "Package": "callr",
      "Version": "3.4.3",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "643163a00cb536454c624883a10ae0bc"
    },
    "cli": {
      "Package": "cli",
      "Version": "2.0.2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "ff0becff7bfdfe3f75d29aff8f3172dd"
    },
    "cluster": {
      "Package": "cluster",
      "Version": "2.1.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "db63a44aab5aadcb6bf2f129751d129a"
    },
    "codetools": {
      "Package": "codetools",
      "Version": "0.2-16",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "89cf4b8207269ccf82fbeb6473fd662b"
    },
    "colorspace": {
      "Package": "colorspace",
      "Version": "1.4-1",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "6b436e95723d1f0e861224dd9b094dfb"
    },
    "crayon": {
      "Package": "crayon",
      "Version": "1.3.4",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "0d57bc8e27b7ba9e45dba825ebc0de6b"
    },
    "desc": {
      "Package": "desc",
      "Version": "1.2.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "6c8fe8fa26a23b79949375d372c7b395"
    },
    "digest": {
      "Package": "digest",
      "Version": "0.6.25",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "f697db7d92b7028c4b3436e9603fb636"
    },
    "dplyr": {
      "Package": "dplyr",
      "Version": "0.8.5",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "57a42ddf80f429764ff7987128c3fd0a"
    },
    "ellipsis": {
      "Package": "ellipsis",
      "Version": "0.3.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "7067d90c1c780bfe80c0d497e3d7b49d"
    },
    "evaluate": {
      "Package": "evaluate",
      "Version": "0.14",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "ec8ca05cffcc70569eaaad8469d2a3a7"
    },
    "fansi": {
      "Package": "fansi",
      "Version": "0.4.1",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "7fce217eaaf8016e72065e85c73027b5"
    },
    "farver": {
      "Package": "farver",
      "Version": "2.0.3",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "dad6793a5a1f73c8e91f1a1e3e834b05"
    },
    "generics": {
      "Package": "generics",
      "Version": "0.0.2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "b8cff1d1391fd1ad8b65877f4c7f2e53"
    },
    "ggplot2": {
      "Package": "ggplot2",
      "Version": "3.3.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "911561e07da928345f1ae2d69f97f3ea"
    },
    "glue": {
      "Package": "glue",
      "Version": "1.4.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "2aefa994e8df5da17dc09afd80f924d5"
    },
    "gridExtra": {
      "Package": "gridExtra",
      "Version": "2.3",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "7d7f283939f563670a697165b2cf5560"
    },
    "gtable": {
      "Package": "gtable",
      "Version": "0.3.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "ac5c6baf7822ce8732b343f14c072c4d"
    },
    "heemod": {
      "Package": "heemod",
      "Version": "0.12.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "957d4b1b0ef742bccfc00f94c32615af"
    },
    "igraph": {
      "Package": "igraph",
      "Version": "1.2.5",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "3878c30ce67cdb7f2d7f72554e37f476"
    },
    "irlba": {
      "Package": "irlba",
      "Version": "2.3.3",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "a9ad517358000d57022401ef18ee657a"
    },
    "isoband": {
      "Package": "isoband",
      "Version": "0.2.1",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "9b2f7cf1899f583a36d367702ecf49a3"
    },
    "labeling": {
      "Package": "labeling",
      "Version": "0.3",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "73832978c1de350df58108c745ed0e3e"
    },
    "lattice": {
      "Package": "lattice",
      "Version": "0.20-41",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "fbd9285028b0263d76d18c95ae51a53d"
    },
    "lazyeval": {
      "Package": "lazyeval",
      "Version": "0.2.2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "d908914ae53b04d4c0c0fd72ecc35370"
    },
    "lifecycle": {
      "Package": "lifecycle",
      "Version": "0.2.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "361811f31f71f8a617a9a68bf63f1f42"
    },
    "lsa": {
      "Package": "lsa",
      "Version": "0.73.1",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "236ca1b8e48e7678663be0cf2eee94e2"
    },
    "magrittr": {
      "Package": "magrittr",
      "Version": "1.5",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "1bb58822a20301cee84a41678e25d9b7"
    },
    "memoise": {
      "Package": "memoise",
      "Version": "1.1.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "58baa74e4603fcfb9a94401c58c8f9b1"
    },
    "mgcv": {
      "Package": "mgcv",
      "Version": "1.8-31",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "4bb7e0c4f3557583e1e8d3c9ffb8ba5c"
    },
    "mice": {
      "Package": "mice",
      "Version": "3.8.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "9d0710fc48d992c6c1817f9d811dcfc2"
    },
    "munsell": {
      "Package": "munsell",
      "Version": "0.5.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "6dfe8bf774944bd5595785e3229d8771"
    },
    "mvnfast": {
      "Package": "mvnfast",
      "Version": "0.2.5",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "c9ebc6c55c5feb11f335475e188ee2d2"
    },
    "nlme": {
      "Package": "nlme",
      "Version": "3.1-147",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "dd4a1423f1472a2a8e05dd2c8945c3af"
    },
    "philentropy": {
      "Package": "philentropy",
      "Version": "0.4.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "10e6498f8ee4727f1f6e0525e599f944"
    },
    "pillar": {
      "Package": "pillar",
      "Version": "1.4.3",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "fa3ed60396b6998d0427c57dab90fba4"
    },
    "pkgbuild": {
      "Package": "pkgbuild",
      "Version": "1.0.6",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "899835dfe286963471cbdb9591f8f94f"
    },
    "pkgconfig": {
      "Package": "pkgconfig",
      "Version": "2.0.3",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "01f28d4278f15c76cddbea05899c5d6f"
    },
    "pkgload": {
      "Package": "pkgload",
      "Version": "1.0.2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "5e655fb54cceead0f095f22d7be33da3"
    },
    "plogr": {
      "Package": "plogr",
      "Version": "0.2.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "09eb987710984fc2905c7129c7d85e65"
    },
    "plyr": {
      "Package": "plyr",
      "Version": "1.8.6",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "ec0e5ab4e5f851f6ef32cd1d1984957f"
    },
    "praise": {
      "Package": "praise",
      "Version": "1.0.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "a555924add98c99d2f411e37e7d25e9f"
    },
    "prettyunits": {
      "Package": "prettyunits",
      "Version": "1.1.1",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "95ef9167b75dde9d2ccc3c7528393e7e"
    },
    "processx": {
      "Package": "processx",
      "Version": "3.4.2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "20a082f2bde0ffcd8755779fd476a274"
    },
    "pryr": {
      "Package": "pryr",
      "Version": "0.1.4",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "bdc2df7c4155553e5a97a56a6c9f0a0d"
    },
    "ps": {
      "Package": "ps",
      "Version": "1.3.2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "98777535b61c57d1749344345e2a4ccd"
    },
    "purrr": {
      "Package": "purrr",
      "Version": "0.3.4",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "97def703420c8ab10d8f0e6c72101e02"
    },
    "renv": {
      "Package": "renv",
      "Version": "0.9.3",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "c1a367437d8a8a44bec4b9d4974cb20c"
    },
    "reshape2": {
      "Package": "reshape2",
      "Version": "1.4.4",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "bb5996d0bd962d214a11140d77589917"
    },
    "rlang": {
      "Package": "rlang",
      "Version": "0.4.5",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "1cc1b38e4db40ea6eb19ab8080bbed3b"
    },
    "rprojroot": {
      "Package": "rprojroot",
      "Version": "1.3-2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "f6a407ae5dd21f6f80a6708bbb6eb3ae"
    },
    "rstudioapi": {
      "Package": "rstudioapi",
      "Version": "0.11",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "33a5b27a03da82ac4b1d43268f80088a"
    },
    "scales": {
      "Package": "scales",
      "Version": "1.1.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "a1c68369c629ea3188d0676e37069c65"
    },
    "stringdist": {
      "Package": "stringdist",
      "Version": "0.9.5.5",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "734fb9c45f9a36bcd1f1aa0c37a8a5b1"
    },
    "stringi": {
      "Package": "stringi",
      "Version": "1.4.6",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "e99d8d656980d2dd416a962ae55aec90"
    },
    "stringr": {
      "Package": "stringr",
      "Version": "1.4.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "0759e6b6c0957edb1311028a49a35e76"
    },
    "testthat": {
      "Package": "testthat",
      "Version": "2.3.2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "0829b987b8961fb07f3b1b64a2fbc495"
    },
    "tibble": {
      "Package": "tibble",
      "Version": "3.0.1",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "1c61e4cad000e03b1bd687db16a75926"
    },
    "tidyr": {
      "Package": "tidyr",
      "Version": "1.0.2",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "fb73a010ace00d6c584c2b53a21b969c"
    },
    "tidyselect": {
      "Package": "tidyselect",
      "Version": "1.0.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "7d4b0f1ab542d8cb7a40c593a4de2f36"
    },
    "utf8": {
      "Package": "utf8",
      "Version": "1.1.4",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "4a5081acfb7b81a572e4384a7aaf2af1"
    },
    "vctrs": {
      "Package": "vctrs",
      "Version": "0.2.4",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "6c839a149a30cb4ffc70443efa74c197"
    },
    "viridisLite": {
      "Package": "viridisLite",
      "Version": "0.3.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "ce4f6271baa94776db692f1cb2055bee"
    },
    "withr": {
      "Package": "withr",
      "Version": "2.2.0",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "ecd17882a0b4419545691e095b74ee89"
    }
  }
}

renv.lock

FROM rocker/tidyverse:4.0.2
MAINTAINER grixor@gmail.com

RUN Rscript -e "install.packages('pmlbr')"
# or RUN install2.r pmlbr

ADD analysis.R /home/rstudio/

Dockerfile

FROM rocker/tidyverse:4.0.2
WORKDIR /pmlb

# renv and R packages
ENV RENV_VERSION 0.9.3
RUN echo "options(renv.consent = TRUE)" >> .Rprofile
COPY renv.lock .
RUN R -e "install.packages('remotes', repos = c(CRAN = 'https://cloud.r-project.org'))"
RUN R -e "remotes::install_github('rstudio/renv@${RENV_VERSION}')"

# restore from copied lockfile
RUN R -e "renv::restore(confirm = FALSE)"

Dockerfile

can I use renv by itself?

FROM rocker/tidyverse:4.0.2
WORKDIR /tda
RUN apt-get update \
    && apt-get -y install libgsl0-dev \
    && apt -y install libxml2-dev
    
# renv and R packages
ENV RENV_VERSION 0.9.3
RUN echo "options(renv.consent = TRUE)" >> .Rprofile
COPY renv.lock .
RUN R -e "install.packages('remotes', repos = c(CRAN = 'https://cloud.r-project.org'))"
RUN R -e "remotes::install_github('rstudio/renv@${RENV_VERSION}')"
# restore from copied lockfile
RUN R -e "renv::restore(confirm = FALSE)"

Dockerfile

publish
your Docker image

https://docs.github.com/en/free-pro-team@latest/actions/guides/publishing-docker-images#publishing-images-to-github-packages

publish/update images with continuous integration

detailed set up guide

readme.md

1. Install Docker in your local machine: https://docs.docker.com/get-docker/​

2. Create a GitHub account.

3. Go to https://github.com/settings/tokens and create a new personal access token, selecting the read:packages scope.
You can name the token anything, for example, "docker login read-only token".
Then run the following command in the Terminal, substituting your github USERNAME and TOKEN:

docker login --username USERNAME --password TOKEN docker.pkg.github.com

4. Navigate to this repository where you cloned it, then run the following command:

docker run \
  --name pmlb \
  --detach --rm \
  --env DISABLE_AUTH=true \
  --publish 7009:8787 \
  --volume $(pwd):/home/rstudio/pmlb \
  docker.pkg.github.com/epistasislab/pmlb/pmlb-analysis

5. Navigate to http://localhost:7009. An RStudio browser should open up.

6. In the R console, run
 

That's it! You're ready to go!

setwd("~/pmlb")

https://github.com/greenelab/iscb-diversity

  • https://rstudio.github.io/renv/articles/renv.html
  • https://ropenscilabs.github.io/r-docker-tutorial/
  • https://github.com/noamross/nyhackr-docker-talk

https://labs.play-with-docker.com/

Thanks!

Made with Slides.com