JS Monorepos

CVJS June 2022

Michael Holroyd, PhD

@meekohi

meekohi.com

Sponsor: VAULT VIRGINIA

  • Co-working
  • Private Offices
  • Meeting/Event Space
  • Workshops / Programming
  • The Bradbury Café

  🖧 ec67fa

  /app

    /lib

Monolith

  🖧 d76ae2

  /app1

  /app2

  /lib

Monorepo

  🖧 f1278d

  /app1

Polyrepo

  🖧 e3aeba

  /app2

  🖧 c45ad1

  /lib

  🖧 ec67fa

  /app

    /lib

Monolith

  🖧 d76ae2

  /app1

  /app2

  /lib

Monorepo

  🖧 f1278d

  /app1

Polyrepo

  🖧 e3aeba

  /app2

  🖧 c45ad1

  /lib

  npm packages

  npm packages

  npm packages

  🖧 ec67fa

  /app

    /lib

Monolith

  🖧 d76ae2

  /app1

  /app2

  /lib

Monorepo

  🖧 f1278d

  /app1

Polyrepo

  🖧 e3aeba

  /app2

  🖧 c45ad1

  /lib

  🖧 ec67fa

  /app

    /lib

Monolith

  🖧 d76ae2

  /app1

  /app2

  /lib

Monorepo

  🖧 f1278d

  /app1

Polyrepo

  🖧 e3aeba

  /app2

  🖧 c45ad1

  /lib

  🖧 d76ae2

  /app1

  /app2

  /lib

Monorepo

  🖧 f1278d

  /app1

Polyrepo

  🖧 e3aeba

  /app2

  🖧 c45ad1

  /lib

 app2

 lib v2

  v2

 

 app1

 lib v2

  v2

 app2

 lib v1

  v1

 app1

 lib v2

  v2

polyrepo enables natural dependencies on old artifacts

  🖧 ec67fa

  /app

    /lib

Monolith

  🖧 d76ae2

  /app1

  /app2

  /lib

Monorepo

  🖧 f1278d

  /app1

Polyrepo

  🖧 e3aeba

  /app2

  🖧 c45ad1

  /lib

  🖧 ec67fa

  /app

    /lib

    /lib2

Monolith

  🖧 d76ae2

  /app1

  /app2

  /lib

  /lib2

Monorepo

  🖧 f1278d

  /app1

Polyrepo

  🖧 e3aeba

  /app2

  🖧 c45ad1

  /lib

---

 

+++

   

---

---

 

+++

   

---

   

---

   

  🖧 76ad32

  /lib2

+++

 

Anti-pattern: versioned packages in a monorepo

  v2.1

  /app1

  /app2

  /lib

  lib v2.0

Anti-pattern: code "co-location"

  v2.1

  /app1

    /lib

  /app2

    /lib

JS Monorepo Tooling (2011 State of JS)

Are monorepos "new"?

Used to call these things a "shared codebase"

 

Just use `make`? (Mandatory Ron DuPlain shout-out)

 

Natural progression when refactoring a growing project -- main question is when to pull something out of "the app" and make it an independent package

Workspaces

Initial set of tools were just to solve the practical problem of making JS imports dynamic (so you don't have to publish and update package.json just to test)

 

yarn beat npm to this (2018 vs 2021), but now both are roughly equivalent:

npm/yarn link -- creates a symlink in /node_modules/your-lib

 

{
  "name": "monorepo",
  "private": true,
  "workspaces": {
    "packages": [
      "lib",
      "ui",
      "auth"
    ]
  }
}

pnpm also solves this problem, tries to save storage and avoid duplicates.

"Just works"

 

What more do we need?

Monorepo tools

How to build the whole repo if the order matters? How do you publish?

How to only run tests on packages that have changed?

turborepo / nx: re-invent make, but with JSON.

{
  "turbo": {
    "baseBranch": "origin/main",
    "pipeline": {
      "build": {
        "dependsOn": ["^build"],
        "outputs": [".next/**"]
      },
      "test": {
        "dependsOn": ["^build"],
        "outputs": []
      }
    }
  }
}

nx is strongly opinionated about the build system (so that it can parallelize/optimize the build)

 

turborepo is still really new (launches beginning of 2022), purchased by Vercel

Monorepo tools

How to build the whole repo if the order matters? How do you publish?

How to only run tests on packages that have changed?

lerna: unified scripting (task runner)

{
  ...
  "dependencies": {
    "lib": "*",
    "auth": "*"
  }
}
$ lerna run build

    ✔  auth:build (501ms)
    ✔  lib:build (503ms)
    ✔  app:build (670ms)

 —————————————————————————————————————————————————————————————————————————————

 >  Successfully ran target build for 3 projects (1s)

Amazing drama last couple years. Finally got "adopted" by Nrwl (same people as nx) 

Monorepo tools

How to build the whole repo if the order matters? How do you publish?

How to only run tests on packages that have changed?

rush: Microsoft-backed monorepo tool

 

+ Automatic dependency graph, incremental builds

+ Enterprisey features (mandatory changelogs, etc)

 

- Introduces a lot of custom commands and config (viz. nx which has little config, but expects you to organize your code in a predictable way)

  • Atomic Changes - make cross cutting code changes across multiple applications (eg /frontend and /backend) in one atomic commit
     
  • Large-scale code refactoring – Since developers have access to the entire project, refactors can ensure that every piece of the project continues to function after a refactor.
     
  • Developer mobility - Consistent way of building and testing applications. Easily search across all projects

MONOREPO GOOD

  • Atomic Changes - make cross cutting code changes across multiple applications (eg /frontend and /backend) in one atomic commit
     
  • Large-scale code refactoring – Since developers have access to the entire project, refactors can ensure that every piece of the project continues to function after a refactor.
     
  • Developer mobility - Consistent way of building and testing applications. Easily search across all projects

MONOLITH GOOD

  • Other people can change your team's code - solvable with CODEOWNERS, but is this a real problem?
     
  • How to CI test "just the part that changed" - annoying problem that gets worse as you scale. Either your CI tool deeply understands your dependency graph (nx, turborepo) or you spend extra to re-build and re-test everything during CI.
     
  • Security / Auditing - exceptions where keeping part of your codebase independent makes external tasks easier.
     
  • Big chonking codebase - if your codebase is so big your IDE is choking and people are only checking out subsets of it anyway...

MONOREPO BAD?

Demo

Demo

Thanks

Made with Slides.com