Готовим

Монорепозиторий

Обо мне

  • SDET с 2018
  • Senior SDET @b2broker
  • Certified node.js application developer (JSNAD 2023)
  • автор TG канала @haradkou_sdet
  • Иногда ментор и консультирую кампании

Telegram: haradkou_sdet

Agenda

  1. Monorepo.

  2. Package manager.

  3. First test app.

  4. First package.

  5. Versioning.

  6. CI/CD.

Frankenstein

Requirements

  1. Many apps

  2. Scalable

  3. Maintainable

  4. Testable

  5. Different Node.js versions

  6. Documentation

Out of Scope

  1. Playwright configuration

  2. Eslint

  3. git hooks (husky)

  4. git branch strategy

  5. Testing libs

  6. Page Object

  7. Data fetching

  8. env management

  9. Containers (aka docker)

Part 1. Monorepo

Input

  • Requirements

  • Team Size

  • Many apps for testing

    • app 1
    • app 2
    • app1--> app2 (e2e)
    • performance

Folder Structure

Package.json

# pwd: <root>
npm init -y && cat package.json
Wrote to /Users/vitali.haradkou/bla-bla-bla/package.json:

{
  "name": "at-workshop",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Modify Package.json

{
  "name": "at-workshop",
  "version": "1.0.0",
  "description": "Monorepo for automation",
  "main": "index.js",
  "private": true,
  "engines": {
    "node": ">=16"
  },
  "engineStrict": true,
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Part 2. Package Manager

Feature pnpm yarn npm
node.js versioning [1] (aka nvm) + - -
monorepo support + + +- (from ver.8)
Installation time [2] avg. (in sec.) 11.2 11.1 12.9
Cross-env script support + + - (cross-env)
is one library? + - (yarn-classic, yarn-pnp) +- (npx)
Isolated node_modules + + -
Autoinstalling peers + - +
Listing licenses + + -
Use Escaping (--) for run with args - - +

Which pm shall I choose?

Monorepo support

# pnpm-workspace.yaml
packages:
  # all packages in direct subdirs of packages/
  - "packages/*"
  # all packages in subdirs of apps/
  - "apps/**"

Проблема 0

npm install is-even

Проблема 0

pnpm install is-even

Проблема 1. env vars

{
  "name": "my-awesome-package",
  "scripts": {
    "test": "NODE_ENV=test node test.js"
  }  
}

Hello Windows!

Проблема 1. env vars

# pnpm-workspace.yaml
shellEmulator: true

Фича позаимствована у yarn

Проблема 2. Разные версии nodejs

  • nvm
    • for windows
    • for linux (other)
  • А как сделать версии строгими?
# pnpm-workspace.yaml
shellEmulator: true
useNodeVersion: 22.14.0 # pnpm exec <cli command>
engineStrict: true # panic when node js different
nodeVersion: 22.14.0 # (node -v)

Проблема 3. Разные версии пакетного менеджера

packageManagerStrictVersion

When enabled, pnpm will fail if its version doesn't exactly match the version specified in the packageManager field of package.json.

# pnpm-workspace.yaml
shellEmulator: true
useNodeVersion: 22.14.0 # pnpm exec <cli command>
engineStrict: true # panic when node js different
nodeVersion: 22.14.0 # (node -v)
packageManagerStrictVersion: true

Проблема 4. Версии пакетов

Fakerjs drama

This project was originally created and hosted at https://github.com/marak/Faker.js/ - however around 4th Jan, 2022 - the author decided to delete the repository (for unknown reasons).

Bump exact version by default

# pnpm-workspace.yaml
shellEmulator: true
useNodeVersion: 22.14.0 # pnpm exec <cli command>
engineStrict: true # panic when node js different
nodeVersion: 22.14.0 # (node -v)
packageManagerStrictVersion: true
save-prefix: '' # exact version

Итоговый фаил

# pnpm-workspace.yaml
packages:
  - apps/*
  - packages/*
shellEmulator: true
useNodeVersion: 22.14.0 # pnpm exec <cli command>
engineStrict: true # panic when node js different
nodeVersion: 22.14.0 # (node -v)
packageManagerStrictVersion: true
save-prefix: '' # exact version
saveWorkspaceProtocol: rolling

Add Typescript

# pwd: <root>

pnpm add typescript @types/node ts-node @tsconfig/node18 -D -w

Tsconfig.json

{
  "extends": "@tsconfig/node18",
}
{
  "$schema": "https://json.schemastore.org/tsconfig",
  "display": "Node 18",

  "compilerOptions": {
    "lib": ["es2022"],
    "module": "commonjs",
    "target": "es2022",

    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true,
    "moduleResolution": "node"
  }
}

Part 3. First Test Application

Folder Structure

App1/package.json

{
  "name": "app1",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Modify some

{
  "name": "@apps/app1",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

add 🎭 for app1

# pwd: <root>/apps/app1

pnpm create playwright

Page Object. Login

// filename: app1/pages/login.ts

export type User = {
  username: string;
  password: string;
}

export class Login {
  async login(user: User) {
    console.log('APP1. LoginPage. USER:', user);
  }
}

Page Object. index

// filename: app1/pages/index.ts

import { Login, type User } from './login';

export { Login, User };
export default { Login };

App1. index

// filename: app1/index.ts

// export for futher usage
export * from './pages';

App1. tsconfig.json

{
  "extends": "../../tsconfig.json",
  "compilerOptions": {
    "declaration": true,
    "outDir": "build"
  },
  "include": [
    "pages/**/*.ts",
    "index.ts"
  ]
}

App1. Build Step

{
  "name": "@apps/app1",
  "version": "1.0.0",
  "description": "Automation for application 1",
  "main": "index.js",
  "scripts": {
    "build": "tsc -p ./tsconfig.json"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@playwright/test": "1.28.1"
  }
}

Monorepo. Build Step

{
  "name": "at-workshop",
  "version": "1.0.0",
  "description": "Monorepo for automation",
  "main": "index.js",
  "private": true,
  "engines": {
    "node": ">=16"
  },
  "engineStrict": true,
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "build": "pnpm -r --parallel --if-present build"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@tsconfig/node18": "1.0.1",
    "@types/node": "18.11.14",
    "ts-node": "10.9.1",
    "typescript": "4.9.4"
  }
}

Test build

# pwd: <root>
pnpm build

> at-workshop@1.0.0 build /bla-bla-bla/at-workshop
> pnpm -r --parallel --if-present build

apps/app1 build$ tsc -p ./tsconfig.json
apps/app1 build: Done

App1. Update main

{
  "name": "@apps/app1",
  "version": "1.0.0",
  "description": "Automation for application 1",
  "main": "build/index.js",
  "scripts": {
    "build": "tsc -p ./tsconfig.json"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@playwright/test": "1.28.1"
  }
}

Structure

E2E. 🎭

# pwd: <root>/apps/e2e-app1-app2

pnpm init && pnpm create playwright
Wrote to /Users/vitali.haradkou/bla-bla-bla/at-workshop/apps/e2e-app1-app2/package.json
{
  "name": "e2e-app1-app2",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Done in 1.9s
Writing playwright.config.ts.
Writing tests/example.spec.ts.
Writing tests-examples/demo-todo-app.spec.ts.
Writing package.json.

Happy hacking! 🎭

E2E. tsconfig.json

{
  "extends": "../../tsconfig.json",
  "compilerOptions": {
    "declaration": true,
    "outDir": "build"
  }
}

E2E. intall app1

pnpm add @apps/app1
../..                                   

dependencies:
+ @apps/app1 1.0.0 <- ../app1

Done in 1.6s

E2E. package.json

{
  "name": "@apps/e2e-app1-app2",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "playwright test"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@playwright/test": "1.28.1"
  },
  "dependencies": {
    "@apps/app1": "workspace:*"
  }
}

E2E. Add import

// filename: <root>/apps/e2e-app1-app2/tests/example.spec.ts
import { test } from '@playwright/test';

import { Login } from '@apps/app1/pages';


test('homepage has title and links to intro page', async ({ }) => {
  const login = new Login();
  login.login({ username: 'qwe', password: 'qwe' });
});

E2E. Test

# pwd: <root>/apps/e2e-app1-app2
pnpm test

> e2e-app1-app2@1.0.0 test /Users/vitali.haradkou/bla-bla-bla/at-workshop/apps/e2e-app1-app2
> playwright test


Running 3 tests using 3 workers
[chromium] › example.spec.ts:6:1 › homepage has title and links to intro page
APP1. LoginPage. USER: { username: 'qwe', password: 'qwe' }
[firefox] › example.spec.ts:6:1 › homepage has title and links to intro page
APP1. LoginPage. USER: { username: 'qwe', password: 'qwe' }
[webkit] › example.spec.ts:6:1 › homepage has title and links to intro page
APP1. LoginPage. USER: { username: 'qwe', password: 'qwe' }

  3 passed (754ms)

To open last HTML report run:

  npx playwright show-report

Part 4. First Package

Зависимости

  • dependencies: Packages required by your application in production.

 

  • devDependencies: Packages that are only needed for local development and testing.

Dependencies

Dev Dependencies

Init

# pwd: <root>/packages/async
pnpm init
Wrote to /Users/vitali.haradkou/bla-bla-bla/at-workshop/packages/async/package.json

{
  "name": "@utils/async",
  "version": "1.0.0",
  "description": "Async Utility for project X",
  "main": "build/index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Foreach

// filename: <root>/packages/async/foreach.ts
type Callback<T> = (item: Awaited<T>, index: number, array: readonly T[]) => Promise<void> | void;
type Options = {
  serial?: boolean;
};

export default async function foreach<T>(array: T[], callback: Callback<T>, options?: Options) {
  const serial = options?.serial ?? false;

  if (serial) {
    for (let index = 0; index < array.length; index++) {
      const element = await array[index];
      await callback(element, index, array);
    }
    return;
  }
  await Promise.all(array.map(async (v, id, arr) => {
    callback(await v, id, arr);
  }));
}

Index

// filename: <root>/packages/async/index.ts
import foreach from './foreach';

export { foreach };
export default { foreach };

Update package.json

{
  "name": "@utils/async",
  "version": "1.0.0",
  "description": "",
  "main": "build/index.js",
  "exports": {
    ".": "./build/index.js",
    "./package.json": "./build/index.js",
    "./foreach": "./build/foreach.js"
  },
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "build": "tsc -p ./tsconfig.json"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Monorepo. Build

# pwd: <root>
pnpm build
> at-workshop@1.0.0 build /Users/vitali.haradkou/bla-bla-bla/at-workshop
> pnpm -r --parallel --if-present build

Scope: 3 of 4 workspace projects
packages/async build$ tsc -p ./tsconfig.json
apps/app1 build$ tsc -p ./tsconfig.json
apps/app1 build: Done
packages/async build: Done

Add to E2E

# pwd: <root>/apps/e2e-app1-app2
pnpm add @utils/async

dependencies:
+ @utils/async 1.0.0 <- ../../packages/async

Done in 1.8s

Test it!

import { test } from '@playwright/test';

import { Login } from '@apps/app1/pages';
import foreach from '@utils/async/foreach';

test('homepage has title and links to intro page', async ({ }) => {
  const login = new Login();
  login.login({ username: 'qwe', password: 'qwe' });
  foreach([1, Promise.resolve(2), 3], (item) => {
    console.log(`async Works: ${item}`);
  });
});

It Works!

# pwd: <root>/apps/e2e-app1-app2
pnpm test

> @apps/e2e-app1-app2@1.0.0 test /Users/vitali.haradkou/bla-bla-bla/at-workshop/apps/e2e-app1-app2
> playwright test

Running 3 tests using 3 workers
[firefox] › example.spec.ts:7:1 › homepage has title and links to intro page
APP1. LoginPage. USER: { username: 'qwe', password: 'qwe' }
async Works: 1
async Works: 2
async Works: 3

Part 5. Versioning

Versioning Variants

  1. Independent

  2. Independent Patches/Minors 

  3. Dependent

  4. NoVersion

Git based

  1. Branch based

  2. Tag based

  3. Commit based

Devops with Google

# pwd: <root>
pnpm add zx -Dw
// filename: <root>/scripts/release.mjs
await $`cat package.json | grep name`

let branch = await $`git branch --show-current`
await $`dep deploy --branch=${branch}`

await Promise.all([
  $`sleep 1; echo 1`,
  $`sleep 2; echo 2`,
  $`sleep 3; echo 3`,
])

let name = 'foo bar'
await $`mkdir /tmp/${name}`

Dependent

Independent

Mixed. Major/Minor

Comparison

Affect Root? Dependent InDependent Mixed NoVersion
Patch + - - -
Minor + - +- (strategy) -
Major + - + -

Dependent or Mixed !

Angular@2

Changeset init

# pwd: <root>
pnpm add -Dw @changesets/cli && pnpm changeset init

🦋  Thanks for choosing changesets to help manage your versioning and publishing
🦋  
🦋  You should be set up to start using changesets now!
🦋  
🦋  info We have added a `.changeset` folder, and a couple of files to help you out:
🦋  info - .changeset/README.md contains information about using changesets
🦋  info - .changeset/config.json is our default config

Config

{
  "$schema": "https://unpkg.com/@changesets/config@2.2.0/schema.json",
  "changelog": "@changesets/cli/changelog",
  "commit": false,
  "fixed": [],
  "linked": [],
  "access": "restricted",
  "baseBranch": "main",
  "updateInternalDependencies": "patch",
  "ignore": []
}

Run Versioning

WTF File?

Whats inside?

---
"@utils/async": major
"@apps/app1": patch
"@apps/e2e-app1-app2": patch
---

Add async package

Run Version

pnpm changeset version
🦋  All files have been updated. Review them and commit at your leisure

ave Version !

Part 6. CI/CD

Prepare script

#!/usr/bin/env zx
import { $, argv, fs, os } from "zx";

const type = argv.type || argv.t;
let msg = '';
const upcomingFile = 'upcoming.md';

function assertType(type) {
  if (!['major', 'minor', 'patch'].includes(type)) {
    throw new Error(`Invalid type: ${type}`);
  }
}

const template = (type, msg) => `
---
"@apps/app1": ${type}
"@apps/e2e-app1-app2": ${type}
"@utils/async": ${type}
---

${msg}
`;

assertType(type);

if (!fs.existsSync(upcomingFile)) {
  throw new Error(`! No upcoming.md found, create and fill ${upcomingFile} first`);
}

msg = await fs.readFile(upcomingFile, 'utf8');

await $`chmod -R 777 .changeset`; // update permissions for execution
await $`chmod -R 777 releases`; // update permissions for releases
// await fs.copyFile(upcomingFile, `.changeset/${upcomingFile}`);

// create empty changeset file
await $`pnpm changeset --empty`;
await $`exit 0`; // exit with success

const { stdout: filename } = await $`find ${process.cwd()}/.changeset -type f -name '*.md' ! -name 'README.md'`; // find filename

await fs.writeFile(filename.trim(), template(type, msg)); // write changeset

await $`pnpm changeset version`; // add changeset

const { stdout: version } = await $`pnpm version ${type} --no-git-tag-version`;

// write to releases folder release.
await fs.writeFile(`releases/${version.trim()}.md`, msg);
// fill upcoming.md release with empty string
console.log('cleanup upcoming file');
await fs.writeFile(upcomingFile, os.EOL);

const commitMsg = `${'Release ' + version.trim() + '\n\n' + msg}`;

// git commit message
await $`git add .`;
await $`git commit -m ${commitMsg}`;


// create git tag without pushing
await $`git tag -a ${version.trim()} -m ${commitMsg}`;

// push tag
await $`git push --follow-tags origin`;

// push main
await $`git push -u origin main`;

Monorepo. Release.

{
  "name": "at-workshop",
  "version": "1.0.0",
  "description": "Monorepo for automation",
  "main": "index.js",
  "private": true,
  "engines": {
    "node": ">=16"
  },
  "engineStrict": true,
  "scripts": {
    "release": "pnpm zx ./scripts/release.mjs",
    "test": "echo \"Error: no test specified\" && exit 1",
    "build": "pnpm -r --parallel --if-present build"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@changesets/cli": "2.25.2",
    "@tsconfig/node18": "1.0.1",
    "@types/node": "18.11.14",
    "ts-node": "10.9.1",
    "typescript": "4.9.4",
    "zx": "7.1.1"
  }
}
# This workflow will run tests using node and then publish a package to GitHub Packages when a release is created
# For more information see: https://help.github.com/actions/language-and-framework-guides/publishing-nodejs-packages

name: Release.

on:
  workflow_dispatch:
    inputs:
      release_type:
        type: choice
        options:
          - patch
          - minor
          - major
        description: "version type, can be 'patch', 'minor', 'major'"
        required: true
        default: patch
    tags:
      - "*"

jobs:
  publish:
    name: "Build and publish to npm"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
      - name: Setup pnpm
        uses: pnpm/action-setup@v2.2.2
        with:
          version: 7.18.1
          run_install: |
            - args: []
      - run: pnpm build
      - name: Setup git config
        run: |
          git config --global user.name "github-actions[bot]" &&
          git config --global user.email "github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>"
          echo "GIT_USER=${GITHUB_ACTOR}:${{ secrets.GITHUB_TOKEN }}" >> $GITHUB_ENV
      - name: Get upcoming changes
        run: |
          RELEASE_LOG=$(< ./upcoming.md)
      - run: pnpm release --type ${{ github.event.inputs.release_type || 'patch' }}
      - name: Get release log from changelog
        run: |
          TAG=`git describe --tags --abbrev=0`
          echo "workaround described here: https://trstringer.com/github-actions-multiline-strings/"
          echo "RELEASE_LOG<<EOF" >> $GITHUB_ENV
          echo "$RELEASE_LOG" >> $GITHUB_ENV
          echo "EOF" >> $GITHUB_ENV
      - uses: fregante/release-with-changelog@v3
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          commit-template: "- {title} ← {hash}"
          template: |
            ### Changelog

            ${{ env.RELEASE_LOG }}

            ### PRs

            {commits}

            Full changelog: {range}

            made with ❤ from Vitali Haradkou

Test versioning

Run versioning

Run versioning

Спасибо

Bonus: Blog

References

  1. Versioning. https://pnpm.io/cli/env#use
  2. Intall benchmark. https://pnpm.io/benchmarks#lots-of-files 
  3. pnpm licenses https://pnpm.io/cli/licenses
  4. Yarn licenses: https://classic.yarnpkg.com/en/docs/cli/licenses
  5. Workspace file: https://pnpm.io/pnpm-workspace_yaml
  6. Workspace protocol: https://pnpm.io/npmrc#save-workspace-protocol
  7. Demo Repo: https://github.com/vitalics/Workshop
  8. Personal Blog: https://github.com/vitalics/vitalics/blob/main/blog/README.md

Heisenbug 2025. Monorepo

By vitalic gorodkov

Heisenbug 2025. Monorepo

  • 191