

React Workspaces
F1LT3R
https://F1LT3R.io (my blog)
Alistair MacDonald
Senior Software Architect
@ Houghton Mifflin Harcourt


https://hmh.engineering (HMH blog)
HMH / What we do
Interested in working with us?
mailto: alistair.macdonald@hmhco.com
What is a Monorepo?
- Monolith ≠ monorepo - https://gomonorepo.org/
- Google, Facebook, Microsoft, Uber Airbnb & Twitter
- Not a magic bullet!
- Open Source Monorepo Tools
- Lerna (Community)
- Yarn Workspaces (Facebook)
A monorepo
(syllabicabbreviation ofmonolithicrepository)is a software development strategy where code for many projects are stored in the same repository. (Wikipeda)
But why?
Deps Get Complex, Quick

React Monorepos
Why is HMH adopting a monorepo?
- One way of developing react apps as a company
- Shared, reusable components
- Shared, reusable architecture
- Shared, reusable configuration
- No cross referenced PRs
- Simplify upgrade path across many apps
Killing Innovation?

Killing Innovation?
No.
⏳ Less time
- Setting up new apps, pipelines, capabilities etc.
🚀 More time
- Creating new experiences
- Contributing to common architecture
(any engineer can commit contribute to the architecture to the monorepo)
Best Practices
-
Best Practices are great ideals
- Under stress we fall back to complacency
-
Tie ourselves to the ship
- Codify best practices
- Tooling/gating around best practices
- Lock iteration of product to quality gates
-
Continuous Integration
- "Move fast with stable infra" - Zuck
... are great ideals
Where are we?
Since March 2019
- 1x 🐉 Lerna + 🐈 Yarn Workspaces Monorepo
- 7x ⚛️ CRA App's (Create-React-App)
- 100x 🔬 Components
- 5x 🏫 Component Libraries
- 5x 📖 Storybooks - rollup the component libraries
- 18x 👷 Engineers - actively contributing
- 4x 👥 Teams
React Workspaces


React Workspaces

-
Clone: https://github.com/react-workspaces/react-workspaces-playground
-
Storybook: cd packages/storybook; yarn storybook
-
Show repo structure in VS-Code
-
Package-entry: main:src
-
Start the app: cd packages/apps/app-one; yarn start
-
Hot-Reloading Components: Storybook & App-One
-
Deploying to GitHub Pages: yarn deploy
https://github.com/react-workspaces/react-workspaces-playground -
Run tests: yarn test
How does
it work?

Answer:
Turtles
(sort of)

React-Scripts
Fork
Insight: we can use the same create-react-app config across every React part of our Monorepo.
(Its turtles all the way down).

--scripts-version
create-react-app
create-react-app --scripts-version @your-org/react-scripts your-app
A little-known feature of create-react-app lets you specify your own custom version of react-scripts.
package.json
<workspaces-root>
{
"workspaces": {
"packages": [
"packages/apps/*",
"packages/components",
"packages/storybook"
],
"production": true,
"development": true,
"package-entry": "main:src"
}
}
webpack.config.js
(react-scripts fork)
--- a/./facebook/react-scripts/config/webpack.config.js
+++ b/react-workspaces/react-scripts/config/webpack.config.js
@@ -9,7 +9,6 @@
'use strict';
const fs = require('fs');
const isWsl = require('is-wsl');
const path = require('path');
const webpack = require('webpack');
const resolve = require('resolve');
@@ -28,15 +27,14 @@ const WatchMissingNodeModulesPlugin = require('react-dev-utils/WatchMissingNodeM
const ModuleScopePlugin = require('react-dev-utils/ModuleScopePlugin');
const getCSSModuleLocalIdent = require('react-dev-utils/getCSSModuleLocalIdent');
const paths = require('./paths');
const modules = require('./modules');
+const workspaces = require('./workspaces');
const getClientEnvironment = require('./env');
const ModuleNotFoundPlugin = require('react-dev-utils/ModuleNotFoundPlugin');
const ForkTsCheckerWebpackPlugin = require('react-dev-utils/ForkTsCheckerWebpackPlugin');
const typescriptFormatter = require('react-dev-utils/typescriptFormatter');
// @remove-on-eject-begin
const getCacheIdentifier = require('react-dev-utils/getCacheIdentifier');
// @remove-on-eject-end
// Source maps are resource heavy and can cause out of memory issue for large source files.
const shouldUseSourceMap = process.env.GENERATE_SOURCEMAP !== 'false';
@@ -53,12 +51,22 @@ const cssModuleRegex = /\.module\.css$/;
const sassRegex = /\.(scss|sass)$/;
const sassModuleRegex = /\.module\.(scss|sass)$/;
+const workspacesConfig = workspaces.init(paths);
+
// This is the production and development configuration.
// It is focused on developer experience, fast rebuilds, and a minimal bundle.
module.exports = function(webpackEnv) {
const isEnvDevelopment = webpackEnv === 'development';
const isEnvProduction = webpackEnv === 'production';
+ const workspacesMainFields = [workspacesConfig.packageEntry, 'main'];
+ const mainFields =
+ isEnvDevelopment && workspacesConfig.development
+ ? workspacesMainFields
+ : isEnvProduction && workspacesConfig.production
+ ? workspacesMainFields
+ : undefined;
+
// Webpack uses `publicPath` to determine where the app is being served from.
// It requires a trailing slash, or the file assets will get an incorrect path.
// In development, we always serve from the root. This makes config easier.
@@ -279,6 +282,7 @@ module.exports = function(webpackEnv) {
extensions: paths.moduleFileExtensions
.map(ext => `.${ext}`)
.filter(ext => useTypeScript || !ext.includes('ts')),
+ mainFields,
alias: {
// Support React Native Web
// https://www.smashingmagazine.com/2016/08/a-glimpse-into-the-future-with-react-native-for-web/
@@ -330,7 +335,11 @@ module.exports = function(webpackEnv) {
loader: require.resolve('eslint-loader'),
},
],
- include: paths.appSrc,
+ include: isEnvDevelopment && workspacesConfig.development
+ ? [paths.appSrc, workspacesConfig.paths]
+ : isEnvProduction && workspacesConfig.production
+ ? [paths.appSrc, workspacesConfig.paths]
+ : paths.appSrc,
},
{
// "oneOf" will traverse all following loaders until one will
@@ -352,7 +361,12 @@ module.exports = function(webpackEnv) {
// The preset includes JSX, Flow, TypeScript, and some ESnext features.
{
test: /\.(js|mjs|jsx|ts|tsx)$/,
- include: paths.appSrc,
+ include:
+ isEnvDevelopment && workspacesConfig.development
+ ? [paths.appSrc, workspacesConfig.paths]
+ : isEnvProduction && workspacesConfig.production
+ ? [paths.appSrc, workspacesConfig.paths]
+ : paths.appSrc,
loader: require.resolve('babel-loader'),
options: {
customize: require.resolve(
yarn-workspaces.js
(react-scripts fork)
'use strict';
const fse = require('fs-extra');
const path = require('path');
const findUp = require('find-up');
const glob = require('glob');
const loadPackageJson = packagePath => {
try {
const packageObj = fse.readJsonSync(packagePath);
return packageObj;
} catch (err) {
throw err;
}
};
const getWorkspacesRootConfig = dir => {
const packageJsonUp = findUp.sync('package.json', {cwd: dir});
if (packageJsonUp === null) {
return false;
}
const packageObj = loadPackageJson(packageJsonUp);
if (Reflect.has(packageObj, 'workspaces')) {
const workspacesRootConfig = {
root: path.dirname(packageJsonUp),
workspaces: packageObj.workspaces
};
return workspacesRootConfig;
}
const dirUp = path.dirname(dir);
return getWorkspacesRootConfig(dirUp);
};
const getPackagePaths = (root, workspacesList) => {
const packageList = [];
workspacesList.forEach(workspace => {
const workspaceDir = path.dirname(workspace);
const workspaceAbsDir = path.join(root, workspaceDir);
const packageJsonGlob = path.join('**!(node_modules)', 'package.json');
const packageJsonAbsPaths = glob
.sync(packageJsonGlob, {cwd: workspaceAbsDir})
.map(pkgPath => path.join(workspaceAbsDir, pkgPath));
packageList.push(...packageJsonAbsPaths);
});
return packageList;
};
const getDeep = (obj, keyChain) => {
const nextKey = keyChain.shift();
const has = Reflect.has(obj, nextKey);
const val = obj[nextKey];
if (keyChain.length === 0) {
return val;
}
if (has) {
return getDeep(val, keyChain);
}
return false;
};
const resolveBabelLoaderPaths = ({root, workspacesList}, packageEntry) => {
const packageJsonPaths = getPackagePaths(root, workspacesList);
const babelLoaderPaths = [];
packageJsonPaths.map(absPkgPath => {
const packageJson = loadPackageJson(absPkgPath);
const mainSrcFile = getDeep(packageJson, [packageEntry]);
if (mainSrcFile) {
const mainSrcPath = path.dirname(mainSrcFile);
const packageAbsDir = path.dirname(absPkgPath);
const absSrcPath = path.join(packageAbsDir, mainSrcPath);
babelLoaderPaths.push(absSrcPath);
}
});
return babelLoaderPaths;
};
const loadAppSettings = appPackageJson => {
const result = {workspaces: {}, dependencies: {}};
const appPackageObj = loadPackageJson(appPackageJson);
const dependencies = getDeep(appPackageObj, ['dependencies']);
const devDependencies = getDeep(appPackageObj, ['devDependencies']);
if (!dependencies && !devDependencies) return result;
if (dependencies) {
result.dependencies = Object.assign(result.dependencies, dependencies);
}
if (devDependencies) {
result.dependencies = Object.assign(
result.dependencies,
devDependencies
);
}
const reactScripts = getDeep(appPackageObj, ['react-scripts']);
if (!reactScripts) return result;
const workspaces = getDeep(reactScripts, ['workspaces']);
result.workspaces = workspaces;
if (!workspaces) return result;
return workspaces;
};
const guard = (appDirectory, appPackageJson) => {
if (!appDirectory) {
throw new Error('appDirectory not provided');
}
if (typeof appDirectory !== 'string') {
throw new Error('appDirectory should be a string');
}
if (!appPackageJson) {
throw new Error('appPackageJson not provided');
}
if (typeof appPackageJson !== 'string') {
throw new Error('appPackageJson should be a string');
}
};
const getPkg = path => {
const pkgPath = findUp.sync('package.json', {cwd: path});
const pkg = loadPackageJson(pkgPath);
return pkg;
};
const getDeps = pkg => {
const deps = getDeep(pkg, ['dependencies']);
const devDeps = getDeep(pkg, ['devDependencies']);
let dependencies = {};
if (deps) {
dependencies = Object.assign(dependencies, deps);
}
if (devDeps) {
dependencies = Object.assign(dependencies, devDeps);
}
return dependencies;
};
const depsTable = {};
const filterDeps = deps =>
Reflect.ownKeys(deps).filter(dep => Reflect.has(depsTable, dep));
const filterDepsTable = () => {
Reflect.ownKeys(depsTable).forEach(depName => {
const depsList = depsTable[depName].deps;
const workspacesOnlyDeps = filterDeps(depsList);
depsTable[depName].deps = workspacesOnlyDeps;
});
};
const buildDepsTable = srcPaths => {
srcPaths.forEach(path => {
const pkg = getPkg(path);
const name = pkg.name;
const deps = getDeps(pkg);
depsTable[name] = {path, deps};
});
};
const filterSrcPaths = (srcPaths, dependencies) => {
const filteredPaths = [];
srcPaths.forEach(path => {
const pkg = getPkg(path);
if (dependencies && Reflect.has(dependencies, pkg.name)) {
filteredPaths.push(path);
const subDeps = depsTable[pkg.name].deps;
const subPaths = filterSrcPaths(srcPaths, subDeps);
filteredPaths.push(...subPaths);
}
});
return filteredPaths;
};
const init = paths => {
guard(paths.appPath, paths.appPackageJson);
const config = {
root: null,
paths: [],
packageEntry: 'main:src',
development: true,
production: true
};
const {root, workspaces} = getWorkspacesRootConfig(paths.appPath);
const workspacesList = [];
// Normally "workspaces" in package.json is an array
if (Array.isArray(workspaces)) {
workspacesList.push(...workspaces);
}
// Sometimes "workspaces" in package.json is an object
// with a ".packages" sub-array, eg: when used with "nohoist"
// See: https://yarnpkg.com/blog/2018/02/15/nohoist
if (workspaces && !Array.isArray(workspaces) && Reflect.has(workspaces, 'packages')) {
workspacesList.push(...workspaces.packages);
}
if (workspacesList.length === 0) {
return config;
}
console.log('Yarn Workspaces paths detected.');
config.root = root;
const appSettings = loadAppSettings(paths.appPackageJson);
if (Reflect.has(appSettings.workspaces, 'development')) {
config.development = appSettings.workspaces.development ? true : false;
}
if (Reflect.has(appSettings.workspaces, 'production')) {
config.production = appSettings.workspaces.production ? true : false;
}
if (Reflect.has(appSettings.workspaces, 'package-entry')) {
config.packageEntry = appSettings.workspaces['package-entry'];
}
const babelSrcPaths = resolveBabelLoaderPaths(
{root, workspacesList},
config.packageEntry
);
buildDepsTable(babelSrcPaths);
const applicableSrcPaths = [...new Set(filterSrcPaths(
babelSrcPaths,
appSettings.dependencies
))];
console.log(
`Found ${babelSrcPaths.length} path(s) with "${
config.packageEntry
}" entry.`
);
if (applicableSrcPaths.length > 0) {
config.paths.push(...applicableSrcPaths);
}
console.log('Exporting Workspaces config to Webpack.');
console.log(config);
return config;
};
module.exports = {
init
};

Building on
custom react-scripts
at HMH
React-Client-Common
-
Tour of our React monorepo structure
-
codeowners
-
rcc-config in package.json
-
settings, scripts, media
-
packages: suites, apps, components, storybooks
-
build pipes: jenkins, maven
-
-
RCC Menu: yarn menu
-
SGM Storybook
-
Demo app: sgm/app-ri
React-Workspaces
patterns & features
@HMH
No Local Builds
features
-
All builds happen in Jenkins.
-
No transpiled code lives on developer machines.
-
Components are only built when transpiled with an app.
CRA Developer Experience
features
-
For all components and apps:
-
CRA Hot reloading
-
CRA Linting
-
CRA Open browser bugs/warnings in your IDE
-
Relevant Source Only
features
-
Babel-Loader Paths
@react-workspaces/react-scripts builds your app's dependency tree and adds *only* the relevant paths to babel-loader in Webpack; so you only lint and transpile source your app uses.
-
Tree-Shaking
Only the code in your app's dependency tree will be transpiled in CI when your app is built and deployed.
Trunk Based Development
features

Trunk Based Development
features

Say
"No!"
to
Git-Flow
Fix Forward
features
don't roll back

fix forward!
React Workspaces: Roadmap
-
Continue to stay current with create-react-app
-
Dependency tree mapping -
tree of component & app dependencies -
Know what changed? glorified git diff
-
Only test what changed
-
Only build & deploy changed apps in CI
-
yarn-workspaces upgrade required
-
-
Only build & deploy changed apps in CI
-
(currently we build all jobs on PR/merge & promote to prod. manually)
-
-
Support TypeScript, Flow, etc.
-
PR react-scripts fork back into create-react-app (& kill this project)

[the END]
questions?

React Workspaces
By Alistair MacDonald (f1lt3r)
React Workspaces
- 3,765