Dependencies

dependencies, dependencies and dependencies

git clone git@github.com:sithmel/dependencies-workshop.git

Before we begin

🇬🇧

⌚ Time

⛅ Weather

🇮🇹

⌚ Tempo

⛅ Tempo

Package Dependencies

Purpose:

download and make available npm packages to be consumed in Node.js

  • Uses package.json (*dependencies)
  • Uses semantic versioning (semver)
  • Copy packages in a certain folder structure
  • Manages lockfile
  • Offers other utilities

"Npm" flattens packages under node_modules

pnpm uses hardlinks and symlinks

🗒️ Let's put it into practice

mkdir dependencies-workshop
cd dependencies-workshop
npm init -y
npm install measure-speed
npm install memoize-cache

Check what happens to:

  • node_modules
  • lockfile
git checkout package

or

Package managers try to dedupe dependencies. But can also manage multiple versions

See also:

  • npm link
  • npm dedupe

Node Dependencies

Purpose:

while executing a Node.js script. it loads the modules in memory and execute them in the correct order

❗Note

Packages are resolved by path: package.json dependencies are only used for installation.

For example: nested dependencies have the precedence.​

🗒️ Let's put it into practice

mkdir dependencies-workshop
cd dependencies-workshop
npm init -y
mkdir -p node_modules/dep1
cd node_modules/dep1
npm init -y
echo "module.exports = 'dep1'" > index.js
# repeat for dep2 and dep3

or

git checkout node-commonjs

🗒️ Let's put it into practice

Add index.js:

const dep1 = require('dep1');
const dep2 = require('dep2');
const dep3 = require('dep3');
console.log(dep1, dep2, dep3)
node index.js

Try:

🗒️ Let's put it into practice

  • Let's try to add nested dependencies with the same name
  • the file position determines the behaviour!
  • Try create/importing new modules
  • Try create and import a json

Multiple versions of a dependency are supported. But:

  • Global namespace
  • Mismatched "types"

(we will talk later about it)

Packages: https://nodejs.org/api/packages.html

It works in 2 formats:

  • commonjs
  • ESM (sort of)
// commonjs
const something = require('./something');

// ESM
import something from './something.mjs'; // this is a path or a package

// "real ESM" used by the browser
import something from './something.js'; // this is a module name or a URL

// NOTE: a browser module is the equivalent of package in node.js

Commonjs:

https://nodejs.org/api/modules.html
ESM:
https://nodejs.org/api/esm.html

Full documentation

and algorithm to resolve files 🗺️

Node.js: Modules vs Packages

Modules: are just files in a package

Packages: a folder with a package.json that can be published

 

Import rules of packages and modules are determined by the package.json

Package.json

https://nodejs.org/api/packages.html#nodejs-packagejson-field-definitions

and

https://docs.skypack.dev/package-authors/package-checks

  • "main" The default module when loading the package
  • "type": "module" load modules as ESM, instead of commonjs
  • "exports" maps package entrypoints per environment
{
  "main": "./index-cjs.js",
  "type": "module",
  "exports": {
    "require": "./index-cjs.js",
    "import": "./index-esm.js",
    "default": "./index-esm.js",
    // "node", "browser", "deno", etc.
    // (env-specific entrypoints, as needed)
  }
}

Package.json example

https://nodejs.org/api/modules.html#all-together

🗒️ Let's put it into practice

Let's convert our repo to use ESM

git checkout node-esm

or

🗒️ Let's put it into practice

  • Try create/importing new modules
  • New modules need the extension (not packages)
  • try importing a json
git checkout node-esm

or

  • ESM includes a (recent) spec to import files different than javascript. In node it works with json (which was already supported by commonjs)
import info from `./package.json` with { type: "json" };

Bundler Dependencies

Purpose:

allow js code to be bundled together in a single file

  • webpack/next.js are able to resolve paths and packages using node resolution algorithm
  • rollup needs @rollup/plugin-node-resolve

They work in 2 formats:

  • commonjs
  • ESM (sort of)

Package.json

Special fields for bundlers

Same as node.js but we also have

  • "module" same as main but for ESM modules. This is not a standard, but only a convention used by bundlers
  • "sideEffect" if true we declare that the functions in our package don't have side effects and therefore they can be removed by treeShaking
{
  "main": "./index-cjs.js",
  "type": "module",
  "module": "./index-esm.js", // ignored by Node.js
  "sideEffects": true,  // ignored by Node.js
  "exports": {
    "require": "./index-cjs.js",
    "import": "./index-esm.js",
    "default": "./index-esm.js",
    // "node", "browser", "deno", etc.
    // (env-specific entrypoints, as needed)
  }
}

Package.json example

  • Commonjs works out of the box on webpack/next.js but rollup needs @rollup/plugin-commonjs plugin
  • extensions are optional in both ESM and commonjs
  • module format is recognised automatically and can even be mixed
  • everything different from js can be transformed in js by a plugin

ESM can be "treeshaken":

Unimported code is removed from the bundle.

 

It is automatic but can be improved using sideEffects option or marking the functions as pure.

const x = */@__PURE__*/eliminated_if_not_called()

Webpack internals

 

  • A loader is inserted in each bundle
  • Every module is wrapped in a closure
  • support for node.js features is added automatically (__dirname, process, buffer etc.)

Rollup internals

 

  • All code is bundled in the same scope
  • No loader, just native js
  • support for node.js features can be added by a plugin

Vite internals

 

  • In dev mode it uses esbuild/native esm
  • In production mode uses rollup
#
# dev mode
#
vite # aliases: `vite dev`, `vite serve`

#
# production mode (using rollup)
#
vite build
vite preview

🗒️ Let's put it into practice

  • Let's install rollupjs to our repo
  • Note: our manual dependencies will be removed if we run npm install. Let's save and restore them
npm i --save-dev rollup @rollup/plugin-node-resolve @rollup/plugin-commonjs
git checkout bundler

or

🗒️ Let's put it into practice

Add script to our package.json

"rollup index.js --file output.js --format iife -p @rollup/plugin-node-resolve,commonjs"

Run and look at the output

Dynamic imports

import('module').then((module) => {
// module is loaded
});

The module is created in a separated bundle (chunk). The main bundle will have a piece of code that implements the loading of the chunk.

🗒️ Let's put it into practice

Configure to import dynamically one of the dependencies. Some adjustment will be necessary.

Then look at the resulting bundle.

If you declare a dependency as external the bundler won't attempt to resolve it and the import will remain untouched.

Externals

Multiple versions of a dependency are supported. But:

  • Global namespace
  • Mismatched "types"
  • Bundle bloat

(More about it later)

Also

  • minification
  • sourcemaps (with integration with transpilers)

About transpilation

Purpose:

allows to use languages other than javascript and unsupported features

Transpilation happens before bundling and execution

sourcemaps

sourcemaps

We are talking about transpilation ESM to commonjs

 

Transpilation from commonjs to ESM is not possible

{
  "ignore": ["node_modules/**/*"],
  "presets": [
    ["@babel/preset-typescript"],
    [
      "@babel/preset-env",
      {
        "loose": true,
        "modules": false
      }
    ],
    "@babel/preset-react"
  ],
}

Set "modules: false" to disable transpilation from ESM to commonjs

The "module" option in tsconfig determine the output

In terms of resolving the dependencies

 

Typescript interoperability with commonjs is  an extremely complicated topic:

https://www.typescriptlang.org/docs/handbook/modules/reference.html

Package.json

in addition to the other fields we have

  • "types" Typescript definition. Ignored by Node and bundlers
{
  "main": "./index-cjs.js",
  "type": "module",
  "module": "./index-esm.js", // ignored by Node.js
  "sideEffects": true,  // ignored by Node.js
   "types": "./index.d.ts", // ignored by Node.js and bundlers
  "exports": {
    "require": "./index-cjs.js",
    "import": "./index-esm.js",
    "default": "./index-esm.js",
    // "node", "browser", "deno", etc.
    // (env-specific entrypoints, as needed)
  }
}

Package.json example

🗒️ Let's put it into practice

Let's convert our main module in Typescript and let's see how it works (in the context of bundling).

Note: npm install will remove your dep1/dep2/dep3 folders

npm install typescript rollup-plugin-typescript2 --save-dev

npx tsc --init
git checkout bundler-transpiled

Configure include, module and moduleResolution in tsconfig.json.

This is the new script

or

rollup index.ts --file output.js --format iife -p @rollup/plugin-node-resolve,commonjs,typescript2

About multiple versions

💥 They are managed up to a point but tricky

📦 Package dependencies

depend on SEMVER and deduped when possible

SEMVER

https://semver.org/

Package1

  "my_dep": "^1.0.0" 👉 ok: 1.0.3 - 1.1.2  ko: 2.0.0

  "another_dep": "~2.2.0" 👉 ok: 2.2.3 ko: 2.2.1

Package2

  "my_dep": "^1.2.5" 👉 ok: 1.3.4 - 1.2.6  ko: 2.0.0

  "another_dep": "~2.3.0" 👉 ok: 2.3.1 ko: 2.4.0

Result (Package1 + Package2)

"my_dep": "1.2.5"

"another_dep": "2.2.0"

"another_dep": "2.3.0"

⚠️ Duplicated!

🖥️ Node.js dependencies

and

🥡 Bundle dependencies

 

They depend on the folder position

Global namespace (and side effects)

// every lib version overwrites the previous one
window.something = something


// every lib version ADD a new event handler
document.body.addEventListener('click', doSomething);


// every lib version repeat the same initialisation
fetch("https://something.com/specialconfig").then(dosomething)

Mismatched "types"

// this object is generated by version 1 of a package
const data = {
  result: "here is the result"
}

// this is a function in the version 2 of the same package
function printData(data) {
  console(data.output); // error!
}

Bundle bloat

Duplicated dependencies will import an entire new dependency tree!

Browser Dependencies

ESM full documentation:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules

ESM works natively in the browser while commonjs is not supported

<script type="module" src="main.js"></script>

<script type="module">
  /* JavaScript module code here */
</script>

Only allowed in modules

<script type="importmap">
  {
    "imports": {
      "square": "./shapes/square.js"
    }
  }
</script>
import { name, draw, reportArea, reportPerimeter } from "square";

Support urls or bare names

import("./modules/myModule.js").then((module) => {
  // Do something with the module.
});

Support dynamic loading (using promises)

🇬🇧 dependencies

📦 Package dependencies

🖥️ Node.js dependencies

🥡 Bundle dependencies

♻️ Transpiled dependencies

🌎 Browser dependencies

🤯

the end? 👋

Dependencies and dependencies

By Maurizio Lupo

Made with Slides.com

Dependencies and dependencies

  • 108

More from Maurizio Lupo