The Future Belongs to You

(and your JavaScript)

Øredev 2017

Gleb Bahmutov, PhD

Friday Nov 10, 14:20

Homo Agitatus

JavaScript !!!

undefined is not a function

About: @bahmutov

  • Startups / medium size / MathWorks
  • VP of Eng at Cypress.io

"Testing, the way it should be"

  • Computer vision background (C/C++/C#)

CoffeeScript then JavaScript

About: @bahmutov

A software product

User

Data

Servers

UI

C++ / C#

SilverLight

Java

A software product

User

Data

Servers

UI

JavaScript

JavaScript

Desktop

Deploy

Mobile app

JavaScript

Legend

We are going to cover:

  1. Where is JavaScript?
  2. JavaScript versions
  3. Main tools

JavaScript around us

  • Browser

  • Server

  • Desktop

  • Mobile

  • Gray areas:

    • IoT, robots, code snippets, build scripts, sw

<div onclick="alert('hi')">Click me</div>

Modern JavaScript

in the Browser

<script src="app.js"></script>
<div id="app" />

Single Page Applications (SPA)

React, Angular, Vue, Ember

Modern JavaScript

in the Browser

No longer relevant

JScript

VBScript

Dart

Flash

Silverlight

Java

Browser APIs

Payment Request API

Speech Synthesis API

Speech Recognition API

Replacing mobile applications

Modern JavaScript

Browser/mobile

Two choices

Progressive Web App

I don't want to download 200MB app to read tweets

I can't afford to download 200MB app to read tweets

me

billions of people

2nd: ReactNative

Code Web App convert to Native app

node sever.js -p 3000

Modern JavaScript

on Server

const http = require('http')
const hostname = '127.0.0.1'
const port = 3000

const server = http.createServer((req, res) => {
  res.statusCode = 200
  res.setHeader('Content-Type', 'text/plain')
  res.end('Hello World\n')
})
server.listen(port, hostname, () => {
  console.log(`Server running at 
    http://${hostname}:${port}/`)
})

Modern JavaScript

on Desktop

Build cross platform desktop apps with JavaScript, HTML, and CSS

# Clone the Quick Start repository
$ git clone https://github.com/electron/electron-quick-start

# Go into the repository
$ cd electron-quick-start

# Install the dependencies and run
$ npm install && npm start
const electron = require('electron')
let mainWindow = new BrowserWindow()

mainWindow.loadURL(url.format({
  pathname: path.join(__dirname, 'index.html'),
  protocol: 'file:',
  slashes: true
}))

Cypress.io E2E Test Runner

Platform CI Output Binary
Darwin Buildkite Electron
Linux CircleCI Electron
Windows AppVeyor Electron

Qt, Java, ... Node 👍

Cypress GUI is a web app itself running as Electron application

Shell scripts ➡ Nodejs scripts

if [ -d "foo" ] then:
  cp -r foo bar
fi
const shell = require('shelljs')
if (fs.existsSync('foo')) {
  shell.cp('-r', 'foo', 'bar')
}

Works (Linux)

Mostly works (Darwin)

🔥  (Windows)

Just works!

All our build scripts are Nodejs files

I don't want to learn your cryptic script commands

const shell = require('shelljs')
shell.set('-v') // verbose
shell.set('-e') // any error is fatal
shell.exec('npm run binary-build ...')
shell.exec('npm run binary-zip')
shell.ls('-l', '*.zip')
// more commands

CI requires only Node

make, curl, git, bash, zip, s3cmd

FROM: node

Docker

  1. Where is JavaScript?
    • Browser, server, mobile, shell scripts

  2. JavaScript versions
  3. Main tools

Modern JavaScript

Let's look at the JavaScript language

Modern JavaScript

  • 1996: Netscape asks Ecma to make a "standard" JavaScript specifications
  • 1997 Ecma publishes EcmaScript standard

Modern JavaScript

EcmaScript (ES) = standard

JavaScript = implementation

* "JavaScript" is a trademark of Oracle Co

*

Modern JavaScript

  • June 2011: EcmaScript 5.1 (ES5)

All modern browsers

Modern JavaScript

  • June 2011: EcmaScript 5.1 (ES5)
var x = 'foo'
var y = 42
function add(a, b) {
  return a + b
}
add(x, y)
// EcmaScript has no input / output ⚠️

Modern JavaScript

  • June 2011: EcmaScript 5.1 (ES5)
function double(x) {
  return x + x
}
function greatherThan4(x) {
  return x > 4
}
['1', '2', '3'].map(parseFloat)
  .map(double)
  .filter(greatherThan4)
  .forEach(console.log)

Arrays with .map .filter .reduce

Modern JavaScript

  • June 2011: EcmaScript 5.1 (ES5)
function double(x) {
  return x + x
}
function greatherThan4(x) {
  return x > 4
}
['1', '2', '3'].map(parseFloat)
  .map(double)
  .filter(greatherThan4)
  .forEach(console.log)

Pass functions as arguments

Modern JavaScript

  • June 2015: EcmaScript 2015 (ES6)

The present*

* mostly

Modern JavaScript

  • June 2015: EcmaScript 2015 (ES6)
const x = 'foo'
[1, 2, 3].map(x => x + x)
console.log(`x = ${x}`)
// and lots of other things...

Modern JavaScript

  • June 2016: EcmaScript 2016
[1, 2, 3].includes(2) // true
2 ** 3 // 8

Modern JavaScript

  • June 2017: EcmaScript 2017
async function userName () {
  const user = await fetch()
  return user.name
}
// plus a few other features

Modern JavaScript

  • June 2017: EcmaScript 2017
async function userName () {
  try {
    const user = await fetch()
    return user.name
  } catch (err) {
    // deal with error
  }
}

Modern JavaScript

  • June 2015: EcmaScript 2015 (ES6)
  • June 2016: EcmaScript 2016 (ES7)
  • June 2017: EcmaScript 2017
  • June 2018: ...

How an idea becomes an EcmaScript feature

Anyone can propose a new feature

How an idea becomes an EcmaScript feature

I want JavaScript to have X

Stage 0

an idea is born

How an idea becomes an EcmaScript feature

Discussion, demos, polyfills

Feature gets a "champion"

Stage 1: proposal

How an idea becomes an EcmaScript feature

TC39 committee: about 25 members

Mozilla, Google, Apple, Intel, universities

Stage 1: proposal

How an idea becomes an EcmaScript feature

Draft and precise spec language is being developed

Stage 2: draft

How an idea becomes an EcmaScript feature

All semantics, syntax and API are completed described

Stage 3: candidate

How an idea becomes an EcmaScript feature

Feature has tests

Two reference implementations

Stage 4: finished

How an idea becomes an EcmaScript feature

Stage 4 features as of March

become part of the standard published in June

What does a browser support?

try {
  eval('2 ** 8')
} catch () {
  // ** is not supported
}
// ** is ✅

What does a browser support?

  • Evergreen browsers are 100% ES2015/2016/2017 compatible

*

* except for (maybe) ES6 modules

What does a browser support?

  • Mobile browsers really patchy

1: Adding polyfills

Detect and polyfill only what's missing with modernizr.com

<script src="polyfills.js"></script>
<script>
[1, 2, 3].includes(2) // works now
</script>

1: Adding polyfills

Does not work with every feature

<script src="polyfills.js"></script>
<script>
2 ** 3
// SyntaxError
</script>

2: Transpiling

Rewrite code to older standard

2 ** 3 // app.js
babel app.js
Math.pow(2, 3)

ES2016

ES5

* not every feature can be transpiled

*

Talk progress

  1. Where is JavaScript?
  2. JavaScript versions
  3. Main tools

Use a Modern Browser

Evergreen browser

Chrome, FF, Safari Technical Preview, Edge

  • Local overrides demo

  • Code scripts demo

NPM Registry

aka OSS heaven

source: https://npms.io

Creating your own module

$ mkdir my-app
$ cd my-app
$ git init
$ npm init -y
$ npm publish
$ npm init -y
Wrote to my-app/package.json
{
  "name": "my-app",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}
$ yo node-bahmutov
{
  "name": "stop-build",
  "version": "0.0.0-development",
  "description": "Exits with non-zero code if there are modified Git files",
  "author": "Gleb Bahmutov <gleb.bahmutov@gmail.com>",
  "bugs": "https://github.com/bahmutov/stop-build/issues",
  "config": {
    "next-update": {
      "commands": {
        "pre-git": "./.git/hooks/pre-commit"
      }
    },
    "pre-git": {
      "commit-msg": "simple",
      "pre-commit": [
        "npm run pretest",
        "npm run deps",
        "npm run ban",
        "git add ."
      ],
      "pre-push": [
        "npm test",
        "npm run secure",
        "npm run license",
        "npm run ban -- --all",
        "npm run size"
      ],
      "post-commit": [],
      "post-merge": []
    }
  },
  "engines": {
    "node": ">=6"
  },
  "files": [
    "src/*.js",
    "!src/*-spec.js"
  ],
  "bin": {
    "stop-build": "src/index.js"
  },
  "homepage": "https://github.com/bahmutov/stop-build#readme",
  "keywords": [
    "build",
    "ci",
    "exit",
    "git",
    "utility"
  ],
  "license": "MIT",
  "main": "src/",
  "publishConfig": {
    "registry": "http://registry.npmjs.org/"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/bahmutov/stop-build.git"
  },
  "scripts": {
    "ban": "ban",
    "deps": "deps-ok",
    "issues": "git-issues",
    "license": "license-checker --production --onlyunknown --csv",
    "pretty": "prettier-standard 'src/*.js'",
    "prelint": "npm run pretty",
    "lint": "standard --verbose --fix src/*.js",
    "pretest": "npm run lint",
    "secure": "nsp check",
    "size": "t=\"$(npm pack .)\"; wc -c \"${t}\"; tar tvf \"${t}\"; rm \"${t}\";",
    "test": "npm run unit",
    "unit": "mocha src/*-spec.js",
    "semantic-release": "semantic-release pre && npm publish && semantic-release post"
  },
  "devDependencies": {
    "ban-sensitive-files": "1.9.0",
    "deps-ok": "1.2.0",
    "git-issues": "1.3.1",
    "license-checker": "11.0.0",
    "mocha": "3.4.2",
    "nsp": "2.6.3",
    "pre-git": "3.15.0",
    "prettier-standard": "5.1.0",
    "semantic-release": "^6.3.6",
    "standard": "10.0.2"
  },
  "dependencies": {
    "ggit": "1.15.1",
    "pluralize": "5.0.0"
  }
}

Yeoman

$ yo node-bahmutov
{
  "name": "stop-build",
  "version": "0.0.0-development",
  "description": "Exits with non-zero code if there are modified Git files",
  "author": "Gleb Bahmutov <gleb.bahmutov@gmail.com>",
  "bugs": "https://github.com/bahmutov/stop-build/issues",
  "config": {
    "next-update": {
      "commands": {
        "pre-git": "./.git/hooks/pre-commit"
      }
    },
    "pre-git": {
      "commit-msg": "simple",
      "pre-commit": [
        "npm run pretest",
        "npm run deps",
        "npm run ban",
        "git add ."
      ],
      "pre-push": [
        "npm test",
        "npm run secure",
        "npm run license",
        "npm run ban -- --all",
        "npm run size"
      ],
      "post-commit": [],
      "post-merge": []
    }
  },
  "engines": {
    "node": ">=6"
  },
  "files": [
    "src/*.js",
    "!src/*-spec.js"
  ],
  "bin": {
    "stop-build": "src/index.js"
  },
  "homepage": "https://github.com/bahmutov/stop-build#readme",
  "keywords": [
    "build",
    "ci",
    "exit",
    "git",
    "utility"
  ],
  "license": "MIT",
  "main": "src/",
  "publishConfig": {
    "registry": "http://registry.npmjs.org/"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/bahmutov/stop-build.git"
  },
  "scripts": {
    "ban": "ban",
    "deps": "deps-ok",
    "issues": "git-issues",
    "license": "license-checker --production --onlyunknown --csv",
    "pretty": "prettier-standard 'src/*.js'",
    "prelint": "npm run pretty",
    "lint": "standard --verbose --fix src/*.js",
    "pretest": "npm run lint",
    "secure": "nsp check",
    "size": "t=\"$(npm pack .)\"; wc -c \"${t}\"; tar tvf \"${t}\"; rm \"${t}\";",
    "test": "npm run unit",
    "unit": "mocha src/*-spec.js",
    "semantic-release": "semantic-release pre && npm publish && semantic-release post"
  },
  "devDependencies": {
    "ban-sensitive-files": "1.9.0",
    "deps-ok": "1.2.0",
    "git-issues": "1.3.1",
    "license-checker": "11.0.0",
    "mocha": "3.4.2",
    "nsp": "2.6.3",
    "pre-git": "3.15.0",
    "prettier-standard": "5.1.0",
    "semantic-release": "^6.3.6",
    "standard": "10.0.2"
  },
  "dependencies": {
    "ggit": "1.15.1",
    "pluralize": "5.0.0"
  }
}

generator-node-bahmutov

NPM registry

  • There are plenty high quality JS modules in the NPM registry
  • You can always write and publish your own NPM module

Most popular server framework: express.js

almost 1 million downloads PER DAY

Offline server-side rendering

var express = require('express')
var app = express()
 
app.get('/', function (req, res) {
  res.send('Hello World')
})
 
app.listen(3000)

The page HTML arrives ready to be rendered by the browser

The browsers are REALLY REALLY good at rendering static pages

Server-side rendering is good

If the server goes down in the forest ...

<html manifest="example.appcache">
  ...
</html>

Application Cache

CACHE MANIFEST
# v1 2011-08-14
index.html
style.css
image1.png
# Use from network if available
NETWORK:
network.html
# Fallback content
FALLBACK:
/ fallback.html

declarative list

Application Cache

Turns out declaring caching strategy is hard.

ServiceWorker

Server

browser

Web Workers

ServiceWorker

(JavaScript)

Transforms

the response

Transforms

the request

Server

browser

service

worker

No changes to the page 😊

Load ServiceWorker

navigator.serviceWorker.register(
    'app/my-service-worker.js')

Chrome, Opera, Firefox, Edge

Must be https

Inside ServiceWorker

self.addEventListener('install', ...)
self.addEventListener('activate', ...)
self.addEventListener('message', ...)
self.addEventListener('push', ...)
self.addEventListener('fetch', function (event) {
  console.log(event.request.url)
  event.respondWith(...)
})
// Cache API
self.addEventListener('install', e => {
  const urls = ['/', 'app.css', 'app.js']
  e.waitUntil(
    caches.open('my-app')
      .then(cache => 
        cache.addAll(urls))
  )
})

Cache resources on SW install

self.addEventListener('fetch', e => {
  e.respondWith(
    caches.open('my-app')
      .then(cache =>
        cache => match(e.request))
  )
})

Return cached resource

Fetch event

browser

ServiceWorker

Request

Response

Server

Server

browser

ServiceWorker

Request

Response

express.js

http.ClientRequest

JavaScript

http.ServerResponse

JavaScript

Fetch event

browser

ServiceWorker

Server

express.js

Server

browser

ServiceWorker

express.js

http.ClientRequest(Request)

http.ServerResponse(Response)

Express-service

Express-service

There is no client-side* JavaScript :)

<link rel="serviceworker" scope="/" href="/sw.js">

*after first ServiceWorker registration

Express-service

bonus

$ npm i -g nativefier
$ nativefier https://express-service.gleb-demos.com/
Packaging app for platform darwin x64 using electron v1.1.3
App built to /Users/gleb/git/ExpressService-darwin-x64

JavaScript is scripting language ⚠️

Use static linters to catch simple mistakes

Start "index.js" file

  • Sublime Text
  • Visual Studio Code
  • vim
  • WebStorm IDE

Good auto-formatting

Prettier

Prettier is an opinionated code formatter

Static source linter
Standard

One JavaScript Style to Rule Them All

$ npm home prettier
$ npm home standard
$ npm home prettier
$ npm home standard
$ npm install --save-dev \
  prettier-standard standard

+ standard@10.0.3
+ prettier-standard@6.0.0
{
  "scripts": {
    "prelint": "prettier-standard '*.js'",
    "lint": "standard --fix *.js",
    "pretest": "npm run lint",
    "test": "exit 1"
  }
}

package.json

$ npm run lint
// index.js
const add = (a, b) =&gt; a + b
const sub = (a, b) =&gt; {
  return a + b
}
module.exports.add = ad
module.exports.sub = sub
> standard --fix *.js

  my-app/index.js:1:7: 
   'add' is assigned a value but never used.
  my-app/index.js:5:22: 
   'ad' is not defined.

Bonus: NPX

Node v8.x.x includes NPM v5.x.x and NPX

A helper tool for executing binaries from packages

$ node_modules/.bin/standard *.js
       or
$ $(npm bin)/standard *.js
$ npx standard *.js

Formatting and linting

  • Enforce consistent code style using a formatting tool like "prettier"
  • Catch simple errors using a linter like "standard"

Unit testing JavaScript

Write tests first

Keep refactoring while tests are passing

Tape, QUnit, Mocha, Ava, Jest

Delightful JavaScript Testing

$ npm install --save-dev jest
$ npx jest
/* eslint-env jest */
describe('add', () => {
  const add = require('.').add
  it('adds numbers', () => {
    expect(add(1, 2)).toBe(3)
  })
})
$ npx jest
 PASS  ./test.js
  add
    ✓ adds numbers (4ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        0.763s, estimated 1s
Ran all test suites.
 PASS  ./test.js
  add
    ✓ adds numbers (7ms)
  sub
    ✓ subs numbers (1ms)

Snapshot Summary
 › 2 snapshots written in 1 test suite.
it('adds numbers', () => {
  expect(add(1, 2)).toMatchSnapshot()
  expect(add(100, -50)).toMatchSnapshot()
})
$ cat __snapshots__/test.js.snap 
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`add adds numbers 1`] = `3`;

exports[`add adds numbers 2`] = `50`;

Saved snapshot files

expect(value).toMatchSnapshot()
    
Received value does not match stored snapshot 1.
    
    - 3
    + 21
it('adds numbers', () => {
  expect(add(1, 20)).toMatchSnapshot()
  expect(add(100, -50)).toMatchSnapshot()
})

Unit testing JS

  • There are plenty of testing tools and frameworks
  • Snapshot testing is becoming mainstream

Loading JS Modules

Writing a single JavaScript file is a bad idea, but how should one JS file load another JS file?

CommonJS modules

// add.js
function add (a, b) {
  return a + b
}
module.exports = add

Node implements CommonJS standard

CommonJS modules

// index.js
const add = require('./add')
add(2, 3) // 5

Node loads a file synchronously

Two huge problems

CommonJS modules

  • Browser cannot load files synchronously

Solution: bundling

  add.js
  sub.js
  index.js
<script src="bundle.js"></script>
bundle.js

Second problem

CommonJS modules

  • JS engine cannot tell what a module exports until it runs module's code

Huge JS bundles because have to include everything, even if code is unused

ES6 Modules

// add.js
export function add (a, b) {
  return a + b
}
// index.js
import {add} from './add'
$ node add.js 
./add.js:2
export function add (a, b) {
^^^^^^
SyntaxError: Unexpected token export
  • Node 8 does not implement ES6 modules

  • Browser support is patchy

* Target: Node 10, April 2018

*

$ npm i -S @std/esm
{
  "@std/esm": {
    "esm": "js"
  },
  "scripts": {
    "start": "node -r @std/esm index.js"
  }
}

add to package.json

install dependency

ES6 Modules in Node Today

// index.js
import {add} from './add'
import {sub} from './sub'
console.log(add(2, 3))
$ npm start

> node -r @std/esm index.js

5

Building browser bundle

$ npm i -D rollup
{
  "scripts": {
    "build": "rollup index.js -o bundle.js --f iife"
  }
}

add to package.json

install dependency

$ npm run build

build the bundle

Tree-shaking

// index.js
import {add} from './add'
import {sub} from './sub'
console.log(add(2, 3))

The bundle is minimal

Thanks, explicit "import / export" declarations! 

(function () {
'use strict';
function add (a, b) {
  return a + b
}
console.log(add(2, 3));
}());
$ cat bundle.js

There is no "sub" code in the bundle!

<head>
    <!-- in case ES6 modules are supported -->
    <script src="app/index.js" type="module"></script>

    <!-- in case ES6 modules aren't supported -->
    <script src="dist/bundle.js" defer nomodule></script>
  </head>

Browser: ship both bundles

JavaScript ES6 Modules

  • "ES6 module" is the new format with "export / import" keywords 
  • A helper loader like "@std/esm" allows current Node to load ES6 modules
  • A tool like "rollup" can create minimal browser bundles from ES6 modules 

Is this it?

There is so much more

IoT, robots, databases,

WebSockets, Audio and Video, P2P, WebAssembly, VR, compiled to JS languages

JavaScript language is running the modern web

The tools and libraries are incredibly diverse and can satisfy anyone's needs

There is a good process for introducing new language features every year

Future Belongs to You JS

Gleb Bahmutov, PhD

Øredev 2017

The Future Belongs to You and Your JavaScript

By Gleb Bahmutov

The Future Belongs to You and Your JavaScript

Overview of modern JavaScript language and most commonly used tools. Browsers, servers, desktop and every place in between where JS can run. Code snippets and ServiceWorkers. Presented at Øredev 2017

  • 1,348
Loading comments...

More from Gleb Bahmutov