Node.js Interactive

Node.js YouTube Channel

watch here for the talks to be released

Wed Morning Keynotes

No notes 😬

• Welcome and Node.js Update- Mark Hinkle, Node.js Foundation
• You Know What They Say About Good Intentions…- Kim Crayton, Community Engineer and Advocate
• Developer Efficiency and Production Success for Node.js Applications: Create, Deploy, Manage - Michael Dawson, IBM
• The Node Movement and How to Play a Part- Cian O'Maidin, nearForm
• The V8 Engine and Node.js- Franzi Hinkelmann, Google
• JS Foundation Panel: The Many Facets of Sustaining an Open Source Ecosystem- Jory Burson, Bocoup (Moderator); Maggie Pint, Microsoft; Tracy Hinds, Node.js; Erin McKean, IBM

Franziska Hinklemann

V8 ❤️ Node

 

Node is a first-class citizen in V8

No V8 commit can land if it breaks Node

A complete IoT workshop

Jeremy Foster

Devices

All are basically microcontrollers with the exception of the system on a chips

• Arduino
• RPi 3 - system on a chip -NTG
• Beaglebone black - system on a chip -NTG
• mxchip - from MicroSoft
• RPi Zero W - small, cheap -NTG
• Particle Photon
• ESP 8622 -NTG
• ESP 32
• Intel Edison - no more support :sadface: -NTG

Node Threshold of Glory

Can I run node on it?

There is SO MUCH available in NPM

Three types of architecture:

Edge Architecture

• "Things without the internet"
• Ex: things in the room are displayed on a screen in the room
  • mult things, gateway, ui on gateway, visualization

Cloud Architecture

• things connected directly to cloud
• UI talks to the cloud
• can access from anywhere, don't have to be in the room
• if internet is down for the things, no way to see stuff

Hybrid architecture

• Things connected to gateway, gateway connected to cloud, UI connected to Cloud AND Gateway, can switch between when cloud / thing's internet is down

Boat Example

• bunch of sensors connect to server
• when boat has wifi to cloud, talk to cloud, otherwise, talk internally
• blablabla something about MS Azure is great blablabla (presenter works at MS)

github

codefoster/iot-workshop

We connected the thing, and did the stuff...

Click the button, a photo is taken, sent to Cloudland somewhere and analyzed, and the JSON analysis of that photo is logged...

Maybe get this thing if you want to know how energy "hungry" your setup is:

https://www.adafruit.com/product/2690

Node.js Does 30 Billion Transactions Per Day [B] - Joran Siu, IBM

30 billion transactions per day!

• Node.js
• Linux
• Mongo.DB
• Acme air - app to mimic real world
  • several variants db
  • micro / monolith

IBM Linux One

• mission critical data
• scalability
• enterprise computing
• single mongo instance = 17?? transactions
• 2mill docker containers in a single box
• ZED = Zero down time? 30sec down time in year
• security - crypto for private keys, tamper resistant - wipe on tamper

Earthquake test

• Japan's earthquake, mainframes went through ok

Node.js runtime is relatively small

• monitor and understand hotpots in your app

This is turning into a LinuxONE advertisement 🤔

N-API - Next Generation Node API for Native Modules [I] - Michael Dawson, IBM & Arunesh Chandra, Microsoft

N-API

N-API is a stable node api layer for native modules that provides api compatibility, guarantees across different node versions and flavors

Experimental in Node 8

Why

• reduces friction from upgrading
• save cost for maintenance on a native module

Is it the new NaN?

• isolation from V8
• compile once and run mult versions
• is it the new NaN? (what is NaN?)

N-API demo:

• leveldb and leveldown?
• how does the thing change when the node binary is changed underneath it
• it.... doesn't?

Wait, what is levelDB?

• http://leveldb.org/

API shape

• it is in the docs
• https://nodejs.org/dist/latest-v8.x/docs/api/n-api.html

it's written in C

Honestly, it sucks to present right after lunch. 😴

Backward compatibility

• avail in 8.x
• plan to backport to 6.x
• 4.x - with caveats

Halp them

https://github.com/nodejs/abi-stable-node

They meet on Thursdays

ABI - Application Binary Interface

TLDR - Node uses JS and C++, some of that C++ is async 🙀

Node core ~40k req/s, Express ~21k req/s

between those two example servers. Where did the 19k go?

Compare features between Express, Hapi, Fastify

How is fast?

• preinitialized with fast data structures
• you were too fast!!!

the problem with closures

• function with big data - that data lives a long time
• callback hell
• big data is alive until last of the nested closures
• no garbage collection until last closure is done
• function are short lived and hard to optimize by runtime
• avoid nested closures

OMG srsly he is going too fast.

Check out helmet right away and put it in your express application.

huh, ok.

Most code does not need to go at ludicrous speed

High Performance JS in V8 [I] - Peter Marshall, Google

Runtime - runs and compiles your code

why do we care about speed?

• servers cost money
• users like speed (and will go away if things don't load fast)
• changes the way that languages are used

Why optimizations?

• Language is growing
• mapping high level language to assembly requires tradoffs
• performance on low-end mobile is definitely not solved - 20-30 sec to load a web page 😭
• * People change the way they use JavaScript*

there are many ways to do things in JS

leads to dialects, idioms, styles

"CrankshaftScript"

• a JS dialect whose only purpose is to run fast on V8 Crankshaft
• it's gone now
• optimized for what *used to be* in V8
• jscode => fullcodegen => optimization in Crankshaft
• We totally deleted it
• could not support new lang features
• lots of unexpected performance cliffs.... etc

TurboFan goals

fewer deoptimizations

 

Deoptimizations are when code is "optimized" and then reverted

Crankshaft Performance Advice:

You might recognize some of this advice, which is NO LONGER TRUE:

 

Don't use let/const, try/catch. try/finally, for... in

Code example optimized for Crankshaft

• brittle binding between code and engine
• easily outdated
• harder to read, maintain

fast JS for TurboFan

• Should look like regular readable JS
• use regular es6 features
• any engine

Microbenchmarks

• Great at measuring one thing
• hard to figure out what that one thing is

Benchmarks

• peak performance gains are impressive but don't tell the whole story
• Think about predictability of performance, too
• get some real-world benchmarks

Takeaways

• Don't worry about writing for Crankshaft anymore
• performance somethingsalknnlkdssnlk
• crankshaft and turbofan can have sim perfomance in super simple same same same tests, but turbofan is much faster once things get more complicated
• crankshafts' speculative optimization was terrible, resulted in deoptimizations often

Thursday!

The Node.js Performance Workshop (I) - Matteo Collina & David Mark Clements, nearForm

David, Matteo, Luca

• build things that are fast for/in Node
• work at nearForm
• consulting for companies that need optimization

Why optimize

• 40% of people abandon websited that take more than 3 seconds to load
• a 1 second delay can result in a 7% reduction in conversations
• poor performance is est to cost US e-commerce industry $3bill/year

Benchmarking

• establish a baseline
  • like how we found out try/catch is ok now
  • don't use myths/legends to know what is slow, test it and measure it
• stress testing - apache bench testing (ab)
• soak testing (wrk autocannon)
  • wrk - used in node core, written in C, want to bring the process to a halt and see how it does
  • autocannon - 
• microbenchmarking - find piece of process where the bottleneck is, write a microbenchmark for that one process
• optimizing needs to be in the VERY SAME environ, or things get messy

Performance workflow

• Establish a baseline
• Find the bottleneck
• Optimize

repeat as necessary

GH nearform/slow-rest-api

0x is better than the devtools flamegraph

devtools is JS only, 0x shows also the c code

Flame graph

width is time, color is hot functions

tells the file and line number

be sure to close all other processes during testing

polymorphism can cause performance issues!!!

remember, v8 analyses your code and optimizes functions, including optimizations based on the shape of your objects

a polymorphic function should not be a hot function

It's not about typescript

It's about the shape of the object. Object needs to stay the same shape. Order of properties added to object is important.

Tidy code is not necessarily faster, but tidy up.

On flame graph, try clicking tiers to check where these things come from

Ludicrous speed - remove your framework 🙀

Check sans-restify???

https://medium.com/the-node-js-collection/get-ready-a-new-v8-is-coming-node-js-performance-is-changing-46a63d6da4de

The State of Node.js Security [I] - Tim Kadlec, Snyk

Secure by default

• tension between accessibility and security
• teddy bear hack
• cloud pets - database in prod, accessible by anyone anywhere
• who is to blame?
• mongo db made the decision to ship something insecure by default
• databases deleted and replaced with a ransom note

Node

• 156,000 package auth
• 9m different users
• used in enterprise, iot, nasa space suits
• Many people are touched by the things we build
• the npm blog now has 2factor auth: https://bit.ly/npm-2fa

2017

• published 526 new vulnerabilities so far
• many of them are not new
• 66% high severity
• 32% med
• 2% low

types of vulnerabilities

• directory traversal
• servestatic method needed an update

bit.ly/next-fix

a great response to a security incident. A+ response

Resources over insecure protocol

• http is easy to sniff, manipulate
• use https everywhere
• no reason to pull resources over http

Cross-site scripting (xss)

• it's all over the place
• script injected into a page
• reflected attack or stored in a database
• very common
• bit.ly/angular-xss
• pass malformed html, browser tries to figure it out, creates clickable link that creates alert
• difficult to anticipate all the attack vectors, blacklist is not "enough"
• whitelist is better
• Content Security Policy (CSP)
• keep the user safe, don't trust the users!

Malicious packages

• bit.ly/malicious-packages
• typo squatting
• cross-env = the right one
• crossenv = brings cross-env down, credit original auth, package looks p legit
  • look at the postinstall - malicious
  • devs won't realize envs are leaking
• was up from jul 19th - aug 1st, but not a lot of people downloaded
• preys on human error - tough fix
• bit.ly/npm-crossenv

Regex DOS aka ReDoS

• Regular Expression Denial of Service
• var regex = /A(B|C+)+D/;
• test "ACCCCCCCCCCCCCCCCCCCCCCCCD" vs "ACCCCCCCCCCCCCCCCX", start adding Cs in two and time to compute will just get super sad
• regex engines are naively optimistic
• those two pluses are "competing"
• bit.ly/save-regex can detect possibly catastropic, exponential-time regular expressions

ReDoS vulnerabilities

• windows of exposure
• starts when discovered
• increasingly well know, more risk
• patched
• users begin patching and vulnerabilities begins to be reduced

80% OS maintainers have no public facing disclosure policy

bit.ly/security-text

a standard that allows websites to ....

in OS release notes:

more deprecation of vulnerable versions

bump dependency versions

get snyk, see vulnerabilities

142 different contributers

bit.ly/malicious-npm

response was incredible to malicious packages in npm

community is great about responding to security issues

JSF Architect: Lambda on Easy Mode- Brian LeRoux, Begin.com

Cloud functions

we just started doing this, we don't know what we are doing, but we are getting better at it.

Deploy quickly, effectively free

10m executions a month, 90cents?

high degree of isolation

Use case: we were building a slack bot, thought this would be the way to go

as lambda functions get bigger, things get more complicated

started to separate lambda functions out by route, had to maintain each one individually manually, checklists, hard to reproduce environments, not awesome

AWS landing page has a lot of stuff

cognitive load... it's not pretty

Architecture as text

.arc file

comments start with #

sections start with @

everything else is adhjhfadshfdjlak

uses npm scripts 🎉

whitespace is only important for tables

runs node 6x because aws makes you

deploy takes like 2 seconds

Rolling deploy on aws with lamda functions is fast and makes a huge diff

seconds versus manual => hours

.arc yay

arc.codes

Modules Modules Modules [I] - Myles Borins, Google

Brad Farias did a lot to help with modules

Modules

node --experimental-use-modules or something

can use import statments?

about our current implementations

files have .mjs extension

support bare imports

can import common js

cannot require esm

dynamic import coming soon

es4 introduced the concept of packages

similar to C++ namespaces

was intentded to be like jars

ripped out of standards to never be seen again

requirejs

amd

umd - exports depending on env

esmodules landed in ecma262 in 2015

can use them in chrome, edge

loader was not specified

loader is a generic term for a workflow: fetch, transform, evaluate

CommonJS has synchronous load and inline execution

Modes of grammar

• strict mode
  • no html comments
  • await is reserved
  • divergence may increase over time
• sloppy mode
• something else
• node.js

Potential future goals

• Binary AST
• WASM
• WebPackage
• HTML Modules

Loading esm from cjs

• import('esm')
• cannot require('esm')

loading cjs from esm

• import fs from "fs"
• no named imports
• named exports for core possible
• userland modeuls only provide "default"
• can be fixed with restrictions
• maybe we just don't do this?

esm does not have a way to lexically scope variables

lexically scoped variables: __filename, export......

What does the future look like?

esm is one of many goals - not binary

node, esm, solve inconsistancies - not only problem

we need to solve this problem in a way

Hooks are coming to web and node.js

• resolve hook?

• evaluation hook?

• no transform hooks.

• hooks are going to deoptimize

.mjs is in the IETF standard track

will land in official mime registry

mime types determined by file extension

removes ambiguity

.js will always be ambigous

Grokking Asynchronous Work in Node.js [I] - Thorsten Lorenz, NodeSource

typically prod node.js runs hundreds of processes?

Async hooks API

lots of changes to core

~5300 insertions, half were tests, but stuff all over

async resources tracked in C++

inherites from AsyncWrap

 

tons of providers of async resources, inheritance chain

lifetime events of async resources

• init
• before
• after
• destroy

Code and Learn

http://nodetodo.org/getting-started/

https://github.com/nodejs/node/blob/master/BUILDING.md

stop after making the tests, no need to continue from there

 

It's cool, I updated the docs 💅🏽💻🤘🏽

When you `make test`

any console.logs will be muted

To run a single test, and to see the console.logs: 

$ tools/test.py path/to/test

check out the contributing guide to know how to do things

https://github.com/nodejs/node/blob/master/CONTRIBUTING.md

http://nodetodo.org/next-steps/

Find things that you can do to contribute!

TLDR

V8 Performance article

https://medium.com/the-node-js-collection/get-ready-a-new-v8-is-coming-node-js-performance-is-changing-46a63d6da4deBullet Two

Benchmark your app and find bottlenecks with 0x

(to a point)

https://github.com/nearform/slow-rest-api

Check out the Choo framework

https://github.com/choojs/choo

Check out the Hapi framework

hapijs.com

Check out Helmet

helmetjs.github.io

HarperDB is in Denver and supposedly small enough to run on a RPi

"Unified Data Model" supports SQL and 😒 noSQL

harperdb.io

I don't know. PostgreSQL is my jam. but maybe.

Check out .arc files for easy deploys to AWS Lambda

arc.codes

Get Node installed locally

And contribute when you can!

github.com/nodejs/node

Collaboration Summit

Benchmarking Working Group

https://benchmarking.nodejs.org/

angle:

Marketing

(node is this fast!)

or

making benchmarks better?

there is a set of benchmarks that are of interest to vm, node core, general users

Do we just aggregate the benchmarks and analyze it more generally?

https://github.com/nodejs/benchmarking

if you try to run all the benchmarks on your machine it could take overnight to a week 🙀

security, diagnostics, benchmarking, commcom, testing and standards, vm

Commcom

higher ed, school ed, google summer of code, help with commcom onboarding, updating readme, helping members onboard to the org, node together, becoming commcom member, consolidating all the ways we communicate

Benchmarking

cleanup, run more often. intel, v8, microsoft, ibm? put benchmarkign stuff in node core, simplify benchmarks, look at how other langs benchmark, nodes will be on github

Diagnostics

lots of pain in tooling, async contect, how callbacks get invoked, formalize what async contect means, standardized....? open an issue about formal semantics, use cases, how to realize, tracewriter api on windows has issues, unhandled promise rejections

Security

who are the poeple who get notified about security vuln?

what is the mem criteria?

how many people should be on the team?

what information is private?

will they ever change?

what are they supposed to do?

transparency

security policy - package issue - talk to npm or node?

can we release stuff to a group to do... stuff or something...

VM

vm diversity, bringing more vms to the table, vm neutrality, advance features, competition is great....

Chakra and V8 now, maybe future more VMs

How do we get there? ABI

in experimental, native modules in api???

testing and standards

standardize interface node exposes, how we can better test certain lts interfaces, remain stable in releases, how better run test in browsers in node.js, text endcoder and text decoder, integrate web platform tests, how to support different ecmascript features, something green, test262 for ecmascript into node.js

@emilyplatzer

github/craftninja

emilyplatzer.com