Using SOA to Scale Your App and Your Team

London Web Performance Group meetup, 2015-04-09

Laurie Voss

CTO of npm Inc


What are we talking about?

Service Oriented Architecture (SOA)

  1. What SOA is
  2. Why it's good
  3. Why SOA is getting popular (again)
  4. How npm uses SOA
  5. Best practices for using SOA

Impostor Syndrome Disclaimer

Speed up your website with this 1 
weird trick discovered by a mom!

What is              ?


npm is not one thing

  1. npm the CLI
  2. npm the public registry
  3. npm Enterprise

What is SOA?

Over-simplified example time!

This is not a new idea

Monolithic apps

They're not that bad!

MVC: the ubiquitous monolith

As seen in Ruby on Rails, etc.

"Oh, it's like Rails. Okay."

ORM is an antipattern:

Mo' lithic, mo' problems

Monolithic apps scale, but expensively

Scaling monolithic architecture

Monoliths get expensive and slow

Scaling is easy

If you have infinite money

Scaling this way is dumb

You're paying for 

the wrong stuff

You can be dumb 

for a long time

Everybody is doing it

"The app is down"


"The company is down"

Scaling monolithic teams

Big teams move slower

SOA to the rescue

A picture of the Rescue Rangers here
would be a copyright violation, but 
you can imagine them for free.

SOA is a distributed system

Features of distributed systems:
  1. Concurrency
  2. No global clock
  3. Independent failure of components

Race conditions galore!

Architectural advantages

1. Distributed systems fail more gracefully
(if you design them properly)

Better use of resources

Do more by doing less

Better resource allocation = do more

Distributed systems 

are easier to scale

If you have infinite hardware
(and you do!)

Scale the busy parts

Operational advantages

Pick the appropriate level of redundancy

Faster debugging

There's only one thing that can possibly go wrong.

Team advantages

Our monkey brains are small.
Don't tax them unnecessarily.

Smaller systems = smaller teams

Calm down, math(s) nerds

Stronger ownership


better code

Development advantages

Smaller systems = less stuff to break
Better isolation = less chance of breaking stuff

Move fast

and don't break things

These advantages are nontrivial

(If you design it right)

Cost advantages

SOAs run cheaper

But it's not all roses

Mo' pieces, mo' problems

Probability math:

  • If any machine will fail = 0.1
  • 1-box system will fail = 0.1


Same 0.1 hardware, 10-box system:
  • partially fail = 1.0
  • totally fail = 0.1^10 = 0.0000000001

Always slightly down.
Never totally down.

You need redundancy

You needed it anyway, though.

Redundancy is easier now

(If you design it right)

Deployment automation 

is essential

We use Ansible. It's nice!

Interface definition

Versioned APIs will help.

Keep your contracts

Or you will have lots of boring meetings.


You have to get all your data over the network.

Everything's an API

Defining data sources strictly means
fewer unexpected side-effects.

Requires forethought

Or a crystal ball.

Shared logic

We already re-invented this wheel!

You need shared libraries

Obvious product plug incoming

Hey, that's that thing we do!

(What a coincidence!)

npm install all-the-things

Private modules are neat

npm install @mycompany/secret-thing

You need versioned APIs

And you need to support old behaviour for longer.

Distributed systems are hard

"Backpressure" is a fancy word for
"this thing is too slow".

Race conditions

will happen out of order.
Anything that can happen out of order,

Plan for failure

Why SOA now?

We like changing things for no reason.

Technical change 1:


The phrase "the cloud"
can always be replaced with
"some computers in eastern Virginia"

Real hardware 

involves carrying 

actual heavy things

Real hardware is awful

Virtualization is awesome

Virtual hardware


disposable hardware

You won't always get to pick when it's disposed of.

Deploy to new hardware

Why roll back software when you can roll back hardware?

Technical change 2: 

distribution by default

We had all this HTTP lying around...

The Internet is one big SOA

Big change 3: cost

Smaller, cheaper, faster.

Hardware is cheap

Developers are expensive

Enough Why. Time for How.

npm registry architecture

npm registry architecture 2

npm registry architecture 3

Reality is harder

Not shown:
  • Multiple datacenters
  • Redundant hardware
  • Balancer proxies
  • Nginx for SSL termination
  • Legacy shims
  • Backup systems
  • External replication

Metadata store


That's the registry

npm Enterprise

Scaling down

License API is external

Policy follower

What npm Enterprise is for

Control of binaries

What npm Enterprise is for

Control of access

What npm Enterprise is for

Legal requirements

Enter the Policy Follower

Filters incoming replication

Policies are code

npm install @mycompany/our-policy

Licensing filters

This is SOA being great

SOA scaling saves us dev time

Fix bugs once only.

But wait, there's more!

www is lightweight

Registry API

A relational registry

Me loving SQL:

Putting it all together

Lesson 1: deployment automation

Lots of boxes = lots of failures

Take crappy jobs away from humans

SOAs are more complicated

Many boxes = many configs

Service management

Cross-OS services are hard

Enter ndm


more info:

How we do 

deployment automation

Ansible playbooks

Services by ndm

One-click deploys

Deploy by git push

Deployment automation

 gives you time back

Configuration management

Too much to remember

The configurator

Pulls everything from etcd

Standard config is better

Service monitoring

Don't fail silently

Monitor everything

Service-specific checks

Using AWS simply

Pretty ranty old blog post:

We use AWS for everything

We don't use 

everything from AWS

Reason 1: because we can

Not everybody can

Reason 2: control

Control is a luxury

We are willing to pay for it.

Reason 3: community

Reason 4: portability

Not ready to commit

It's not you, AWS. It's us.


Why metrics are important

Metrics are a debugging tool

"Dashboard" is not a dirty word

Dashboards saved us $60,000

Scaling Node.js services

SSL termination sucks

Use balancers for multiple cores

Beware maxSockets

What about io.js?

In Summary...

1. Monolithic apps are bad

Sort of. Sometimes.

2. SOA is good

Sort of. Sometimes. Mostly.

3. SOA is easier than ever

Thanks, Amazon!

4. npm is SOA all the way down

5. Our best practices

  • deployment automation
  • service management
  • configuration management
  • service monitoring
  • metrics

6. Bonus: node.js best practices

  • use balancers to terminate SSL
  • use balancers to maximize cores
  • set maxSockets to Infinity
  • probably don't use CouchDB for anything



SOA for LWPG meetup

By seldo

SOA for LWPG meetup

  • 3,865
Loading comments...

More from seldo