Making APIs fast

learning from 40 years of SQL

@joakimlundborg

github.com/cortex

Wrapp 

SQL

in 2017?

at an API Conference?

Is it Web Scale?

http://www.mongodb-is-web-scale.com/

More control to apps

What is 'fast'?

Never let the

human wait

Latency > throughput

How do we make things fast

	 	  1 ns cpu cache reference
	        100 ns main memory reference
	     10 000 ns 1kB over GB network
	  1 000 000 ns 1MB from SSD
	150 000 000 ns roundtrip to netherlands


     	  1 m  = coffe on your desk
        100 m  = coffee around the corner
         10 km = subway to Solna
      1 000 km = distance to Amsterdam
    150 000 km = half way to the moon

        	  1 s = 1s
            100 s = 1.66 minutes
         10 000 s = 2.7 hours
      1 000 000 s = 11 .57 days
    150 000 000 s = 4.75 years

YOU ARE HERE

A brief history of *SQL

  • 1970s SEQUEL
  • 1987 ANSI SQL
  • 1995 The third manifesto
  • 1995 MySQL
  • 1996 PostgresQL
  • 2003 Memcached
  • ~2005 NoSQL
  • 2005 CouchDB
  • 2008 Cassandra
  • 2009 Redis, MongoDB

Data warehousing

Star schemas

denormalization

The NoSQL movement

Key/value stores

Lesson learnt

model data after your access patterns

Lesson learnt

KISS

Lesson learnt

Consistency > minimum speed

JOINs are evil

SELECT thing FROM huge_list
JOIN all_the_related_things


const things = await fetch_huge_list();

const result = things.map(thing => {
    fetch_all_the_related_things(thing)
})

The N+1 problem

The N+1 ORM problem

The N+1 ORM bad abstraction problem

Query planning is not a backend problem anymore

EXPLAIN GET /items/1

Summary

how to make things fast

Give clients control of what they fetch

Help client developers understand what is slow

Know your access patterns

Know your data

Design your user API experience for low latency

Design for cacheability

Design for prefetching

It is not about the tools

it is how you use them

Text

Thanks!

Making APIs fast

By cortex

Making APIs fast

  • 862
Loading comments...

More from cortex