ABS

Always Be Streaming

Johnny Ray Austin

Creative Technologist - iStrategyLabs

@recursivefunk

What is a stream?

Streams

  • Source
  • Destination
  • Chunks

Old & Busted

User requests data, your application connects to a data source then waits for all the data to come back, your application returns the data to the user.

New (old?) Hotness

User requests data, your application connects to a data source, as soon as some data is available, that chunk of data is sent back to the user immediately. Repeat until there is no more data.

Why?

  • Data flow management

    • ​Pipes
  • ​Variable data volume
    • ​Check volume periodically
  • ​Memory buffer protection

(spoiler alert: users are impatient)

Perception is Key

Actual performance increase

VS 

Perceived performance increase

Actual Performance

  • Database Indexes
  • Data Caches
    • <3 Redis
  • Web Sockets
    • Keep it real (time)
      • May go wrong

Perceived Performance

  • User centric
    • Time to respond
  • Progressive rendering*

What about the server?

Of Course

  • Boiler plate
    • User auth/privileges
  • Very fast database hits
    • Think twice when database joins are involved

How can I do this?

  • Database
    • PostgresSql
  • Server
    • Node.js
    • Python
  • Cache
    • Redis
  • Messaging
    • ZeroMQ
  • Browser
    • highland.js

Questions?