Streaming in JavaScript
Definition
A stream is an abstract interface implemented by various objects in Node.js.
All streams are instances of EventEmitter.
Approach of programming.
Types
- writable
- readable
- duplex
Streams in Node.js
Examples of Streams
- process.stdin (readable)
- process.stdout (writeable)
- HTTP request on the server (readable)
- HTTP response on the server (writable)
- fs (readable, writable)
- TCP sockets (duplex)
My interview task
- We have some customer records in a text file one customer per line, JSON-encoded.
- We want to invite any customer within 100km of our Dublin office (GPS coordinates 53.3381985, -6.2592576) to some food and drinks on us.
- Write a program that will read the full list of customers
- Sort by user id (ascending)
- Output the names and user ids of matching customers (within 100km)
Solution draft
- read file -> string
- split LF -> array[string]
- parse -> array[object]
- validate -> array[object]
- calculate distance and filter items -> array[object]
- sort -> array[object]
- pick name and id -> array[string]
- join LF -> string
Simple approach
- Read entire data to string
- Split string to get an array
- Loop each item, validate, filter, etc. and push it into new array
- Sort array
- Loop each item in order to display it
BUT
Better to treat this as a stream of data
() -> split -> parse -> validate -> filter -> toString -> join -> ()
Functional approach
Streaming approach
Playing with streams
- add throttle for testing
- @demo
- parallelize compute intensive task
- @filtering
Benefits of Streams
- live processing - no need to wait for entire input data - it is processed as soon as it can
- potential for performance boost when paralleling tasks
- infinite amount of input data - no memory is needed to store input data
Things to check out
Thanx! Questions?
Streaming in JavaScript
By Miłosz Chmura
Streaming in JavaScript
- 1,118