Latence applicative...

@ctranxuan

@StreamdataIO

£at€nc€... i$ Møn€¥ £øss

Amazon found every 100 milliseconds of apps latency cost them 1% in sales.

Source: http://perspectives.mvdirona.com/2009/10/31/TheCostOfLatency.aspx

Google found an extra 0.5 seconds in search page generation time dropped traffic by 20%

Source: http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it

-100ms = -1% revenue & -5% page views

Source: http://www.webperformancetoday.com/2011/11/08/velocity-faster-mobile-sites-business-kpis-case-studies/

Source: http://www.ebaytechblog.com/2013/03/29/measuring-real-user-experience-with-site-speed-gauge/

Source: https://blog.radware.com/applicationdelivery/applicationaccelerationoptimization/2013/06/web-performance-poverty-line/

0,5 s

Latency Graal

Round Trip Delays (ping time)

In ADSL, first useful piece of data in 60ms (HTTPS) In 4G / LTE, first useful piece of data in 300 ms (450 ms in 3G)

Round Trip Delays (ping time)

Constraints

  • HTTP 1.1

  • SSL / TLS

  • Increasing global traffic

  • Mobile First (3G / 4G)

Google /Netflix/ Twitter

Google

SPDY ➡ HTTP/2

HTTP

1996

1999

HTTP/1.0

HTTP/1.1

  • static content

  • small size objects

  • small # of objects

  • dynamic content

  • larger size objects

  • more objects

201x

Matthew Yohe [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons

  • more objects

  • even more larger objects

HTTP

Web page:

  • ~ 2 Mb

  • ~ 80 elements

  • ~ 38 TCP connections

source: http://httparchive.org

HTTP/1.x issues

Web page:

  • ~ 2 Mb

  • ~ 80 elements

Head of line blocking

HTTP/1.x issues

1996

1999

HTTP/1.0

HTTP/1.1

201x

Hacks

HTTP/1.x: sharding

.css

.png, .jpg

.html, .js

HTTP/1.x: spriting

By Kriplozoik CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)], via Wikimedia Commons

#home {
    left: 0px;
    width: 46px;
    background: url('img_navsprites.gif') 0 0;
}

#prev {
    left: 63px;
    width: 43px;
    background: url('img_navsprites.gif') -47px 0;
}

#next {
    left: 129px;
    width: 43px;
    background: url('img_navsprites.gif') -91px 0;
}

HTTP/1.x: inlining

<img src="data:image/png;base64,iVBORw0KGgoAAA
         ...E2AAA7JN+/OU8bK5CYII=" 
     alt="html_inline_alpesjug.png" 
     title="html_inline_alpesjug.png">

HTTP/1.x: concatenation

1 change => invalidate all the cache

<html>
<head>
<title>AlpesJUG</title>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta content="width=device-width,initial-scale=1" name="viewport">
<link rel="stylesheet" href="assets/styles/main-f11f93b619.css">
<link rel="stylesheet" href="assets/styles/theme-564f6756da.css">
<link rel="stylesheet" href="assets/styles/ie-d41d8cd98f.css">
</head>
<body>...</body>
<script src="scripts/vendor-a6d64f3f39.js"></script>
<script src="scripts/app-0d43bb3b4b.js"></script>
<script src="scripts/theme-439fdb6f87.js">
</html>

HTTP

1996

1999

2009

2015

HTTP/1.0

HTTP/1.1

SPDY

HTTP/2

  • static content

  • small size objects

  • small # of objects

  • dynamic content

  • larger size objects

  • more objects

  • more objects

  • even more larger objects

  • mobile devices

Matthew Yohe [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons

RFC 7540

HTTP/2

Retro-compatibility with HTTP/1.x

Goal: optimization of the transport

HTTP/2: multiplexing

Multiplexing of requests

  • concurrent requests

  • 1 TCP connection

  • streams: bidirectional frames

source: http://www.splicemarketing.co.uk/_blog/Splice_Blog/post/the-arrival-of-http2/#.VlrbuiCrSV4

HTTP/2: headers compression

Method: GET
Scheme: HTTPS
Accept: image/webp,image/*,*/*;q=0.8
Accept-Language: en-US,en
Path: /resource
Host: http://www.alpesjug.org/
User-Agent: Mozilla/5.0




Path: /images

+ HPACK compression

2nd, 3rd, etc. req

HTTP/2: Server push

HTTP/2: TLS

  • optional TLS (but for all browsers, it's mandatory)

  • TLS 1.2 min

  • no compression

  • no renegociation

  • stronger crypto

HTTP/2: Benefits

<img src="data:image/png;base64,iVBORw0KGgoAAA
         ...E2AAA7JN+/OU8bK5CYII=" 
     alt="html_inline_alpesjug.png" 
     title="html_inline_alpesjug.png">
<html>
<head>
<title>AlpesJUG</title>
<link rel="stylesheet" href="assets/styles/main-f11f93b619.css">
<link rel="stylesheet" href="assets/styles/theme-564f6756da.css">
<link rel="stylesheet" href="assets/styles/ie-d41d8cd98f.css">
</head>
<body>...</body>
<script src="scripts/vendor-a6d64f3f39.js"></script>
<script src="scripts/app-0d43bb3b4b.js"></script>
<script src="scripts/theme-439fdb6f87.js">
</html>

HTTP/2

HTTP/2: browsers support

HTTP/2: server implementations

source: https://github.com/http2/http2-spec/wiki/Implementations

...

HTTP/2: next bottleneck?

1001010101011                           01100011010

000010101011010                   101010101011

0110100101110010 01010111111110001010

 01011101010111                        10101010101

  00000101010110                          101010110101

 

TCP

Google

Google - QUIC

source: https://peering.google.com/about/quicfaq.html

Netflix

source: http://www.programmableweb.com/news/why-rest-keeps-me-night/2012/05/15

2+ billions requests / day

Netflix

source: http://www.programmableweb.com/news/why-rest-keeps-me-night/2012/05/15

800+ different type of devices

Netflix

source: http://www.programmableweb.com/news/why-rest-keeps-me-night/2012/05/15

OSFA API

(One Size Fits All API)

Netflix: problematic

source: http://techblog.netflix.com/2013/01/optimizing-netflix-api.html

Netflix

OSFA API

optimized for none of the devices

chatty API (lots of calls)

Netflix

source: http://techblog.netflix.com/2013/01/optimizing-netflix-api.html

Netflix

source: http://techblog.netflix.com/2013/01/optimizing-netflix-api.html

Netflix

Netflix

single items multiple items
synchronous Object Iterable
asynchronous Future Observable

Netflix

Marble diagram

Netflix

source: Ben Christensen, Functional reactive programming in the Netflix API

Netflix

source: Ben Christensen, Functional reactive programming in the Netflix API

def Observable<Map> getVideos(userId) {
    return VideoService.getVideos(userId)
         // we only want the first 10 of each list
        .take(10)
        .flatMap({ Video video ->
            def m = video.getMetaData();
            def b = video.getBookmark();
            def r = video.getRating();
            // compose these together
            return Observable.zip(m, b, r {
                metadata, bookmark, rating -> 
                    // no transform to complete dictionary
                    // of data we want for each Video
                    return [id: video.videoId]
                        << metadata << bookmark << rating
            })
        })
}

The "zip" operator combines the 3 asynchronous Observables into 1

Netflix: hystrix

Netflix: problematic

source: http://techblog.netflix.com/2013/01/optimizing-netflix-api.html

Netflix: problematic

Rest API

http://netflix.com/genreList/1249

"Rest-ish" API

http://netflix.com/genreList/1249?titleprops=name,ratings

"RPC-ish" API

http://netflix.com/genreList/1249?rowOffset=0&rowSize=5&titleprops=name,boxshot
http://netflix.com/genreList/1249/setRating?titleId=5&view=movieDetailPage

Netflix: problematic

« Netflix's domain is a graph problem not a tree problem »

Netflix

Netflix

⇒ JSON extension: JSONGraph (Virtual JSON)

⇒ Middleware

⇒ Cache

Netflix

/model.json

Netflix

https://...

/model.json

Cache

Model

HTTP

D

a

t

a

S

o

u

r

c

e

 

Router DataSource 

Client

View

Middleware

Recommandation Service

Title

Service

Rating

Service

Express

Netflix

DEMO

Netflix

« The data is the API: 

if you know the data, you know the API »

Netflix: UI refactoring

+

source: http://techblog.netflix.com/2015/08/making-netflixcom-faster.html

Twitter

What's wrong with polling?

Bob

Caroll

Alice

Erik

Pull too fast: waste resources
Pull at the wrong moment: get blocked
Pull too slow: run behind

source: Erik Meijer https://www.youtube.com/watch?v=FvMuPtuvP5w

Real-time UX... i$ Rea£-time Mon€¥

Amazon found every 100 milliseconds of apps latency cost them 1% in sales.

Source: http://perspectives.mvdirona.com/2009/10/31/TheCostOfLatency.aspx

Google found an extra 0.5 seconds in search page generation time dropped traffic by 20%

Source: http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it

Which ingredients for

     real-time UX?

Solutions for dynamic data

  • Long polling

  • WebSockets

  • Server-Sent Events

Long-Polling

HTTP Hack ⇒ Don't use it anymore

WebSockets

Server-Sent Events

  • Push techno
  • W3C
  • 2008
  • Push techno
  • W3C
  • 2006

WebSockets

Server-Sent Events

WebSockets

Server-Sent Events

  • TCP
  • low-level protocol
  • HTTP upgrade handshake
GET /chat HTTP/1.1
Host: example.com:8000
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
GET /stream HTTP/1.1 1
Host: example.com
Accept: text/event-stream

WebSockets

Server-Sent Events

proxies and load balancers reconfiguration

No need to reconfigure! 
(it's HTTP)

WebSockets

Server-Sent Events

WebSockets

Server-Sent Events

binary + text

native browsers support 

text

... but polyfills!

https://goo.gl/cCMIGf

WebSockets

Server-Sent Events

var websocket =
  new WebSocket('ws://websocketserver/echo');
websocket.onopen = function () {
	...
};

websocket.onmessage = function (e) {
	...
};

websocket.onerror = function (error) {
	...
};
var eventSource =
  new EventSource('http://sseserver/echo');
eventSource.onopen = function () {
	...
};

eventSource.onmessage = function (e) {
	...
};

eventSource.onerror = function (error) {
	...
};
eventSource.addEventListener('foo', function(e) {
  ...
}, false);

"Perfs"

Use case: Préchargement de 500 Tweets sur une page web (nginx configuré en tant que proxy)

Source http://matthiasnehlsen.com/blog/2013/05/01/server-sent-events-vs-websockets/

Solutions for Dynamic Data

Streaming vs HTTP Polling

10 messages of 1 byte : 7ms vs 220 ms


100 messages of 10 bytes : 57 vs 986 ms


1000 messages of 100 bytes : 200 vs 10210 ms


5000 messages of 1000 bytes : 1,2 sec vs 54 sec

Source http://blog.arungupta.me/rest-vs-websocket-comparison-benchmarks/​

Support (bis)

Some Java Servers

Clients libraries in...

WebSockets

Server-Sent Events

  • chat, chat, chat !

  • ​Share living editing

  • GPS GoogleMap-like

  • Games

Use cases

  • Fintech / Trading

  • Betting

  • Games

  • Realtime timetables

  • Animated data apps (charts, monitoring, etc.)

What if...?
... I have just a JSON API?

Proxy as a Service: push to client

API Server Load

polling

w/ streamdata.io

10,000 concurrent clients

API Server CPU

Cache

JSON-Patch (RFC 6902)

[{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"},
 {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"},
 {"title":"Value 2","price":85,"param1":"1","param2":"22","param3":"33"},
 {"title":"Value 3","price":21,"param1":"31","param2":"12","param3":"4"},
 {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"},
 {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"},
 {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}]
[{"op":"replace","path":"/2/price","value":5},
 {"op":"replace","path":"/3/param2","value":"32"}]
[{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"},
 {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"},
 {"title":"Value 2","price":5,"param1":"1","param2":"22","param3":"33"},
 {"title":"Value 3","price":21,"param1":"31","param2":"32","param3":"4"},
 {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"},
 {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"},
 {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}]

Efficiently Turn APIs into

Real-Time Experiences

Push, don't poll

Dynamic

Cache

Incremental

Data

Animated Data

Refresh and Waiting

Poor UX: low emotional engagement

Poor SEO: poor dwell time

Which ingredients for

     real-time UX?

Friendly frameworks / libs

Android

Swift

ObjectiveC

Demos

Poll Demo

Xignite Demo

JCDecaux Demo

0.5s is the latency objective

“The more others invest in amazing UI, the more yours seems lousy”

Einstein « Relativity concepts applied to UI », OpenRoadMedia, 1937

Conclusions

The world is more and more dynamic and we've got the tools

Be ready to animate data, become streamers!

Thank you!

@StreamdataIO

https://goo.gl/cCMIGf

References

JS Polyfills

References

WebSockets / SSE

Made with Slides.com