History and Future of HTTP For Busy Developers

Dr Gleb Bahmutov PhD

You cannot adapt to or survive this

survival is possible* but we need to act now

  • change your life
  • change your bank
  • join an organization
  • vote and protest

Speaker: Gleb Bahmutov PhD

C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing

Gleb Bahmutov

Sr Director of Engineering

Science: 📚

or

☠️

12 March

1989

vague but exciting

Edit and view HTML documents, 1991

source: http://digital-archaeology.org/the-nexus-browser/

first website: info.cern.ch

1991

HTTP/0.9

1991

650 words

3800 characters

HTTP/0.9

1991

HTTP/0.9

1991

HTTP/0.9

If the port number is not specified, 80 is always assumed for HTTP

because port 79 and 81 were taken (RFC 1060)

1991

HTTP/0.9

The TCP-IP connection is broken by the server when the whole document has been transfered

1991

GET /fuzzy_bunnies.txt

HTTP/0.9

1991

The response to a simple GET request is a message in hypertext mark-up language ( HTML ). This is a byte stream of ASCII characters.

HTTP/0.9

1991

No error response codes

HTTP/0.9

1991

Only GET

No cookies / sessions / server state

HTTP/0.9

Y - you

A - aren't

G - gonna

N - need

I - it

What Problem Does HTTP/0.9 Solve?

<html>
  <p>More info
    <a href="http://foo.com/bar.html">here</a>
  </p>
</html>

I want to read this HTML document

TCP

IP

TCP

IP

how?

where?

HTTP

1991

Future HTTP protocols will be back-compatible with this protocol.

HTTP/0.9

1991 was

a few years ago ...

World Wide Web went from zero to "everywhere" in about two years

Then:

Date # of web sites
'93 130
'94 2700
'95 23,500
'96 100,000
<html>
  <p>See diagram
    <img src="http://foo.com/bar.jpg" />
  </p>
</html>

I want to see this JPEG image

Non-academic users:

HTTP/1.0

We Are Gonna Need It (WAGNI)

HTTP/1.0

1996

RFC 1945

60 pages

GET /mypage.html HTTP/1.0
User-Agent: NCSA_Mosaic/2.0 (Windows 3.1)

200 OK
Date: Tue, 15 Nov 1994 08:12:31 GMT
Server: CERN/3.0 libwww/2.17
Content-Type: text/html
<HTML> 
A page with an image
  <IMG src="/myimage.gif">
</HTML>

HTTP/1.0

  • Non-HTML data (images!)

  • Methods (HEAD & POST)

  • Status Codes

  • User preferences (User-Agent)

  • Format negotiation (Content-Type)

1996

RFC 1945

HTTP/1.1

Formalize best practices

HTTP/1.1

1999

RFC 2616

27 drafts, latest 2014

The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems.

HTTP/1.1

  • Performance (Caching, connections)

  • Security (HTTPS, 3rd party cookies)

  • Usability (IP scarcity, error codes)

1999

RFC 2616

GET /bar.html

<html>
  title: bar
</html>

*** **

bar.html

** ***

index.html

title: bar

*** **

bar.html

Give me the page ...

HTTP/1.1 is still /0.9

request

response

* I am really ignoring all server-to-server communication happening over HTTP

*

HTTP/1.1 is uniform and everywhere

Reality: multiple versions, implementations, extensions, working drafts

"The Tangled Web: A Guide to Securing Modern Web Applications"

by Michal Zalewski

ISBN-13: 978-1593273880

👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍

When I open a browser ...

State (cookies)

Cache control

Security, security, security

tracking pixels / cookies

58 requests to load

this page!

AJAX:

From websites

to web apps

i.e. "avoiding page reloads"

GET /index.html

*** **

** ***

<iframe src="bar.html">

index.html

GET /bar.html

1996: iframe in Internet Explorer

XMLHttpRequest

Gets the data from the server by a client side script

1999 Internet Explorer plugin

2004 Google Gmail, Kayak.com

2006 first spec draft

2014 latest spec draft

IP

TCP

HTTP

REST

GraphQL

2000

2012

Client - Server communications

<div> ... </div>

<script>

....

</script>

index.html

<div> ... </div>

<script>

....

</script>

index.html

HTTP/1 => HTTP/2  or QUIC

WebSockets

WebRTC

ServiceWorker

browser

browser

ServiceWorker

Blurring the line between client and server

If a tree falls while you are in the forest ...

<html manifest="example.appcache">
  ...
</html>

Application Cache

CACHE MANIFEST
# v1 2011-08-14
index.html
style.css
image1.png
# Use from network if available
NETWORK:
network.html
# Fallback content
FALLBACK:
/ fallback.html

declarative list

Application Cache

Turns out declaring caching strategy is hard.

ServiceWorker

Server

browser

Web Workers

ServiceWorker

Transforms

the response

Transforms

the request

Smart caching

OFFLINE SUPPORT

Image / video transcoding

Background data sync

What if an attacker can load malicious ServiceWorker script?

Malicious ServiceWorker injected via XSS can be really hard to get rid of

Please protect yourself from XSS

HTTP is dead

Long live HTTPS

The browser APIs worth discussing all require secure connections

http://localhost or https://...

Usually a single command!

Browser

Server

HTTPS

connections are expensive because

TCP + TLS handshakes!

data

Web Today

Ajax

ServiceWorker

HTTP/2

QUIC

HTTP/3

HTTP/2

Making web faster

*

HTTP/1.1

  • How to quickly load 200 resources?
  • How to load some resources first?
  • How to avoid duplicate data overhead?

Source: Ilya Grigorik https://bit.ly/http2-opt

2009 - Google starts SPDY

Need for Speed 🚤

2015 - RFC 7540

binary, multiplexed protocol - 55% speed up on top sites!

H2: Binary Framing Layer

H2: Multiplexing

H2: Stream Prioritization

3:1

then

then

then

3:1

$ curl -I https://github.com
HTTP/1.1 200 OK
Server: GitHub.com
Date: Sat, 03 Jun 2017 03:08:50 GMT
Content-Type: text/html; charset=utf-8
Status: 200 OK
Cache-Control: no-cache
Vary: X-PJAX
X-UA-Compatible: IE=Edge,chrome=1
Set-Cookie: logged_in=no; domain=.github.com; path=/; 
expires=Wed, 03 Jun 2037 03:08:50 -0000; secure; HttpOnly
Set-Cookie: _gh_sess=eyJzZXNzaW9uX2lkIjoiYzFlYTc0YTQ0OTNhZTdlNzI4MTgwNzI5N2QyNTlkNWMiLCJfY
3NyZl90b2tlbiI6IngzaHhRREMxNEpLUFNid1ZmMkc4d0N1OG1xdjZ3MkdmWmh4YkRFazNYQkU9In0%3D--b7171bd148da0b79b248fb561b8bfd4aadf16ff5; path=/; secure; HttpOnly
X-Request-Id: 75c9420698c53b1a707b10b2a7a510cc
X-Runtime: 0.056786
Content-Security-Policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; 
child-src render.githubusercontent.com; connect-src 'self' uploads.github.com 
status.github.com collector.githubapp.com api.github.com www.google-analytics.com 
github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com 
github-production-user-asset-6210df.s3.amazonaws.com wss://live.github.com; font-src 
assets-cdn.github.com; form-action 'self' github.com gist.github.com; frame-ancestors 
'none'; img-src 'self' data: assets-cdn.github.com identicons.github.com 
collector.githubapp.com github-cloud.s3.amazonaws.com *.githubusercontent.com; 
media-src 'none'; script-src assets-cdn.github.com; style-src 'unsafe-inline' 
assets-cdn.github.com
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
Public-Key-Pins: max-age=5184000; pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="RRM1dGqnDFsCJXBTHky16vi1obOlCgFFn/yOhI/y+ho="; pin-sha256="k2v657xBsOVe1PQRwOsHsw3bsGT2VzIqz5K+59sNQws="; pin-sha256="K87oWBWM9UZfyddvDfoxL+8lpNyoUB2ptGtn0fv6G2Q="; pin-sha256="IQBnNBEiFuhj+8x6X8XLgh01V9Ic5/V3IRQLNFFc7v4="; pin-sha256="iie1VXtL7HzAMF+/PVPR9xzT80kQxdZeJ+zduCB3uj0="; pin-sha256="LvRiGEjRqfzurezaWuj8Wie2gyHMrW5Q06LspMnox7A="; includeSubDomains
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
X-Runtime-rack: 0.060928
Vary: Accept-Encoding
X-Served-By: e878d09eac725c89f5f15204c1326660
X-GitHub-Request-Id: F1E1:2F2B:6F2906B:A48C07A:59322842

2100 characters

~55% of HTTP/0.9 spec!

H2: Header Compression

pseudo headers

H2: Header Compression

Common headers static table on client and server

HTTP/2 is changing the best performance practices

H2 changes some performance best practices

Source: Ilya Grigorik https://bit.ly/http2-opt

DO LESS WORK

H2 changes some performance best practices

Source: Ilya Grigorik https://bit.ly/http2-opt

MEASURE FIRST!

Web Today

Ajax

ServiceWorker

HTTP/2

QUIC

HTTP/3

HTTP/2 Achilles' heel

H2: If a packet is lost

TCP: all have to wait!

IP

TCP

HTTP/1.1

WebSockets

HTTP/2

TLS

Order of packets guarantee

TCP

HTTP/1.1

WebSockets

HTTP/2

TLS

UDP

IP

QUIC

QUIC

like SPDY but over UDP

'99

2009

2015

HTTP/1.1

SPDY

HTTP/2

QUIC

HTTP/0.9

'91

HTTP/1 - SPDY - HTTP/2 - QUIC timeline

2013

(deprecated)

Nov 2018

HTTP over QUIC becomes HTTP/3

HTTP: Conclusions

Every version of HTTP

and related protocols was a solution to a real-world problem

HTTP: Conclusions

Most of the time the problem was not apparent until real world adoption

HTTP/0.9 - loading remote HTML documents

HTTP/1.x - popularity and growth of World Wide Web

Ajax - loading data without full page reload

WebSockets - realtime communication with the server

ServiceWorker - scriptable caching

HTTP/2 - performance for loading modern websites

QUIC & HTTP/3 - an alternative take on performance for loading modern websites

Only failed products stay unchanged

Tomorrow a new protocol will appear to solve problems that HTTP/2 and HTTP/3 will exhibit

History and Future of HTTP For Busy Developers

Gleb Bahmutov @bahmutov

History and Future of HTTP For Busy Developers

By Gleb Bahmutov

History and Future of HTTP For Busy Developers

I will walk through the history of HTTP, its original design goals, and how they shape the modern Internet. We will look at the current standard, and what is coming tomorrow. Finally, I will answer the question on every web developer's mind: is it better to bundle the JavaScript files, or can HTTP efficiently serve thousands of small source files? Presented at CityJS Athens Greece 2022

  • 1,878