HTTP communication

Overview

Request

Overview

Request

Response

Request

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

HTTP verbs

GET: read and do not change

state

POST: change the state

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

HTTP verbs

GET: read and do not change

state

POST: change the state

Other verbs: HEAD, PUT, OPTIONS, DELETE, ...

Semantic is a convention

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Path

String processed by server

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Path

String processed by server

Encoding convention:

space -> %20

https://en.wikipedia.org/wiki/Percent-encoding

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Protocol

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Protocol

HTTP/1.1

ASCII (text) based

HTTP/2.0

Binary based

Reduce latency

Backward compatibility

https://en.wikipedia.org/wiki/HTTP/2

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Headers

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Headers

Host is mandatory for all

requests

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Accept
format in which we
ASK the data
application/json

text/xml
text/html
...

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Content-Type
format in which we
SEND the data
application/json

text/xml
multipart/form-data
...

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

A string the server
previously sent to us

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Content-Length

Mandatory for POST and
PUT request.

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

Request

POST /messages HTTP/1.1
Host: www.example.com
Content-Type: application/json
Cookie: sessionid=aa03x;
Content-Length: 79

{ "from": "Marvin", "to": "Arthur",
  "content": "Did I say something wrong?" }

Body: the data we send to the server -- optional -- after a blank line.

Raw string
JSON
XML

... whatever the server
accepts

GET / HTTP/1.1
Host: www.google.fr
Accept: text/html

A lot of custom headers

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers

A LOT of custom headers

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers

Overview

Request

Response

HTTP/1.1 200 OK
Date: Tue, 24 Jan 2012 18:34:40 GMT
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: text/html
Set-Cookie: sessionid=jkWXBR;

<h1>Hello world!</h1>

HTTP/1.1 400 BAD_REQUEST
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: application/json


{ "error": "email is required" }

Response

HTTP/1.1 200 OK
Date: Tue, 24 Jan 2012 18:34:40 GMT
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: text/html
Set-Cookie: sessionid=jkWXBR;

<h1>Hello world!</h1>

HTTP/1.1 400 BAD_REQUEST
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: application/json


{ "error": "email is required" }

Protocol:

HTTP/1.1
HTTP/2.0

Response

HTTP/1.1 200 OK
Date: Tue, 24 Jan 2012 18:34:40 GMT
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: text/html
Set-Cookie: sessionid=jkWXBR;

<h1>Hello world!</h1>

HTTP/1.1 400 BAD_REQUEST
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: application/json


{ "error": "email is required" }

Status code

   200 OK
   201 CREATED
   301 MOVE
PERMANENTLY
   307 TEMPORARY
   REDIRECT
400 BAD REQUEST
403 FORBIDDEN
404 NOT FOUND
418 I'M A TEAPOT
500 SERVER ERROR
503 SERVICE
   UNAVAILABLE

Response

HTTP/1.1 200 OK
Date: Tue, 24 Jan 2012 18:34:40 GMT
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: text/html
Set-Cookie: sessionid=jkWXBR;

<h1>Hello world!</h1>

HTTP/1.1 400 BAD_REQUEST
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: application/json


{ "error": "email is required" }

Headers.

What the server wants to tell to us.

Response

HTTP/1.1 200 OK
Date: Tue, 24 Jan 2012 18:34:40 GMT
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: text/html
Set-Cookie: sessionid=jkWXBR;

<h1>Hello world!</h1>

HTTP/1.1 400 BAD_REQUEST
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: application/json


{ "error": "email is required" }

Content-Type: the actual format in which the server send the data.

Can be different from the request's Accept header.

Response

HTTP/1.1 200 OK
Date: Tue, 24 Jan 2012 18:34:40 GMT
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: text/html
Set-Cookie: sessionid=jkWXBR;

<h1>Hello world!</h1>

HTTP/1.1 400 BAD_REQUEST
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: application/json


{ "error": "email is required" }

Set-Cookie: the server

asks us to keep in memory a string

This string should be send back to the server at the next request.

Response

HTTP/1.1 200 OK
Date: Tue, 24 Jan 2012 18:34:40 GMT
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: text/html
Set-Cookie: sessionid=jkWXBR;

<h1>Hello world!</h1>

HTTP/1.1 400 BAD_REQUEST
Server: Apache/2.2.21 (Debian)
Content-Length: 21
Content-Type: application/json


{ "error": "email is required" }

Body -- optional -- after a blank line.

The actual payload of the response.

Overview

Request

Response

Only conventions!

Hands on!

Open the developer tools, select "Network" tab.

Enter the (exact) url https://wikipedia.org an observe the requests/responses!

Cookies: sid=aJ5Ke5;__utma=142543.221.52;__utmz=521441

http://my-site.com

Cookies: ssid=5aKa47ec;alert=false;v=0.19

http://example.com

Cookies: __utma=98465.316542213.65;__utmz=152357.6368432

http://bingo.org

are used to identify users

Cookies: sid=aJ5Ke5;__utma=142543.221.52;__utmz=521441

http://my-site.com

Cookies: ssid=5aKa47ec;alert=false;v=0.19

http://example.com

Cookies: __utma=98465.316542213.65;__utmz=152357.6368432

http://bingo.org

are isolated

Cookies: sid=aJ5Ke5;__utma=142543.221.52;__utmz=521441

http://my-site.com

Cookies: ssid=5aKa47ec;alert=false;v=0.19

http://example.com

Cookies: __utma=98465.316542213.65;__utmz=152357.6368432

http://bingo.org

Hands on!

Open the developer tools, select "Storage" tab.

Browse in multiple websites and observe the cookies!

Security

Plain Text : Every body can intercept...

Plain Text : ... or even modify the messages

Solution: HTTPS

"S" for "Secure"
https://en.wikipedia.org/wiki/HTTPS

Use for example

Let's encrypt

https://letsencrypt.org/

Use for example

Let's encrypt

https://letsencrypt.org/

Issue requests

Issue GET requests

GET /fr/ HTTP/1.1
Host: www.mozilla.org

Issue GET requests

<form action="/search">
    <input name="q" type="text">
    <input name="num" type="number">
    <input type="submit">
</form>

Issue GET requests

<form action="/search">
    <input name="q" type="text">
    <input name="num" type="number">
    <input type="submit">
</form>

GET /search?q=Marvin&num=3 HTTP/1.1
Host: www.example.com

Submit

Issue GET requests

<form action="/search" method="GET">
    <input name="q" type="text">
    <input name="num" type="number">
    <input type="submit">
</form>

GET /search?q=Marvin&num=3 HTTP/1.1
Host: www.example.com

Default method

Issue GET requests

Issue POST requests

<form action="/message" method="POST">
    <input name="to" type="text">
    <input name="content" type="text">
    <input type="submit">
</form>

Issue POST requests

<form action="/message" method="POST">
    <input name="to" type="text">
    <input name="content" type="text">
    <input type="submit">
</form>

Issue POST requests

<form action="/message" method="POST">
    <input name="to" type="text">
    <input name="content" type="text">
    <input type="submit">
</form>

POST /message HTTP/1.1
Host: www.example.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 52

to=Ford&content=Did%20I%20say%20something%20wrong%3F

Submit

Issue POST requests

<form action="/message" method="POST">
    <input name="to" type="text">
    <input name="content" type="text">
    <input type="submit">
</form>

POST /message HTTP/1.1
Host: www.example.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 52

to=Ford&content=Did%20I%20say%20something%20wrong%3F

Urlencoded: "Did I say something wrong?"

Answer a request

Web frameworks handle details

from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"

if __name__ == '__main__':
    app.run()

Web frameworks handle details

from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"

if __name__ == '__main__':
    app.run()

The user sent the request:

GET / HTTP/1.1
Host: www.example.com

Web frameworks handle details

from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"

if __name__ == '__main__':
    app.run()

The server answers:

HTTP/1.1 200 OK  
Content-Length: 12
Content-Type: text/html

Hello world!

@app.route("/", methods=["GET"])
def hello():
    name = request.args['name']  
    return "Hello " + name + "!"

Get the arguments!

@app.route("/", methods=["GET"])
def hello():
    name = request.args['name']  
    return "Hello " + name + "!"

The user sends the request:

GET /?name=Arthur%20Dent HTTP/1.1
Host: www.example.com

Get the arguments!

@app.route("/", methods=["GET"])
def hello():
    name = request.args['name']  
    return "Hello " + name + "!"

The user sends the request:

GET /?name=Arthur%20Dent HTTP/1.1
Host: www.example.com

Get the arguments!

The server answers:

HTTP/1.1 200 OK  
Content-Length: 18
Content-Type: text/html

Hello Arthur Dent!

@app.route("/", methods=["GET"])
def hello():
    name = request.args['name']  
    return "Hello " + name + "!"

The user send the request:

GET /?name=Arthur%20Dent HTTP/1.1
Host: www.example.com

Get the arguments!

The server answers:

HTTP/1.1 200 OK  
Content-Length: 18
Content-Type: text/html

Hello Arthur Dent!

Never trust user input!

<script>
  id = get_session_id(); 
  send_to_Ford(id);
</script>

<script>
  id = get_session_id(); 
  send_to_Ford(id);
</script>

1. URL-encode

https://attacked-site/
?name=%3Cscript%3E%0A%20
%20id%20%3D%20get_session
_id%28%29%3B%20%0A%20%20s
end_to_Ford%28id%29%3B%0A
%3C%2Fscript%3E

<script>
  id = get_session_id(); 
  send_to_Ford(id);
</script>

1. URL-encode

<p>Hey folks!</p>

<p>Check out my 
<a href="...">
new website!</a>
</p>

2. Send this email

https://attacked-site/
?name=%3Cscript%3E%0A%20
%20id%20%3D%20get_session
_id%28%29%3B%20%0A%20%20s
end_to_Ford%28id%29%3B%0A
%3C%2Fscript%3E

1. sends the link via emails or publishes it on a website

2. clicks on the link

1. sends the link via emails or publishes it on a website

2. clicks on the link

3. Generates the malicious HTML code

1. sends the link via emails or publishes it on a website

2. clicks on the link

3. Generates the malicious HTML code

4. sends back the HTML code

1. sends the link via emails or publishes it on a website

2. clicks on the link

3. Generates the malicious HTML code

4. sends back the HTML code

5. runs the code and sends data

Ford has stolen your identity

@app.route("/", methods=["GET"])
def hello():
    name = request.args['name']  
    return "Hello " + name + "!"

The user sends the request:

GET /?name=Arthur%20Dent HTTP/1.1
Host: www.example.com

Get the arguments!

The server answers:

HTTP/1.1 200 OK  
Content-Length: 18
Content-Type: text/html

Hello Arthur Dent!

@app.route("/", methods=["POST"])
def hello():
    name = request.form['name']  
    return "Hello " + name + "!"

The user sends the request:

POST / HTTP/1.1
Host: www.example.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 18

name=Arthur%20Dent

Get the arguments from POST!

@app.route("/", methods=["POST"])
def hello():
    name = request.form['name']  
    return "Hello " + name + "!"

The user sends the request:

POST / HTTP/1.1
Host: www.example.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 18

name=Arthur%20Dent

Get the arguments from POST!

@app.route("/", methods=["POST"])
def hello():
    name = request.form['name']  
    return "Hello " + name + "!"

The user sends the request:

POST / HTTP/1.1
Host: www.example.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 18

name=Arthur%20Dent

Get the arguments from POST!

Use templates!

@app.route("/", methods=["GET"])
def hello():
    name = request.args['name']  
    return "Hello " + name + "!"

Use templates!

@app.route("/", methods=["GET"])
def hello():
    name = request.args['name']  
    return render_template("hi.html", name=name)

server.py

Use templates!

@app.route("/", methods=["GET"])
def hello():
    name = request.args['name']  
    return render_template("hi.html", name=name)

server.py

<p>Hello {{ name }}</p>

hi.html

Use templates!

@app.route("/", methods=["GET"])
def hello():
    name = request.args['name']  
    return render_template("hi.html", name=name)

server.py

<p>Hello {{ name }}</p>

hi.html

<p>Hello Arthur</p>

sent to the browser

<p>Hello {{ name }}</p>

(almost) any Python expression

Jinja syntax

<p>Hello {{ name }}</p>

(almost) any Python expression

<p>Hello {{ name.title() }}</p>

Jinja syntax

<p>Hello {{ name }}</p>

(almost) any Python expression

<p>Hello {{ name.title() }}</p>

with name: "arthur dent"

<p>Hello Arthur Dent</p>

Jinja syntax

<p>Hello {{ name }}</p>

escaped by default

Jinja syntax

with name: "Arthur <script>alert('...')</script>"

<p>Hello {{ name }}</p>

escaped by default

Jinja syntax

with name: "Arthur <script>alert('...')</script>"

<p>Hello
&lt;script&gt;alert(&#39;...&#39;)&lt;/script&gt;
</p>

sent to the browser

<p>Hello {{ name }}</p>

escaped by default

Jinja syntax

with name: "Arthur <script>alert('...')</script>"

<p>Hello
&lt;script&gt;alert(&#39;...&#39;)&lt;/script&gt;
</p>

sent to the browser

displayed by the browser

{% if likes_count > 100 %}
	<p>Hey, you are very popular, 
       you are liked by {{ likes_count }} 
       people
	</p>
{% elif likes_count > 10 %}
    <p>You are liked by 
       {{ likes_count }} people
	</p>
{% else %}
    <p>Only
       {{ likes_count }} people
      	like you... It is so sad...
	</p>
{% endif %}

If branching

<ul>
{% for name in names %}
   <li>{{ name }}</li>
{% end %}
</ul>

For loops

<ul>
{% for name in names %}
   <li>{{ name }}</li>
{% end %}
</ul>

For loops

with users: [ "Arthur", "Ford", "Marvin"]

<ul>
    <li>Arthur</li>
    <li>Ford</li>
    <li>Marvin</li>
</ul>

<!DOCTYPE html>
<html>
  <head>
    <title>{% block title %}{% endblock %}</title>
  </head>
  <body>
    <h1>Main heading</h1>
    {% block content %}{% endblock %}
  </body>
</html>

Inheritance

base.html

<!DOCTYPE html>
<html>
  <head>
    <title>{% block title %}{% endblock %}</title>
  </head>
  <body>
    <h1>Main heading</h1>
    {% block content %}{% endblock %}
  </body>
</html>

Inheritance

base.html

defined by the programmer

Inheritance

{% extends "base.html" %}

{% block title %}Welcome{% endblock %}

{% block content %}
<p>
  Hello {{ name }}
</p>
{% endblock %}

hello.html

Inheritance

{% extends "base.html" %}

{% block title %}Login{% endblock %}

{% block content %}
<form>
  <input name="login" />
  <input name="password" type="password" />
  <input type="submit" />
</form>
{% endblock %}

Hackathon Elm : October 5, 9.00 - 19.00

https://hackathon-elm-france.github.io/

Meetup : September 26, 19.00-22.00

https://www.meetup.com/fr-FR/Meetup-Elm-Paris/events/264123481/

HTTP Communication - TWTS 03

By sebbes

HTTP Communication - TWTS 03

HTTP communication

Overview

Overview

Overview

Request

Request

Request

Request

Request

Request

Request

Request

Request

Request

Request

Request

Request

Request

A lot of custom headers

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers

A LOT of custom headers

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers

Overview

Response

Response

Response

Response

Response

Response

Response

Overview

Only conventions!

Hands on!

Open the developer tools, select "Network" tab. Enter the (exact) url https://wikipedia.org an observe the requests/responses!

are used to identify users

are isolated

Hands on!

Open the developer tools, select "Storage" tab. Browse in multiple websites and observe the cookies!

Security

Plain Text : Every body can intercept...

Plain Text : ... or even modify the messages

Solution: HTTPS

"S" for "Secure" https://en.wikipedia.org/wiki/HTTPS

Use for example

Let's encrypt

Use for example

Let's encrypt

Issue requests

Issue GET requests

Issue GET requests

Issue GET requests

Issue GET requests

Issue GET requests

Issue POST requests

Issue POST requests

Issue POST requests

Issue POST requests

Answer a request

Web frameworks handle details

Web frameworks handle details

Web frameworks handle details

Get the arguments!

Get the arguments!

Get the arguments!

Get the arguments!

Never trust user input!

Ford has stolen your identity

Get the arguments!

Get the arguments from POST!

Get the arguments from POST!

Get the arguments from POST!

Use templates!

Use templates!

Use templates!

Use templates!

Use templates!

Jinja syntax

Jinja syntax

Jinja syntax

Jinja syntax

Open the developer tools, select "Network" tab.

Enter the (exact) url https://wikipedia.org an observe the requests/responses!

Open the developer tools, select "Storage" tab.

Browse in multiple websites and observe the cookies!

"S" for "Secure"
https://en.wikipedia.org/wiki/HTTPS