Principles of Computer Systems

Autumn 2019

Stanford University

Computer Science Department

Instructors: Chris Gregg

                            Philip Levis

PDF of this presentation

Lecture 16: Networks, HTTP, Proxies and MapReduce

  • Computer networks are a type of computer system
  • Most exciting systems and applications today are networked
  • Today we'll look under the covers a bit about how networks are designed and work
    • Layering and encapsulation
    • The 7 OSI layers
  • This lecture will just skim the surface: if you want to learn more, take CS144

Computer Networks

  • Computer networks mean your application is no longer limited to the data that it has locally on your own machine
    • It can receive data from and send data to other machines
    • This data can be interactive (people), not just documents
  • Today, when we talk about computer networks we generally mean the Internet
    • There were many networking technologies before and concurrent with the Internet
    • The Internet won! Thankfully. Ever heard of IPX, SNA, AppleTalk?
  • Operating systems provide two basic abstractions to the Internet
    • A reliable, in-order byte stream to an application running on another computer
      • Provided by TCP (Transmission Control Protocol)
      • Used for: ssh, http, email, NFS, etc.
    • An unreliable message (finite datagrams) service to an application running on another computer
      • Provided by UDP (User Datagram Protocol)
      • Used for DNS, some latency-sensitive applications (voice, video, gaming)

The Basic Benefit and Two Abstractions

  • Computer networks mean your application is no longer limited to the data that it has locally on your own machine
    • It can receive data from and send data to other machines
    • This data can be interactive (people), not just documents
  • Today, when we talk about computer networks we generally mean the Internet
    • There were many networking technologies before and concurrent with the Internet
    • The Internet won! Thankfully. Ever heard of IPX, SNA, AppleTalk?
  • Operating systems provide two basic abstractions to the Internet
    • A reliable, in-order byte stream to an application running on another computer
      • Provided by TCP (Transmission Control Protocol)
      • Used for: ssh, http, email, NFS, etc.
    • An unreliable message (finite datagrams) service to an application running on another computer
      • Provided by UDP (User Datagram Protocol)
      • Used for DNS, some latency-sensitive applications (voice, video, gaming)

The Basic Benefit and Two Abstractions

Reliable, In-Order Byte Stream

  • A TCP connection behaves much like a bidirectional pipe
    • Both sides can read, both sides can write
    • Establishing a connection has nothing to do with forking: two programs can be on different computers across the globe
    • Usually, one computer (a server) waits for connection requests from clients
      • You've seen this with connect and accept
      • Other approaches are for edge cases and we won't worry about them
  • Making TCP work as well as it does has taken 30 years of continual research and engineering: people still publish papers on it (e.g., Keith Winstein, Balaji Prabhakar)
    • One example: TCP was originally designed for wide area networks, where nodes are milliseconds apart; making it work well in the datacenter, where nodes are microseconds apart, required changing a few of its algorithms (Balaji).
    • Another example: TCP is designed to be extremely robust and work in all kinds of network conditions. Can you tune TCP to perform better in certain network settings (e.g., LTE vs. WiFi vs. wired) and how much improvement will you see (Keith)?

Web: HTTP over Reliable, In-Order Byte Stream

Internet

Web: HTTP over Reliable, In-Order Byte Stream

Internet

open a connection

Web: HTTP over Reliable, In-Order Byte Stream

Internet

 GET / HTTP/1.1

write a request

Web: HTTP over Reliable, In-Order Byte Stream

Internet

 GET / HTTP/1.1

read the request

Web: HTTP over Reliable, In-Order Byte Stream

Internet

 HTTP/1.1 200 OK
 document data

write the response

Web: HTTP over Reliable, In-Order Byte Stream

Internet

 HTTP/1.1 200 OK
 document data

read the response

BitTorrent: Organizing a Swarm

Internet

  • Client learns locations of nodes storing "chunks" of file
  • Opens connections to these nodes, uses a "tit-for-tat"
    algorithm to share data
  • Each connection is a reliable, in-order byte stream
    • Might not actually be TCP for some low-level reasons
    • Modularity and layering: has identical API to
      TCP, just behaves a little differently so it doesn't
      wreck your video chat

BitTorrent peers

Client

Internet: Under the Covers

Internet

  • It's not a magic cloud (although it sometimes seems to be)
  • When you make a web request, there are four layers of name in play
    • Application layer: name of document (e.g., index.html)
    • Transport layer: port identifying what service (e.g., HTTP on port 80)
    • Network layer: global IP address identifying which computer (e.g., 171.67.76.12)
    • Link layer: MAC address specifying which device on a link (e.g., 00:25:90:e7:10:48)

Internet

  • Application layer: name of document (e.g., index.html)
  • Transport layer: port identifying what service (e.g., HTTP on port 80)
  • Network layer: global IP address identifying which computer (e.g., 171.67.76.12)
  • Link layer: MAC address specifying which device on a link (e.g., 00:25:90:e7:10:48)

Application

Transport

Network

Link

Network

Link

Network

Link

Network

Link

Application

Transport

Network

Link

Internet in 4 Layers

Internet

  • Application layer: name of document (e.g., index.html)
  • Transport layer: port identifying what service (e.g., HTTP on port 80)
  • Network layer: global IP address identifying which computer (e.g., 171.67.76.12)
  • Link layer: MAC address specifying which device on a link (e.g., 00:25:90:e7:10:48)

Application

Transport

Network

Link

Network

Link

Network

Link

Network

Link

Application

Transport

Network

Link

Internet in 4 Layers

Internet

  • Application layer: name of document (e.g., index.html)
  • Transport layer: port identifying what service (e.g., HTTP on port 80)
  • Network layer: global IP address identifying which computer (e.g., 171.67.76.12)
  • Link layer: MAC address specifying which device on a link (e.g., 00:25:90:e7:10:48)

Application

Transport

Network

Link

Network

Link

Network

Link

Network

Link

Application

Transport

Network

Link

Internet in 4 Layers

Internet

  • Application layer: name of document (e.g., index.html)
  • Transport layer: port identifying what service (e.g., HTTP on port 80)
  • Network layer: global IP address identifying which computer (e.g., 171.67.76.12)
  • Link layer: MAC address specifying which device on a link (e.g., 00:25:90:e7:10:48)

Application

Transport

Network

Link

Network

Link

Network

Link

Network

Link

Application

Transport

Network

Link

Internet in 4 Layers

Internet

  • Application layer: name of document (e.g., index.html)
  • Transport layer: port identifying what service (e.g., HTTP on port 80)
  • Network layer: global IP address identifying which computer (e.g., 171.67.76.12)
  • Link layer: MAC address specifying which device on a link (e.g., 00:25:90:e7:10:48)

Application

Transport

Network

Link

Network

Link

Network

Link

Network

Link

Application

Transport

Network

Link

Internet in 4 Layers

Internet

  • Application layer: name of document (e.g., index.html)
  • Transport layer: port identifying what service (e.g., HTTP on port 80)
  • Network layer: global IP address identifying which computer (e.g., 171.67.76.12)
  • Link layer: MAC address specifying which device on a link (e.g., 00:25:90:e7:10:48)

Application

Transport

Network

Link

Network

Link

Network

Link

Network

Link

Application

Transport

Network

Link

Internet in 4 Layers

Internet in 4 Layers

Internet

  • Application layer: name of document (e.g., index.html)
  • Transport layer: port identifying what service (e.g., HTTP on port 80)
  • Network layer: global IP address identifying which computer (e.g., 171.67.76.12)
  • Link layer: MAC address specifying which device on a link (e.g., 00:25:90:e7:10:48)

Application

Transport

Network

Link

Network

Link

Network

Link

Network

Link

Application

Transport

Network

Link

Internet in 4 Layers

Internet

  • Application layer: name of document (e.g., index.html)
  • Transport layer: port identifying what service (e.g., HTTP on port 80)
  • Network layer: global IP address identifying which computer (e.g., 171.67.76.12)
  • Link layer: MAC address specifying which device on a link (e.g., 00:25:90:e7:10:48)

Application

Transport

Network

Link

Network

Link

Network

Link

Network

Link

Application

Transport

Network

Link

Hop by Hop

Internet

  • A packet from one node to another takes many hops to get to its destination
  • At each hop, the packet is sent to the link layer address of the next hop
  • Each hop  checks the IP (network) address to see if it's the final destination
    • If no, forward to next hop using a new link layer destination
    • If yes, bubble up to transport and potentially application layers

Application

Transport

Network

Link

Network

Link

Network

Link

Network

Link

Application

Transport

Network

Link

Hop by Hop

Internet

  • A packet from one node to another takes many hops to get to its destination
  • At each hop, the packet is sent to the link layer address of the next hop
  • Each hop  checks the IP (network) address to see if it's the final destination
    • If no, forward to next hop using a new link layer destination
    • If yes, bubble up to transport and potentially application layers

Application

Transport

Network

Link

Network

Link

Network

Link

Network

Link

Application

Transport

Network

Link

How many hops do you think it is from here to MIT? UC Berkeley?

7 Layer Model of Networks

Application

Presentation

Session

Transport

Network

Link

Physical

Application level data: HTTP

Let's not worry about this one

A communication session: TLS

Application: TCP, UDP ports

Node: IP address

A network interface: MAC

How you encode bits

Layering and Modularity

  • One of the seven principles of system design
  • Layering: decomposing a system into a well defined set of services, each with a clear interface it provides above and an interface it uses below
    • Internet and network protocols are the canonical example
  • Transmission Control Protocol (TCP)
    • Provides a bidirectional, reliable, in-order byte stream above
    • Uses an unreliable datagram protocol below
  • If layers meet these interfaces very precisely, they are modular: they can be interchanged
    • TCP has many variants: NewReno, CUBIC, Compound, DCTCP
    • Your application can use any of them, they all provide the same interface
    • They all use the Internet Protocol: IP
    • They are interchangeable and so are modular
    • TLS (Transport Layer Security) is a session layer protocol that secures TCP: it looks likes a bidrectional, reliable, in-order byte stream, so it can often be swapped in for TCP (e.g., https: instead of http:)

AMA about networks

Example Layer 7 Protocol: HyperText Transfer Protocol (HTTP)

HTTP/1.x Properties

  • HTTP is an ASCII protocol: protocol is in text, not binary values
    • In contrast to IP and TCP, which use binary: a 32-bit IP address is a 32-bit number
    • Makes everything much simpler and easier to debug!
    • ASCII is expensive, but if you're sending large files it's a small overhead
    • HTTP/2.0 is binary, trying to optimize things for mobile devices
  • Basic model: request, response

HTTP/1.x Request

  • Three principal methods
    • GET: fetch a resource (page, image, etc.)
    • HEAD: fetch only the headers (useful to see if the resource has changed)
    • POST: send data

HTTP/1.x Response

  • Five classes of status codes
    • 1xx: hold on
    • 2xx: here you go
    • 3xx: go away
    • 4xx: you screwed up
    • 5xx: I screwed up

Let's Look at a Request/Reponse in telnet and Firefox

partysaur:~ pal$ telnet www.scs.stanford.edu 80
Trying 2001:470:806d:1::9...
Connected to www.scs.stanford.edu.
Escape character is '^]'.
GET / HTTP/1.1
Host: www.scs.stanford.edu

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 5624

HTTP Web Proxy

  • A proxy server acts as a go-between a node and other nodes on the Internet
    • Clients opt-in to a proxy server (we'll see in a moment)
    • A web proxy handles client's HTTP requests and responds to them
  • Proxy servers can be useful
    • Cache requests for static data (e.g., images) so it can later serve local copies rather than re-request from the web (happens a lot in mobile devices), welcome to 1999
    • Intercept image requests to scale them down/make them more efficient (e.g., Google Flywheel)
  • If clients don't opt-in (it happens automatically), it's called a middlebox
    • Redirect to a paywall (e.g., what happens at airports)
    • Require agreeing to terms of service (e.g., Stanford Visitor)
    • Block access to certain websites (e.g., YouTube)
    • Block access to certain documents (big documents, .zip files, video streams.)

Web Proxy

Internet

Proxy Server

Client

Web Server

  • Client explicitly asks proxy server to fetch web page for it
    • Proxy  requests it from the server (or maybe another proxy server), maybe modifies it
    • Or, proxy gives a local cached copy

Middlebox

Middlebox

Client

Web Server

  • Client thinks it's requesting from the web server
    • Middlebox interposes, responds to request looking like web server
  • The difference between proxy and middlebox is the IP address to which the client sends its request. With a proxy, the client sends requests to the proxy. With a middlebox, it sends requests to the web server and the middlebox masquerades as the web server.

Middlebox

Middlebox

Client

Web Server

  • Client thinks it's requesting from the web server
    • Middlebox interposes, responds to request looking like web server
  • The difference between proxy and middlebox is the IP address to which the client sends its request. With a proxy, the client sends requests to the proxy. With a middlebox, it sends requests to the web server and the middlebox masquerades as the web server.
  • We have built a very basic proxy for you, which you are going to turn into a full web proxy
  • To test it, you need to set up Firefox (or another browser, but we suggest Firefox) to forward all web requests to the proxy:

Next Assignment: HTTP Web Proxy

  • To set up Firefox, go to Firefox->Preferences, then type "proxy" in the search box, then click on settings. You should have a window as above. Then, click on "Manual proxy configuration," and type in the myth machine and port number you get after starting your proxy:
$ ./proxy
Listening for all incoming traffic on port 19419.
  • Make sure you also select the checkbox for "Use this proxy server for all protocols."
  • Proxy requests use the full URL:

     
  • When a proxy server receives such as request, it forwards the request to www.cornell.edu, with the first line rewritten as follows:

 

Web Proxy Requests

GET http://www.cornell.edu/research/ HTTP/1.1
GET /research/ HTTP/1.1

Internet

Proxy Server

Client

Web Server

GET http://www.cornell.edu/research/ HTTP/1.1
GET /research/ HTTP/1.1
  • Properly proxying requires a couple of extra headers
  • x-forwarded-for: allows the proxy to tell the server who is making the request
    • Should be filled in with the IP address of the requesting client
    • If x-forwarded-for is already present, extend its value into a comma-separated chain of IP addresses the request has passed through before arriving at your proxy. The IP address of the machine you’re directly hearing from would be appended to the end.
      • E.g., if your proxy receives a request from 172.27.76.12 with

         
      • it will rewrite it to be

         

  • x-forwarded-proto: allows a server sitting behind a load-balancing proxy to be sure that the client request to the proxy was secure (https: instead of http:)

    • We're not going to worry about this use case, but we have to support it

    • Set its value to be http.  If x-forwarded-proto is already included in the request header, then simply add it again.

    • Here's a nice explanation

HTTP Web Proxy Headers

x-forwarded-for: 172.27.76.96
x-forwarded-for: 172.27.76.96, 172.27.76.12

AMA about HTTP and proxies

 The Genesis of Datacenter Computing: MapReduce

 The Genesis of Datacenter Computing: MapReduce

  • Problem for large service providers such as Google: computation requires hundreds or thousands of computers
    • How do you distribute the computation?
    • Distributing the computation boils down to distributing data
    • Nodes fail, some nodes are slow, load balancing: difficult to make it work well
  • System came from observation that many early Google systems had a common pattern
  • Designing a computing system around this pattern
    • allows the system (written once) to do all of the hard systems work (fault tolerance, load balancing, parallelization)
    • allows tens of thousands of programmers to just write their computation
  • Canonical example of how the right abstraction revolutionized computing
    • An open source version immediately appeared: Hadoop

Core Data Abstraction: Key/Value Pairs

  • Take a huge amount of data (more than can fit in the memory of 1000 machines)
  • Write two functions:



     
  • Using these two functions, MapReduce parallelizes the computation across thousands of machines, automatically load balancing, recovering from failures, and producing the correct result.
  • You can string together MapReduce programs: output of reduce becomes input to map.
  • Simple example of word count (wc):
map(k1, v1) -> list(k2, v2)
reduce(k2, list(v2)) -> list(v2)
map(String key, String value):
  // key: document name
  // value: document contents
  for word w in value:
    EmitIntermediate(w,"1")
    
reduce(String key, List values):
  // key: a word
  // values: a list of counts
  int result = 0
  for v in values:
    result += ParseInt(v)
  Emit(AsString(result))

"The number of partitions (R) and the partitioning function are specified by the user."

("the", "1") , ("the", "1"), ("The", "1"), ("of", 1),

(number, "1"), ...

"the", ("1", "1") -> "2"

"The", ("1") -> "1"

"of", ("1") -> "1"

"number", ("1") -> "1"

 

 map output

input

 reduce

Key/Value Pairs: How and Where

  • Keys allow MapReduce to distribute and parallelize load



     
  • In the original paper...
    • Each mapper writes a local file for each key in k2, and reports its files to a master node
    • The master node tells the reducer for k2 where the all of the k2 files are
    • The reducer reads all of the k2 files from the nodes that ran the mappers and writes its own output locally, reporting this to the master node
    • There have been lots of optimizations since
  • Core abstraction: data can be partitioned by key, there is no locality between keys
  • One problem: how much do you parallelize load?
    • One map per input file, one reduce per key, is too fine grained (small input)
    • Split the key space into R partitions: partition = (hash(key) % R)
    • Jeff Dean rule of thumb: if you have N nodes, R >= 10N
map(k1, v1) -> list(k2, v2)
reduce(k2, list(v2)) -> list(v2)

MapReduce System Architecture

Table From Paper (August 2004)

Google's IPO was September 2004

AMA about MapReduce

CS110 Lecture 16: Networks, HTTP, Proxies, and MapReduce

By philip_levis

Source: Chris Gregg

CS110 Lecture 16: Networks, HTTP, Proxies, and MapReduce

  • 2,288

More from philip_levis

  • CS110 Lecture 19: Events, Threads and Asynchronous I/O
    philip_levis
    2444
  • CS110 Lecture 12: Multithreading Design Patterns and Thread-Safe Data Structures
    philip_levis
    2350