Internet Protocol Libraries - Python

PyCon India 2010, Bangalore

Senthil Kumaran

About Me

  • Python Core Developer.

  • Maintainer of urllib2 and related modules.

  • Software Engineer at Akamai, India.

Status of Python

  • Python 3.2a2+ (This is the trunk)

  • Python 3.1.2 (release31-maint)

  • Python 2.7 (release27-maint)

Unsupported versions:

  • Python 3.0 is not there!

  • Python 2.6.5 (release26-maint)

Internet Protocols and Support

  • Most of the modules are built on top of socket module.

  • They are highlevel modules implmenting Specifications mostly from RFC.

  • Provide commonly used helper functions.

Applications which are using these

  • Mercurial

  • Google Code (and google-cl)

  • youtube-dl

  • Bittorrent

  • Mailman

Modules in 2 and 3

  • webbrowser

  • cgi

  • wsgiref

  • uuid

Modules in 2 and 3

  • less commonly used protocol modules.

  • ftplib

  • poplib

  • imaplib

  • smtplib

  • telnetlib

urllib package in python3

  • urllib -> urllib.request, urllib.response, urllib.parse

  • urllib2 -> urllib.request, urllib.response, urllib.parse

  • urlparse -> urllib.parse

http://www.python.org/dev/peps/pep-3108/

http package in python3

  • httplib -> http.client

  • cookiekib -> http.cookiejar

  • Cookie -> http.cookies

  • BaseHTTPServer -> http.server

  • SimpleHTTPServer -> http.server

  • CGIHTTPServer -> http.server

  • SocketServer -> socketserver

xmlrpc package in python3

  • xmlrpclib -> xmlrpc.client

  • SimpleXMLRPCServer -> xmlrpc.server

  • DocXMLRPCServer -> xmlrpc.server

webbrowser

  • Module for Handling all different kinds of browsers

url = 'http://www.python.org/'
webbrowser.open_new_tab(url + 'doc/')
webbrowser.open_new(url)

cgi

  • Support for Common Gateway Interface Scripts

print "Content-Type: text/html"
print
print "<h1>Hello,World</h1>"
  • FieldStorage class for handling forms.

import cgitb
cgitb.enable()

wsgiref

  • The Web Server Gateway Interface (WSGI)

  • PEP 333

  • Django, TurboGears, App Engine, all many different Web Frameworks.

  • Standardization helps. You speak one language and follow similar protocols.

  • wsgiref is the reference implementation of WSGI Specification.

wsgiref

from wsgiref.simple_server import make_server
def hello_world_app(environ, start_response):
    status = '200 OK'
    headers = [('Content-type', 'text/plain')]
    start_response(status, headers)
    return ["Hello World"]
httpd = make_server('', 8000, hello_world_app)
httpd.serve_forever()

uuid

  • Universally Unique Identifier (RFC 4122)

>>> import uuid
>>> print uuid.uuid1()
32fb97c6-c8d3-11df-a47f-00216bad94a4
>>> print uuid.uuid4()
f9e0a0cd-23dd-46c8-95d6-e0bb0198c121

urllib

  • Higher Level Interface for Fetching Data from the Web.

  • Clients/ Client libraries which handle URLs.

  • urllib.urlopen

urllib2

  • Next Version of urllib.

  • Create a Request Object.

  • Have different Handlers depending upon the Protocol.

  • Get the Response by the corresponding Handler object.

Design Patterns in urllib2

  • Builder Design Pattern

https://dl.dropboxusercontent.com/s/h8xf08gg7qrgz1u/Builder2.png

urlparse

  • Parse the URLs into its constituents.

  • It is required when are building bigger applications.

  • Referencing sections within pages.

  • Parse IPv6 URLs too.

httplib

  • It is the underlying layer to the urllib.

  • It implements HTTP/1.1 spec.

  • Implements the HTTP States on top of socket.

ftplib

  • Defines the FTP class and related items. The FTP class implments the client side FTP protocol.

  • Internally used by urllib module for handling ftp urls.

  • Got support FTP TLS from Python 2.7 onwards.

SocketServer

  • Handling synchronous network requests.

  • Helper classes for Mixins.

+------------+
| BaseServer |
+------------+
      |
      v
+-----------+        +------------------+
| TCPServer |------->| UnixStreamServer |
+-----------+        +------------------+
      |
      v
+-----------+        +--------------------+
| UDPServer |------->| UnixDatagramServer |
+-----------+        +--------------------+

Other Servers

  • BaseHTTPServer

  • SimpleHTTPServer

  • CGIHTTPServer

python -m SimpleHTTPServer 8000

xmlrpclib

  • XML-RPC client code; it handles all the details of translating between conformable Python objects and XML on the wire.

import xmlrpclib
proxy = xmlrpclib.ServerProxy("http://localhost:8000/")
print "3 is even: %s" % str(proxy.is_even(3))
print "100 is even: %s" % str(proxy.is_even(100))

SimpleXMLRPCServer

import xmlrpclib
from SimpleXMLRPCServer import SimpleXMLRPCServer

def is_even(n):
    return n%2 == 0

server = SimpleXMLRPCServer(("localhost", 8000))
print "Listening on port 8000..."
server.register_function(is_even, "is_even")
server.serve_forever()

Good Resources

Thank you. :)

Senthil Kumaran <orsenthil@gmail.com>

Internet Protocol Libraries of Python

By Senthil Kumaran

Internet Protocol Libraries of Python

Internet Protocol Libraries in Python

  • 1,128