AsyncIO part 1:

Sockets

Taras Voinarovskyi

https://github.com/tvoinarovskyi

Taras Voinarovskyi

https://github.com/tvoinarovskyi

Collaborator:

Member of aio-libs team

What's on todays menu?

```
What is a Socket in modern systems
```
```
How threads work with sockets
```

How can a single thread work with multiple sockets using select

Sockets

Ip Packets

De-multiplexing

Convert host-to-host packet delivery into a process-to-process communication channel

Socket types

Socket - an interface between an application process and transport layer

Stream Sockets (SOCK_STREAM)

```
reliable two-way connection
```
```
TCP, Unix
```

Datagram Sockets (SOCK_DGRAM)

```
unreliable packet type transfer
```

```
...
```

To sum it up

Socket and Threads

# Echo client program
import socket

HOST = 'daring.cwi.nl'    # The remote host
PORT = 50007              # The same port as used by the server
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.connect((HOST, PORT))
    s.send(b'Hello, world')
    data = s.recv(1024)
print('Received', repr(data))

Socket and Threads

# Echo server program
import socket

HOST = ''       # Symbolic name meaning all available interfaces
PORT = 50007    # Arbitrary non-privileged port
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.bind((HOST, PORT))
    s.listen(1)
    conn, addr = s.accept()
    with conn:
        print('Connected by', addr)
        while True:
            data = conn.recv(1024)
            if not data: break
            conn.send(data)

sock.recv(1024)

def recv(sock, recv_length):
    if is_disconnected(sock):
        return b""
    if len(sock._buffer) == 0:
        wait_for_data(sock)
    assert len(sock._buffer) > 0

    res = sock._buffer[:recv_length]
    sock._buffer = sock._buffer[recv_length:]
    return res

```
Blocks if no data available in buffer
```
```
May return only 1 byte of data
```

sock.send(data)

def send(sock, data):
    if is_disconnected(data):
        raise BrokenPipeError
    if len(sock._buffer) == MAX_BUFFER_LEN:
        wait_for_send(sock)
    assert len(sock._buffer) < MAX_BUFFER_LEN

    can_send = MAX_BUFFER_LEN - len(sock._buffer)
    if can_send >= len(data):  # Full send
        sock._buffer += data
        return len(data)
    else:   # Partial send
        sock._buffer += data[:can_send]
        return can_send

```
Blocks if buffer has no free space
```
```
May send as little at 1 byte from data
```

Multitasking on Select

readers = {accept_fd}
writers = set()

while True:
    r, w, _ = select.select(readers, writers, [])

    print("Looping...")
    for fd in r:
        if fd == accept_fd:
            on_accept()
        else:
            on_read_ready(fd)
    for fd in w:
        on_write_ready(fd)

Multitasking on Select

def on_accept():
    conn, addr = accept_sock.accept()
    print('Connected by', addr)
    readers.add(conn.fileno())
    connected[conn.fileno()] = conn

Multitasking on Select

write_buffers = defaultdict(bytearray)

def on_read_ready(fd):
    conn = connected[fd]
    data = conn.recv(1024)
    print("Received", data)
    if not data:
        del connected[fd]
        readers.remove(fd)
        if fd in writers:
            writers.remove(fd)
    # Process data itself. In our case echo it back
    writers.add(fd)
    write_buffers[fd] += data

Multitasking on Select

def on_write_ready(fd):
    conn = connected[fd]

    write_buffer = write_buffers[fd]
    sent_bytes = conn.send(write_buffer)
    if len(write_buffer) != sent_bytes:
        write_buffer[:sent_bytes] = b""
    else:
        write_buffer[:] = b""
        writers.remove(fd)

Questions?

David Beazley - Python Concurrency From the Ground Up:
https://www.youtube.com/watch?v=MCs5OvhV9S4

AsyncIO documentation: https://docs.python.org/3/library/asyncio.html

AsyncIO pitfalls:
https://www.youtube.com/watch?v=GLN_xo4Awcc

Neet references