Gradual Typing and Django - PyCon

Sri Lanka 2022

who am I?

Gradual Typing

Example


class Label(Model):
  name = StringField()

# No type hints
def fetch_from_label_table(name):
    conn = get_connection()
    return list(Label.objects.filter(name=name))


# Type hint code 
def fetch_from_label_table(name: str) -> list[Label]:
    conn = get_connection()
    return list(Label.objects.filter(name=name))

Type Checking

$pip install mypy


$mypy filename.py

Types at runtime

from django.contrib.auth.models import User


type(User.objects.filter(
	email='foo@example.com'))

# output
django.db.models.query.QuerySet
type(("127.0.0.1", 8000))

# output
tuple

Static Checker Type

# filename.py
addr = "127.0.0.1"
port = 8000

reveal_type((addr, port))
$mypy filename.py
note: Revealed type is 
  'Tuple[builtins.str, builtins.int]'
# filename.py
from django.contrib.auth.models import User

reveal_type(User.objects.filter(
  email='foo@example.com'))
$ mypy filename.py
note: Revealed type is 
  'django.contrib.auth.models.UserManager
  [django.contrib.auth.models.User]'
  

mypy config

$cat mypy.ini
...
plugins =
  mypy_django_plugin.main,
  
[mypy.plugins.django-stubs]
django_settings_module = "yourapp.settings"

Annotating Django Code

View

from django.http import (HttpRequest, HttpResponse, 
			HttpResponseNotFound)

def index(request: HttpRequest) -> HttpResponse:
    return HttpResponse("hello world!")

def view_404(request: HttpRequest) -> HttpResponse:
    return HttpResponseNotFound(
      'Page not found')

def view_404(request: 
             HttpRequest) -> HttpResponseNotFound:
    return HttpResponseNotFound(
      'Page not found')

# bad - not precise and not useful
def view_404(request: HttpRequest) -> object:
	return HttpResponseNotFound(
      'Page not found')

Method Resolution Order

HttpResponse.mro()
[django.http.response.HttpResponse,
 django.http.response.HttpResponseBase,
 object]


HttpResponseNotFound.mro()
[django.http.response.HttpResponseNotFound,
 django.http.response.HttpResponse,
 django.http.response.HttpResponseBase,
 object]

LSP

The LSP states 
that in an object-oriented program, 
substituting a superclass 
object reference with an object 
of any of its subclasses, 
the program should not break.

Django Models

Create

from django.db import models
from django.utils import timezone


class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField("date published")


def create_question(question_text: str) -> Question:
    qs = Question(question_text=question_text, 
                  pub_date=timezone.now())
    qs.save()
    return qs

Read


def get_question(question_text: str) -> Question:
    return Question.objects.filter(
      question_text=question_text).first()
error: Incompatible return value type 
(got "Optional[Any]", expected "Question")
from typing import Optional

def get_question(question_text: str) -> 
	Optional[Question]:
    return Question.objects.filter(
      question_text=question_text).first()

mypy config

# mypy.ini
strict_optional = False

def get_question(question_text: str) -> 
	Question:
    return Question.objects.filter(
      	question_text=question_text).first()

Filter

Example

In [8]: Question.objects.all()
Out[8]: <QuerySet [<Question: Question object (1)>, 
                   <Question: Question object (2)>]>

In [9]: Question.objects.filter()
Out[9]: <QuerySet [<Question: Question object (1)>, 
                   <Question: Question object (2)>]>
Box with cup cake

def filter_question(text: str) -> 
	QuerySet[Question]:
    return Question.objects.filter(
      text__startswith=text)
  
def exclude_question(text: str) -> 
	QuerySet[Question]:
    return Question.objects.exclude(
      text__startswith=text)

all, reverse, none, complex_filter, 
union, order_by, distinct, 
defer, only, using, extra, 
select_related, select_for_update, 
prefetch_related

Methods returning QuerySet

Tools

pyannotate

Pyannotate

  • Automatically insert type-hints into the code

 

  • pytest-annotate infer type from test cases

 

Source Code

from django.http import (HttpResponse, 
		HttpResponseNotFound)

# Create your views here.
# annotate the return value
def index(request):
    return HttpResponse("hello world!")

def view_404_0(request):
    return HttpResponseNotFound(
      'Page not found')

Test Code

from polls.views import *
from django.test import RequestFactory

def test_index():
    request_factory = RequestFactory()
    request = request_factory.post('/index')
    index(request)


def test_view_404_0():
    request_factory = RequestFactory()
    request = request_factory.post('/404')
    view_404_0(request)

Run test command

$DJANGO_SETTINGS_MODULE="mysite.settings" 
PYTHONPATH='.' poetry run pytest 
-sv polls/tests.py 
--annotate-output=./annotations.json
$cat annotations.json
[...
    {
        "path": "polls/views.py",
        "line": 7,
        "func_name": "index",
        "type_comments": [
            "(django.core.handlers.wsgi.WSGIRequest) -> 
          django.http.response.HttpResponse"
        ],
        "samples": 1
    },
    {
        "path": "polls/views.py",
        "line": 10,
        "func_name": "view_404_0",
        "type_comments": [
            "(django.core.handlers.wsgi.WSGIRequest) -> 
          django.http.response.HttpResponseNotFound"
        ],
        "samples": 1
    }
]

Apply the changes

$poetry run pyannotate --type-info 
./annotations.json 
-w polls/views.py --py3
from django.http import HttpResponse, HttpResponseNotFound
from django.core.handlers.wsgi import WSGIRequest
from django.http.response import HttpResponse
from django.http.response import HttpResponseNotFound

def index(request: WSGIRequest) -> HttpResponse:
    return HttpResponse("hello world!")

def view_404_0(request: WSGIRequest) -> 
	HttpResponseNotFound:
    return HttpResponseNotFound(
      '<h1>Page not found</h1>')

Auto-modified file

Learning Resource

Python Typing Koans

Should my new project use gradual typing?

Now

When?

Developer Productivity

  • Tests (unit, functional, integration)
  • Lints (pyflake)
  • Formatter (black)
  • Test Coverage
  • Documentation
  • CI/CD
  • Static Analysis (semgrep)
  • Type Hints

How to measure gains?

Thank you!

Twitter: @kracetheking

Github: kracekumar

Made with Slides.com