Pydantic

 

Give your data classes super powers with pydantic

Alexander Hultnér

Python Pizza, Remote 2020

@ahultner

Alexander Hultnér

Founder of Hultnér Technologies

 

 

@ahultner

Outline

  • Quick refresher on python data classes
  • Pydantic introduction
  • Cool features worth mentioning
  • The future

@ahultner

Dataclasses

Let's start with a quick @dataclass-refresher.

Well use pizza-based examples in the spirit of python.pizza 🐍🍕

from dataclasses import dataclass
from typing import Tuple

@dataclass
class Pizza:
    style: str
    toppings: Tuple[str, ...]
>>> Pizza(1, ("cheese", "ham"))
Pizza(style=1, toppings=('cheese', 'ham'))

And now let's try it out

@ahultner

Dataclasses

Now we may want to constrain the toppings to ones we actually offer.


Our pizzeria doesn't offer pineapple 🚫🍍 as a valid topping, hate it 😡 or love it 💕

class Topping(str, Enum):
    mozzarella = 'mozzarella'
    tomato_sauce = 'tomato sauce'
    prosciutto = 'prosciutto'
    basil = 'basil'
    rucola = 'rucola'
    

@dataclass
class Pizza:
    style: str
    toppings: Tuple[Topping, ...]
>>> Pizza(2, ("pineapple", 24))
Pizza(style=2, toppings=('pineapple', 24))

Let's see what happens if we try to create a pizza with pineapple 🍍 topping.

@ahultner

  • Python Library 🐍
  • Great documentation 📖
  • Data validation using python type annotations 🧹🧐
  • Runtime type enforcement 💯
  • User-friendly errors 👨‍💻
  • No convoluted syntax, pure pythonic classes
  • (De)serialisation
  • Predecessors 🏛
    • Dataclasses, attrs, marshmallow, valideer, ORM-libraries, etc.

Pydantic

Quick introduction


 

@ahultner

With dataclasses the types aren't enforced.

But in this case we'll lean on the shoulders of a giant, pydantic 🧹🐍🧐

from pydantic.dataclasses import dataclass

@dataclass
class Pizza:
    style: str
    toppings: Tuple[Topping, ...]
from pydantic import ValidationError
try:
    Pizza(2, ("pineapple", 24))
except ValidationError as err:
    print(err)

And with that simple chage we can see that our new instance of an invalid pizza actually raises errors 🚫🚨

Additionally these errors are very readable!

2 validation errors for Pizza

toppings -> 0
  value is not a valid enumeration member; permitted: 'mozzarella', 'tomato sauce', 'prosciutto', 'basil', 'rucola' (type=type_error.enum; enum_values=[ … ])

toppings -> 1
  value is not a valid enumeration member; permitted: 'mozzarella', 'tomato sauce', 'prosciutto', 'basil', 'rucola' (type=type_error.enum; enum_values=[ … ])

Pydantic

Runtime type-checking, data class drop-in replacement


 

@ahultner

So let's try to create a valid pizza 🍕✅

Pizza(
  "Napoli", 
  (
    Topping.tomato_sauce, 
    Topping.prosciutto, 
    Topping.mozzarella, 
    Topping.basil,
  )
)
Pizza(
  style='Napoli', 
  toppings=(
    <Topping.tomato_sauce: 'tomato sauce'>,
    <Topping.prosciutto: 'prosciutto'>,
    <Topping.mozzarella: 'mozzarella'>,
    <Topping.basil: 'basil'>
  )
)

Pydantic

Runtime type-checking


 

@ahultner

Pydantic

BaseModel, JSON

So what about JSON? 🧑‍💻  

from pydantic import BaseModel


class Pizza(BaseModel):
    style: str
    toppings: Tuple[Topping, ...]

Dataclass dropin replacement is great for compability

  • Pydantic BaseModel does more!
  • (de)serialisation
  • First class JSON-support

 

Disclaimer: Pydantic is primarly a parsing library

@ahultner

Pydantic

JSON (de)serialisation

So what about JSON? 🧑‍💻  

Specify arguments using kwargs when using BaseModel

Pizza(
  style="Napoli", 
  toppings=(
    Topping.tomato_sauce, 
   	Topping.prosciutto, 
   	Topping.mozzarella, 
   	Topping.basil,
  )
)
Pizza(
  style='Napoli', 
  toppings=(
    <Topping.tomato_sauce: 'tomato sauce'>,
    <Topping.prosciutto: 'prosciutto'>,
    <Topping.mozzarella: 'mozzarella'>,
    <Topping.basil: 'basil'>
  )
)
>>> _.json()
'{"style": "Napoli", "toppings": ["tomato sauce", "prosciutto", "mozzarella", "basil"]}'

We can now easily encode this object as JSON 👾

There's also built-in support for dict, pickle, immutable copy(). Pydantic will also (de)serialise subclasses. 🥒

@ahultner

Pydantic

JSON (de)serialisation

So what about JSON? 🧑‍💻  

Let's reconstruct our object from the JSON output 🏗

>>> Pizza.parse_raw('{"style": "Napoli", "toppings": ["tomato sauce", "prosciutto", "mozzarella", "basil"]}')
Pizza(
  style='Napoli', 
  toppings=(
    <Topping.tomato_sauce: 'tomato sauce'>,
    <Topping.prosciutto: 'prosciutto'>,
    <Topping.mozzarella: 'mozzarella'>,
    <Topping.basil: 'basil'>
  )
)

We'll use the parse_raw(…) function

@ahultner

Pydantic

JSON (de)serialisation

So what about JSON? 🧑‍💻  

Errors raises a validation error, these can also be represented as JSON 🚨🚫🚧

try:
    Pizza(style="Napoli", toppings=(2,))
except ValidationError as err:
    print(err.json())
[
  {
    "loc": [
      "toppings",
      0
    ],
    "msg": "value is not a valid enumeration member; permitted: 'mozzarella', 'tomato sauce', 'prosciutto', 'basil', 'rucola'",
    "type": "type_error.enum",
    "ctx": {
      "enum_values": [
        "mozzarella",
        "tomato sauce",
        "prosciutto",
        "basil",
        "rucola"
      ]
    }
  }
]

@ahultner

Pydantic

JSONSchema

JSONSchema can be exported directly from the model

 

Useful for external clients or to feed a Swagger/OpenAPI-spec 📜✅

>>> Pizza.schema()
{'title': 'Pizza',
 'type': 'object',
 'properties': {'style': {'title': 'Style', 'type': 'string'},
  'toppings': {'title': 'Toppings',
   'type': 'array',
   'items': {'enum': ['mozzarella',
     'tomato sauce',
     'prosciutto',
     'basil',
     'rucola'],
    'type': 'string'}}},
 'required': ['style', 'toppings']}

Caution: Pydantic uses the latest draft 7 of JSONSchema, this will be used in the comming OpenAPI 3.1 spec but the current 3.0.x spec uses draft 4.

@ahultner

Pydantic

Validators


 

That was the built-in validators.

But what about custom ones?

from pydantic import validator, root_validator

class BakedPizza(Pizza):
  oven_temperature: int

  # A validator looking at a single property
  @validator('style')
  def check_style(cls, style):
    house_styles = ("Napoli", "Roman", "Italian")
    if style not in house_styles:
      raise ValueError(f"""
        We only cook the following styles: {house_styles}
        Given: {style}""")
    return style
  
  # Root validators check the entire model
  @root_validator
  def check_temp(cls, values):
    style = values.get("style") 
    temp = values.get("oven_temperature")
        
    if style != "Napoli":
      # No special rules for the rest
      return values

    if 350 <= temp <= 400: 
      # Target temperature 350 - 400°C, ideally around 375°C
      return values

    raise ValueError(f"""
      Napoli style require oven_temperature in 350-400°C range
      Given: {temp}°C""")

Neapolitan pizzas, ideally around 375°C

BakedPizza with oven_temperature and rules based on house styles

@ahultner

Pydantic

Validators


 

Now let's see if we bake 👨‍🍳 some invalid pizzas ⚠️🚨

try: 
    BakedPizza(
      style="Panpizza", 
      toppings=["tomato sauce"], 
      oven_temperature=250
    )
except ValidationError as err:
    print(err)
1 validation error for BakedPizza
style
  We only cook the following styles: ('Napoli', 'Roman', 'Italian'), given: Panpizza (type=value_error)
try: 
    BakedPizza(
      style="Napoli", 
      toppings=["tomato sauce"], 
      oven_temperature=300
    )
except ValidationError as err:
    print(err)
1 validation error for BakedPizza
__root__
  Napoli pizzas require a oven_temperature in the range of 350 - 400°C, given: 300°C (type=value_error)

@ahultner

Pydantic

Validators, functions(…)


 

Gosh these runtime type checkers are rather useful, but what about functions

 

Pydantic got you covered with @validate_arguments.

 

 

 

 

Still in beta, API may change, release 2020-04-18 in version 1.5

from pydantic import validate_arguments

# Validator on function
# Ensure that we use a valid pizza when making orders
@validate_arguments
def make_order(pizza: Pizza):
    ...
try:
    make_order({
        "style":"Napoli",
        "toppings":(
          "tomato sauce", 
          "mozzarella", 
          "prosciutto", 
          "pineapple",
        ),
    })
except ValidationError as err:
    print(err)
1 validation error for MakeOrder
pizza -> toppings -> 3
  value is not a valid enumeration member; permitted: 'mozzarella', 'tomato sauce', 'prosciutto', 'basil', 'rucola' (type=type_error.enum; enum_values=[<Topping.mozzarella: 'mozzarella'>, …])

@ahultner

FastAPI

Pydantic-driven APIs

 

  • Lean micro framework similar to flask
  • Automatic OpenAPI-specs
  • Tight integration with pydantic
  • Async ASGI
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

def make_order(pizza: Pizza):
    # Business logic for making an order
    pass

def dispatch_order(pizza: BakedPizza):
    # Hand over pizza to delivery company
    pass

# Deliver a baked pizza
@app.post("/delivery/pizza")
async def deliver_pizza_order(pizza: BakedPizza):
    dispatch = dispatch_order(pizza)
    return dispatch

@app.post("/order/pizza")
async def order_pizza(pizza: Pizza):
    order = make_order(pizza)
    return order

This is everything we need to create a small API around our models.

@ahultner

That's the beginning

But there is more…

 

That's it, a quick introduction to pydantic!


But this is just the tip of the iceberg 🗻 and I want to give you a hint about what more can be done.  


I'm not going to go into detail in any of this but feel free to ask me about it in the chat, on Twitter/LinkedIn or via email 💬📨

@ahultner

And more!

Cool features worth mentioning

 

  • Post 1.0, reached this milestone about a year ago

  • Support for standard library types
  • Offer useful extra types for every day use
    • Email
    • HttpUrl (and more, stricturl for custom validation)
    • PostgresDsn
    • IPvAnyAddress (as well as IPv4Address and IPv6Address from ipaddress)
    • PositiveInt
    • PaymentCardNumber
    • PaymentCardBrand.[amex, mastercard, visa, other] checks luhn, str of digits and BIN-based lenght.
    • Constrained types (e.g. conlist, conint, etc.)
    • and more…

@ahultner

Conclusion

  • Pure python syntax
  • Better validation
  • Very useful JSON-tools for API's
  • Easy to migrate from dataclasses
  • Lots of useful features
  • More things comming
    • ​Very active development
    • Working on strict mode
  • Try it out!

@ahultner

Questions

Contact me if you have any further

questions.

 

 

Want to learn more?

Available for training, workshops and

freelance consulting.

 

Sign up for Hypothesis course

Links

@ahultner

Python Pizza Remote 2020: Pydantic – Give your python Dataclasses super powers with pydantic

By Alexander Hultnér

Python Pizza Remote 2020: Pydantic – Give your python Dataclasses super powers with pydantic

A quick talk about the fantastic pydantic library and how it makes your dataclasses better!

  • 2,199