about this talk

  • general information about the conference
  • will go briefly through some of the topics
  • links will be provided

the history

  • the first edition took place in Prague in 2009
  • in 2013 it took place in Warsaw
  • this year it was Copenhagen

in numbers

  • 3 conference days
  • 30 talks, 10 workshops
    • lightning talks
    • network drinks
  • 2 sprint days
  • 400 attendees

about the conference

  • no simultaneous talks
  • code of conduct
  • photo policy
  • the ++ rule
  • the pacman rule

is it worth it ?

is it worth it ?

  • travel
  • meet active community members
  • learn about the latest developments
  • attend the sprints and contribute
  • relaxed and welcoming atmosphere

is it worth it ?

  • travel
  • meet active community members
  • learn about the latest developments
  • attend the sprints and contribute
  • relaxed and welcoming atmosphere
  • not many deeply technical talks
  • not all talks directly related to django
  • a lot of talks about the django community
  • all the talks are streamed and available on youtube

...but keep in mind that

serverless

or Functions as a Service (FaaS)

Serverless computing (or serverless for short), is an execution model where the cloud provider is responsible for executing a piece of code by dynamically allocating the resources. And only charging for the amount of resources used to run the code. The code is typically run inside stateless containers that can be triggered by a variety of events including http requests, database events, queuing services, monitoring alerts, file uploads, scheduled events (cron jobs), etc. The code that is sent to the cloud provider for execution is usually in the form of a function. Hence serverless is sometimes referred to as “Functions as a Service” or “FaaS”.

serverless

or Functions as a Service (FaaS)

  • Complete abstraction of servers away from the developer
  • Billing based on consumption and executions, not server instance sizes
  • Services that are event-driven and instantaneously scalable

basic premises:

  • AWS Lambda
  • Azure Functions
  • Google Functions

well known providers:

serverless

zappa

Deploy your WSGI apps on AWS Lambda

 

With Zappa, each request is given its own virtual HTTP "server" by Amazon API Gateway. AWS handles the horizontal scaling automatically, so no requests ever time out. After your app returns, the "server" dies.

serverless

zappa

  • No more tedious web server configuration!
  • No more paying for 24/7 server uptime!
  • No more worrying about load balancing / scalability!
  • No more worrying about web server security!

serverless

zappa up and running

async

  • Python is at a big crossroads. Python 3.5 introduced the async/await keywords

 

  • the async model provides high throughput services, non-blocking HTTP requests.

 

  • only async functions can call other async functions

async

wsgi vs asgi

WSGI is an inherently thread-concurrency interface, so it cannot do any async context switching inside

 

ASGI has an async interface, and includes more than just HTTP request/responses, and is very much more general purpose and highly adabtable.

async

asgi servers

Daphne, Hypercorn, and Uvicorn

asgi frameworks

Starlette, Responder, FastAPI

async

asgi server benefits

  • High raw throughput
  • WebSockets and other real time communications
  • Server Sent Events
  • HTTP/2 server push
  • Background tasks
  • Clock and timer tasks

async

starlette - hello world

from starlette.applications import Starlette
from starlette.responses import JSONResponse
import uvicorn

app = Starlette(debug=True)

@app.route('/')
async def homepage(request):
    return JSONResponse({'hello': 'world'})

if __name__ == '__main__':
    uvicorn.run(app, host='0.0.0.0', port=8000)

async

starlette - db request

@app.route("/notes", methods=["GET"])
async def list_notes(request):
    query = notes.select()
    results = await database.fetch_all(query)
    content = [
        {
            "text": result["text"],
            "completed": result["completed"]
        }
        for result in results
    ]
    return JSONResponse(content)

async

starlette

  • uvicorn is an ASGI web server
  • starlette is an ASGI web framework or toolkit
  • databases is an async database interface
  • orm is an async ORM
  • typesystem provides type validation
  • requests-async can handle any outgoing requests

async

django async roadmap

https://www.aeracode.org/2018/06/04/django-async-roadmap/

  • Django 3.0: Rewrite the internal request handling stack to be entirely asynchronous, add async middleware, forms, caching, sessions, auth. Start deprecation process for any APIs that are becoming async-only.
  • Django 3.1: Continue improving async support, potential async templating changes
  • Django 3.2: Finish deprecation process and have a mostly-async Django.

logging

logging

  • treat logs as events
  • add structure and context
  • trace logs across multiple services
  • format as json (for ELK or graylog) 
import logging
logger = logging.getLogger(__name__)
logger.error(f'Failed to perform operation on object #{object.id} for user #{user.id}')

what logging usually looks like:

ideas:

logging

structlog

import structlog
logger = structlog.getLogger()
logger.error('transaction_failed', 
    user=user.id, target_user=target_user.id, transaction=transaction.id)

2018-04-20 16:20.13 transaction_failed    user=324 target_user=134 transaction=150
  • easier to test for occurence
  • message less likely to change (log based alerts are more reliable)

logging

structlog

logger.bind(user=user.id, target_user=target_user.id)
...
logger.bind(transaction=transaction.id)
...
logger.info('transaction_initiated')
2018-04-20 16:20.13 transaction_initiated    user=324 target_user=134 transaction=150
...
logger.error('transaction_failed')
2018-04-20 16:21.15 transaction_failed    user=324 target_user=134 transaction=150
  • bind/rebind loggers to provided context for subsequent logs

logging

trace logs across multiple services

import uuid, structlog

logger = structlog.get_logger('structlog')

def structlog_middleware(get_response):  # Use as first middleware
    def _inner(request):
        request.trace_id = request.headers.get('x_trace_id') or str(uuid.uuid4)
        logger.bind(trace_id=request.trace_id)
        return get_response(request)
    return _inner

def structlog_user_middleware(get_response):  # Use after authentication middleware
    def _inner(request):
        if request.user.is_authenticated:
            logger.bind(logger, user.id)
        return get_response(request)
    return _inner

logging

https://gitlab.com/MarkusH/django-structlog/tree/master/bank/bank

django orm

django orm

subqueries django >= 1.11

>>> from django.db.models import OuterRef, Subquery
>>> newest = Comment.objects.filter(post=OuterRef('pk')).order_by('-created_at')
>>> Post.objects.annotate(newest_commenter_email=Subquery(newest.values('email')[:1]))

SELECT "post"."id", (
    SELECT U0."email"
    FROM "comment" U0
    WHERE U0."post_id" = ("post"."id")
    ORDER BY U0."created_at" DESC LIMIT 1
) AS "newest_commenter_email" FROM "post"

django orm

queryset.explain django >= 2.1

>>> print(Blog.objects.filter(title='My Blog').explain(verbose=True))
Seq Scan on public.blog  (cost=0.00..35.50 rows=10 width=12) (actual time=0.004..0.004 rows=10 loops=1)
  Output: id, title
  Filter: (blog.title = 'My Blog'::bpchar)
Planning time: 0.064 ms
Execution time: 0.058 ms

django orm

UniqueConstraint django >= 2.2

class Order(Model):
    ...
    class Meta:
        constraints = [
            UniqueConstraint(
                name='limit_pending_orders',
                fields=['customer', 'is_shipped'],
                condition=Q(is_shipped=False)
            )
        ]

django orm

CheckConstraint django >= 2.2

class MontlyBudget(Model):
    class Meta:
        constraints = [
            CheckConstraint(
                check=Q(month__in=range(1, 13)),
                name='check_valid_month',
            )
        ]
]

django orm

bulk_update django >= 2.2

>>> objs = [
...    Entry.objects.create(headline='Entry 1'),
...    Entry.objects.create(headline='Entry 2'),
... ]
>>> objs[0].headline = 'This is entry 1'
>>> objs[1].headline = 'This is entry 2'
>>> Entry.objects.bulk_update(objs, ['headline'])

django orm

partial index django >= 2.2

class Article(models.Model):
    title = models.CharField(max_length=1000)
    upvotes = models.IntegerField()

    class Meta:
        indexes = [
            models.Index(
                fields=['upvotes'],
                name='upvotes_idx',
                condition=Q(upvotes__gt=100)
            )
        ]

django orm

windows functions django >= 2.0

from django.db.models import Avg, F, Window
from django.db.models.functions import ExtractYear

# annotate each movie with the average rating 
# for the movies by the same studio in the same genre 

Movie.objects.annotate(
    avg_rating=Window(
    expression=Avg('rating'),
    partition_by=[F('studio'), F('genre')],
    order_by=ExtractYear('released').asc(),
    ),
)

security headers

security headers

X-XSS-Protection

In Django: set SECURE_BROWSER_XSS_FILTER = True and include the SecurityMiddleware in your middlewares.

stops pages from loading when they detect reflected cross-site scripting

security headers

Strict-Transport-Security

In Django: set SECURE_HSTS_SECONDS to and include SecurityMiddleware.

 

Note: If you break your HTTPS after having set this flag, any browser that has seen is will not be able to see your website again.

Lets a web site tell browsers that it should only be accessed using HTTPS, instead of using HTTP.

security headers

X-Frame-Options

In Django: X_FRAME_OPTIONS = 'DENY' (or use whitelisting), and include the XFrameOptionsMiddleware.

Can be used to indicate whether or not a browser should be allowed to render a page in an inframe

security headers

Content-Security-Policy

In Django: Mozilla provides django-csp, add CSPMiddleware, and then set a couple of values, e.g.CSP_DEFAULT_SRC = 'self'.

img-src *;  # images from anywhere
script-src userscripts.example.com  # scripts only from here

security headers

Feature-Policy

New and not yet supported in all browsers – this header lets you disable browser feature, like autoplay, geolocation, camera, etc.

 

In Django: Install django-feature-policy, add FeaturePolicyMiddleware and set FEATURE_POLICY.

security headers

https://securityheaders.com/

tools

tools

  • black
    • opinionated python formatter
    • endorsed by many open source projects
    • strict subset of PEP 8

tools

 

  • pipenv
    • Automatically finds your project home, recursively, by looking for a Pipfile.
    • Automatically creates a virtualenv in a standard location
    • Automatically adds/removes packages to a Pipfile when they are un/installed
    • Automatically loads .env files, if they exist.
    poetry
    • alternative to pipenv

tools

 

  • graphene
    • python GraphQL framework
    • with django integration
  • strawberry-graphql (alternative to graphene)

tools

 

  • django-migration-linter
    • Detect backward incompatible migrations

Thank you

  • https://2019.djangocon.eu/
  • https://www.flickr.com/photos/djangocon/
  • https://www.youtube.com/channel/UCr7bPKCES6rP1UbX0dTVv-Q

DjangoCon Europe 2019

By zqzak

Made with Slides.com

DjangoCon Europe 2019

  • 301

More from zqzak