DJANGO STRUCTURE FOR SCALE AND LONGEVITY

Radoslav Georgiev, EuroPython 2018

Radoslav "Rado" Georgiev

HackSoft Academy

~ 10 Python / Django courses, preparing people for their first job.

What do we want to achieve?

  • Enable more productive Django teams

  • Have a more stable Django software in the long run

  • Avoid needless abstraction

  • Have a repeatable pattern while developing & debugging (that's why we use frameworks)

Where do we put business logic in Django?

How do we define "business logic"?

  • Everything related to the specific domain of the software that we are developing
     

  • Constraints & relationships in our code

Frameworks give us boxes

Models

  • Define the data model of the application

  • Integral to any Django application

  • Define relations to be used in the "business logic"

  • Models can have properties & static/class methods

  • Additional business logic can be model validation or added somewhere in save() method

Models - examples

class Course(models.Model):
    start_date = models.DateField()
    end_date = models.DateField()

    ...

    @property
    def has_started(self):
        now = get_now()

        return self.start_date <= now.date()

    @property
    def has_finished(self):
        now = get_now()

        return self.end_date <= now.date()

Models - examples

class Course(models.Model):
    start_date = models.DateField()
    end_date = models.DateField()

    ...

    def save(self, *args, **kwargs):
        self.full_clean()
        return super().save(*args, **kwargs)

    def clean(self):
        if self.start_date > self.end_date:
            raise ValidationError("...")

Models - examples

class Course(models.Model):
    ...
    def save(self, *args, **kwargs):
        self.full_clean()
        super().save(*args, **kwargs)

        weeks = self.duration_in_weeks
        start_date = self.start_date
        start_date = start_date - timedelta(days=start_date.weekday())
    
        week_instances = []
        for i in range(1, weeks + 1):
            current = Week(course=self, ....)
    
            start_date = current.end_date + timedelta(days=1)
            week_instances.append(current)
    
        Week.objects.bulk_create(week_instances)

Models should take care of the data model & relations

Business logic in models

Properties*

clean method for additional validation

save method for additional logic

Views & APIs

  • The HTTP interface for the outside world

  • Call things from the "inside"

  • Can hold business logic (As regularly shown in tutorials)

  • Will focus on APIs since there's not enough time for all of it

  • The level of abstraction can go from pretty low to really high

APIs - Examples

from snippets.models import Snippet
from snippets.serializers import SnippetSerializer
from rest_framework import generics


class SnippetList(generics.ListCreateAPIView):
    queryset = Snippet.objects.all()
    serializer_class = SnippetSerializer


class SnippetDetail(generics.RetrieveUpdateDestroyAPIView):
    queryset = Snippet.objects.all()
    serializer_class = SnippetSerializer

APIs - Examples

from rest_framework import viewsets


class SnippetViewSet(viewsets.ModelViewSet):
    queryset = Snippet.objects.all()
    serializer_class = SnippetSerializer

What's happening?

class SnippetViewSet(viewsets.ModelViewSet):
    queryset = Snippet.objects.all()
    serializer_class = SnippetSerializer
  • CreateModelMixin.create() is called

  • SnippetSerializer validation is triggered

  • SnippetSerializer.create() is called

  • If we want additional business logic, we should put it in the serializer.

Serializers are great!

Transform Python / ORM objects to JSON

Transform JSON to Python data / ORM

Should not take care of creating objects & doing additional business logic

Alternative

class CourseCreateApi(APIView):
    def post(self, request):
        serializer = CourseSerializer(data=request.data)
        serializer.is_valid(raise_exception=True)

        course = Course.objects.create(**serializer.validated_data)

        weeks = course.duration_in_weeks
        start_date = course.start_date
        start_date = start_date - timedelta(days=start_date.weekday())

        week_instances = []
        for i in range(1, weeks + 1):
            current = Week(course=course, ....)

            start_date = current.end_date + timedelta(days=1)
            week_instances.append(current)

        Week.objects.bulk_create(week_instances)
        CourseDescription.objects.create(course=course, verbose=description)

        return Response(status=status.HTTP_201_CREATED)

Business logic in APIs

  • We have a view which does the same thing as an API

  • We have a task which does the same thing as an API

  • We have a command which does the same thing as an API

  • We have internal logic which does the same thing as an API

Business logic in APIs

Not the box we are looking for

Often times we need more boxes

Putting things into existing boxes when you actually need new boxes can lead to bad design down the road.

A note on abstraction

If you are putting something in your database, use as little abstraction as possible.

If you are getting out something from your database, use as much abstraction as needed, but don't overdo it.

Existing boxes

  • Models

  • Views / APIs

  • Templates

  • Forms / Serializers

  • Tasks

We are constantly developing a style for writing Django applications.

app/services.py

  • The general unit that holds business logic together.

  • Service = A simple, type annotated function

  • Speaks the domain language of the software that we are creating

  • Can handle permissions & multiple cross-cutting concerns, such as calling other services or tasks.

  • Works mostly & mainly with models

Services - example

def create_user(
    *,
    email: str,
    name: str
) -> User:
    user = User(email=email)
    user.full_clean()
    user.save()

    create_profile(user=user, name=name)
    send_confirmation_email(user=user)

    return user
@transaction.atomic
def create_course(
    *,
    name: str,
    start_date: datetime,
    end_date: datetime,
    description: str
) -> Course:
    course = Course.objects.create(
        name=name,
        start_date=start_date,
        end_date=end_date
    )

    weeks = course.duration_in_weeks
    start_date = course.start_date - timedelta(days=start_date.weekday())
    week_instances = []

    for i in range(1, weeks + 1):
        current = Week(
            course=course,
            number=i,
            start_date=start_date,
            end_date=start_date + timedelta(days=6)
        )

        start_date = current.end_date + timedelta(days=1)
        week_instances.append(current)

    Week.objects.bulk_create(week_instances)
    CourseDescription.objects.create(course=course, verbose=description)

    return course

Every non-trivial operation, where objects are being created, should be a service.

app/selectors.py

  • They take care of business logic around fetching data from the database

  • Not always necessary (unlike services)

  • Can handle permissions, filtering, etc.

Selectors - Example

def get_users(*, fetched_by: User) -> Iterable[User]:
    user_ids = get_visible_users_for(user=fetched_by)

    query = Q(id__in=user_ids)

    return User.objects.filter(query)

Selectors vs Model Properties

If a model property starts doing queries on the model's relations, or outside them, it should be a selector.

Example - better as a selector

class Lecture(models.Model):
   ...

    @property
    def not_present_students(self):
        present_ids = self.present_students.values_list('id', flat=True)

        return self.course.students.exclude(id__in=present_ids)

APIs

  • When using services & selectors, all APIs look the same

  • There's no hidden or complex abstraction

  • We have a nice repeatable pattern for how an API should look like

APIs - Examples

class CourseListApi(SomeAuthenticationMixin, APIView):
    class OutputSerializer(serializers.ModelSerializer):
        class Meta:
            model = Course
            fields = ('id', 'name', 'start_date', 'end_date')

    def get(self, request):
        courses = get_courses()

        data = self.OutputSerializer(courses, many=True)

        return Response(data)

APIs - Examples

class CourseDetailApi(SomeAuthenticationMixin, APIView):
    class OutputSerializer(serializers.ModelSerializer):
        class Meta:
            model = Course
            fields = ('id', 'name', 'start_date', 'end_date')

    def get(self, request, course_id):
        course = get_course(id=course_id)

        data = self.OutputSerializer(course)

        return Response(data)

APIs - Examples

class CourseCreateApi(SomeAuthenticationMixin, APIView):
    class InputSerializer(serializers.Serializer):
        name = serializers.CharField()
        start_date = serializers.DateField()
        end_date = serializers.DateField()

    def post(self, request):
        serializer = self.InputSerializer(data=request.data)
        serializer.is_valid(raise_exception=True)

        create_course(**serializer.validated_data)

        return Response(status=status.HTTP_201_CREATED)

APIs - Examples

class CourseUpdateApi(SomeAuthenticationMixin, APIView):
    class InputSerializer(serializers.Serializer):
        name = serializers.CharField(required=False)
        start_date = serializers.DateField(required=False)
        end_date = serializers.DateField(required=False)

    def post(self, request, course_id):
        serializer = self.InputSerializer(data=request.data)
        serializer.is_valid(raise_exception=True)

        update_course(course_id=course_id, **serializer.validated_data)

        return Response(status=status.HTTP_200_OK)

APIs - Serializers

  • Serializers are nested inside the API.

  • If you are doing list / detail API, you can use ModelSerializer, if needed.

  • If you are doing create / update API, you can only use the normal serializer.

  • Reusing serializers should be done with great care & attention.

To avoid reusing serializers, you can use "inline serializers"

    class Serializer(serializers.Serializer):
        ...

        weeks = inline_serializer(many=True, fields={
            'id': serializers.IntegerField(),
            'number': serializers.IntegerField(),
        })
def create_serializer_class(name, fields):
    return type(name, (serializers.Serializer, ), fields)


def inline_serializer(*, fields, data=None, **kwargs):
    serializer_class = create_serializer_class(name='', fields=fields)

    if data is not None:
        return serializer_class(data=data, **kwargs)

    return serializer_class(**kwargs)

Implementation

Testing Models

  • Testing custom validations or properties

  • Can actually write tests that does not hit the database => fast tests

Testing Services

  • Heavy lifting is done here

  • Mocks are used a lot, to isolate specific parts, such as other services or tasks

Testing APIs

  • There's no general need to test APIs

  • Delegate this to integration tests, that hit the real endpoint

More on testing - check Martin Angelov's lecture -
"Proper Django Testing", 16:05 @ Lammermuir

Avoid business logic in:

Model's save method

Forms & Serializers

Views & APIs

Templates, tags and utility methods

Tasks & utility methods

Selectors & Services

Use services for create / update actions

Use selectors for get / list actions

There's more to it

We have published HackSoft's Django styleguide here:

github.com/HackSoftware/Django-Styleguide 

and we are updating it daily.

Thanks! Questions?

Made with Slides.com