Django Model

Behaviors


Advanced Patterns to Manage Model Complexity



by Kevin Stone
CTO/Founder Subblime

GH: kevinastone | TW: @kevinastone | LI: kevinastone



How do we maintain our Django 

Models as our application grows in 

complexity?



10s-100s of Models

+ Views, Templates, Tests...

Compositional Model Behaviors

The Compositional Model pattern allows you to manage the complexity of your models through compartmentalization of functionality into manageable components.


The Benefits of Fat Models

  • Encapsulation
  • Single Path
  • Separation of Concerns (MVC)


Without the Maintenance Cost

  • DRY
  • Readability
  • Reusability
  • Single Responsibility
  • Testability

Compositional Model Behaviors


Decompose models into core reusable mixins


Traditional


from django.db import models
from django.contrib.auth.models import User


class BlogPost(models.Model):
    title = models.CharField(max_length=255)
    body = models.TextField()
    slug = models.SlugField()
    author = models.ForeignKey(User, related_name='posts')
    create_date = models.DateTimeField(auto_now_add=True)
    modified_date = models.DateTimeField(auto_now=True)
    publish_date = models.DateTimeField(null=True)

Decomposed into Behaviors



from django.db import models
from django.contrib.auth.models import User


class BlogPost(models.Model):
    title = models.CharField(max_length=255)
    body = models.TextField()
    slug = models.SlugField()
    author = models.ForeignKey(User, related_name='posts')
    create_date = models.DateTimeField(auto_now_add=True)
    modified_date = models.DateTimeField(auto_now=True)
    publish_date = models.DateTimeField(null=True)


from django.db import models
from .behaviors import Authorable, Permalinkable, Timestampable, Publishable


class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
    title = models.CharField(max_length=255)
    body = models.TextField()

Reusable Behaviors



from django.contrib.auth.models import User


class Authorable(models.Model):
    author = models.ForeignKey(User)
    
    class Meta:
        abstract = True


class Permalinkable(models.Model):
    slug = models.SlugField()    
    
    class Meta:
        abstract = True

...

Reusable Behaviors (continued)



...

class Publishable(models.Model):
    publish_date = models.DateTimeField(null=True)
    
    class Meta:
        abstract = True


class Timestampable(models.Model):
    create_date = models.DateTimeField(auto_now_add=True)
    modified_date = models.DateTimeField(auto_now=True)
    
    class Meta:
        abstract = True

Models are more than just Fields


That was just common fields, but what about everything else models encapsulate?

  • Properties
  • Custom Methods
  • Method Overloads (save(), etc...)
  • Validation
  • Querysets

Traditional Fat Model



class BlogPost(models.Model):
    ...

    @property
    def is_published(self):
        from django.utils import timezone
        return self.publish_date < timezone.now()

    @models.permalink
    def get_absolute_url(self):
        return ('blog-post', (), {
            "slug": self.slug,
        })

    def pre_save(self, instance, add):
        from django.utils.text import slugify
        if not instance.slug:
            instance.slug = slugify(self.title)

Behaviors with Methods


Maintains separation of concerns

class Permalinkable(models.Model):
    slug = models.SlugField()
    
    class Meta:
        abstract = True
    
    def get_url_kwargs(self, **kwargs):
        kwargs.update(getattr(self, 'url_kwargs', {}))
        return kwargs
    
    @models.permalink
    def get_absolute_url(self):
        url_kwargs = self.get_url_kwargs(slug=self.slug)        
        return (self.url_name, (), url_kwargs)
    
    def pre_save(self, instance, add):
        from django.utils.text import slugify
        if not instance.slug:
            instance.slug = slugify(self.slug_source)

Behaviors with Methods (continued)


Maintains separation of concerns


class Publishable(models.Model):
    publish_date = models.DateTimeField(null=True)
    
    class Meta:
        abstract = True
    
    objects = PassThroughManager.for_queryset_class(PublishableQuerySet)()

    def publish_on(self, date=None):
        from django.utils import timezone
        if not date:
            date = timezone.now()
        self.publish_date = date
        self.save()

    @property
    def is_published(self):
        from django.utils import timezone
        return self.publish_date < timezone.now()

Behavior Based Model



from django.db import models
from .behaviors import Authorable, Permalinkable, Timestampable, Publishable


class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
    title = models.CharField(max_length=255)
    body = models.TextField()
    
    url_name = "blog-post"

    @property
    def slug_source(self):
        return self.title

Naming Tips


Use "<verb>-able" naming pattern for behaviors

  • Readily identifiable as a compositional Mixin and not a standalone Model.
  • The word Mixin is already overly-generic...
  • (even though the naming gets weird quickly e.g.  OptionallyGenericRelateable)

Custom Queryset Chaining


We all know to chain queryset methods, but what about adding custom manager methods?

Let's Find Posts from a Given Author (username1) that are Published (publish_date in the past)

Typical Tutorial Queries


No Encapsulation

from django.utils import timezone
from .models import BlogPost

>>> BlogPost.objects.filter(author__username='username1') \
.filter(publish_date__lte=timezone.now())

Custom Managers


Let's create methods on a custom Manager to handle the past-publication date and author filters


class BlogPostManager(models.Manager):
    
    def published(self):
        from django.utils import timezone
        return self.filter(publish_date__lte=timezone.now())
    
    def authored_by(self, author):
        return self.filter(author__username=author)


class BlogPost(models.Model):
    ...
    
    objects = BlogPostManager()

>>> published_posts = BlogPost.objects.published() >>> posts_by_author = BlockPost.objects.authored_by('username1')

Custom Manager


But what about chaining our filters?


>>> BlogPost.objects.authored_by('username1').published()
AttributeError: 'QuerySet' object has no attribute 'published'

>>> type(Blogpost.objects.authored_by('username1')) <class 'django.db.models.query.QuerySet'>

Solution: Custom Querysets


Combined with PassthroughManager from django-model-utils


from model_utils.managers import PassThroughManager class PublishableQuerySet(models.query.QuerySet): def published(self): from django.utils import timezone return self.filter(publish_date__lte=timezone.now()) class AuthorableQuerySet(models.query.QuerySet): def authored_by(self, author): return self.filter(author__username=author)

class BlogPostQuerySet(AuthorableQuerySet, PublishableQuerySet):
    pass


class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
    ...
    
    objects = PassThroughManager.for_queryset_class(BlogPostQuerySet)()

Chainable Custom Querysets


Now you can chain custom methods inherited from multiple behaviors


>>> author_public_posts = BlogPost.objects.authored_by('username1').published()

>>> type(Blogpost.objects.authored_by('username1'))
<class 'example.queryset.BlogPostQuerySet'>

Encapsulate the Business Logic


What's more legible and maintainable?


   BlogPost.objects.filter(author__username='username1').filter(publish_date__lte=timezone.now())
-or-

    BlogPost.objects.authored_by('username1').published()

Testing Behaviors


Create matching Behavior tests to validate our models


Same Benefits as for Models
  • DRY
  • Readability
  • Reusability
  • Single Responsibility

Existing Unit Test Example



from django.test import TestCase

from .models import BlogPost


class BlogPostTestCase(TestCase):
    def test_published_blogpost(self):
        from django.utils import timezone
        blogpost = BlogPost.objects.create(publish_date=timezone.now())
        self.assertTrue(blogpost.is_published)
        self.assertIn(blogpost, BlogPost.objects.published())

Behavior Test Mixin



class BehaviorTestCaseMixin(object):
    def get_model(self):
            return getattr(self, 'model')
    
    def create_instance(self, **kwargs):
        raise NotImplementedError("Implement me")


class PublishableTests(BehaviorTestCaseMixin):
    def test_published_blogpost(self):
        from django.utils import timezone
        obj = self.create_instance(publish_date=timezone.now())
        self.assertTrue(obj.is_published)
        self.assertIn(obj, self.model.objects.published())

Behavior Based Unit Tests



from django.test import TestCase

from .models import BlogPostfrom .behaviors.tests import PublishableTests


class BlogPostTestCase(PublishableTests, TestCase):
    model = BlogPost
    
    def create_instance(self, **kwargs):
        return BlogPost.objects.create(**kwargs)

Complete Test Case



class BlogPostTestCase(PublishableTests, AuthorableTests, PermalinkableTests, TimestampableTests, TestCase):
    model = BlogPost
    
    def create_instance(self, **kwargs):
        return BlogPost.objects.create(**kwargs)

    def test_blog_specific_functionality(self):
        ...

Additional Model Testing Tips


  • Use Inherited TestCases to validate different scenarios



class StaffBlogPostTestCase(PublishableTests, AuthorableTests, PermalinkableTests, TimestampableTests, BaseBlogPostTestCase):
    det setUp(self):
        self.user = StaffUser()

class AuthorizedUserBlogPostTestCase(PublishableTests, AuthorableTests, PermalinkableTests, TimestampableTests, BaseBlogPostTestCase):
    det setUp(self):
        self.user = AuthorizedUser()

(Same behavior expected for Staff or Authorized User)

Reusability


We eventually build up a Library of Behaviors

  • Permalinkable
  • Publishable
  • Authorable
  • Timestampable

Re-usable both across our own Apps and shareable through the Community

More Examples
  • Moderatable - BooleanField('approved')
  • Scheduleable - (start_date and end_date with range queries)
  • GenericRelatable (the triplet of content_type, object_id and GenericForeignKey)
  • Orderable - PositiveSmallIntegerField('position')

Reusability Example

from django.db import models
from .behaviors import Authorable, Permalinkable, Timestampable, Publishable


class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
    title = models.CharField(max_length=255)
    body = models.TextField()
    
    url_name = "blog-post"

    @property
    def slug_source(self):
        return self.title

class BlogComment(Authorable, Permalinkable, Timestampable, models.Model):
    post = models.ForeignKey(BlogPost, related_name='comments')
    subject = models.CharField(max_length=255)
    body = models.TextField()

    url_name = 'blog-comment'
    
    def get_url_kwargs(self, **kwargs):
        return super(BlogComment, self).get_url_kwargs(post_slug=self.post.slug, **kwargs)
    
    @property
    def slug_source(self):
        return self.subject

Reusability Enforces Standards


  • Common Idioms, esp. in Templates and Template Tags
  • Permissions and Security
  • Testability

It's ultimately about Separation of Concerns

  • Keep the business logic encapsulated in the behavior
  • Standardize the interface to shared behaviors for consistency
  • Authorable always means obj.author, not obj.user or obj.owner

Recommended App Layout


  • querysets.py
  
  • behaviors.py (uses querysets)
  • models.py (composition of querysets and behaviors)
  • factories.py (uses models)
  • tests.py (uses all, split this into a module for larger apps)


I usually have a common app that has the shared behaviors, model and behavior test mixins with no dependencies on other apps.

Limitations/Pitfalls


Basically the challenges of Django Model Inheritance

Leaky Abstractions

  • Meta Options don't implicitly inherit (ordering, etc)
  • Manager vs Queryset vs Model (some duplication of logic)
  • ModelField options (toggling default=True vs default=False)


You often need to handle the composition yourself
(such as merging custom QuerySet classes)
(or combining Meta Options)

3rd Party Helpers


Don't Re-invent the Wheel



Test Helpers




Questions?



The End





About the Author:

Kevin Stone is the CTO and Founder of Subblime

Interested in working on these challenges?  Subblime is hiring

Django Model Behaviors

By Kevin Stone

Django Model Behaviors

Exploring reusable model behaviors as a pattern to manage complexity in the Django model layer.

  • 13,126