Django Model
Behaviors
Advanced Patterns to Manage Model Complexity
by Kevin Stone
CTO/Founder Subblime
GH: kevinastone | TW: @kevinastone | LI: kevinastone
How do we maintain our Django
Models as our application grows in
complexity?
10s-100s of Models
+ Views, Templates, Tests...
Compositional Model Behaviors
The Compositional Model pattern allows you to manage the complexity of your models through compartmentalization of functionality into manageable components.
The Benefits of Fat Models
- Encapsulation
- Single Path
- Separation of Concerns (MVC)
Without the Maintenance Cost
- DRY
- Readability
- Reusability
- Single Responsibility
- Testability
Compositional Model Behaviors
Decompose models into core reusable mixins
Traditional
from django.db import models
from django.contrib.auth.models import User
class BlogPost(models.Model):
title = models.CharField(max_length=255)
body = models.TextField()
slug = models.SlugField()
author = models.ForeignKey(User, related_name='posts')
create_date = models.DateTimeField(auto_now_add=True)
modified_date = models.DateTimeField(auto_now=True)
publish_date = models.DateTimeField(null=True)
Decomposed into Behaviors
from django.db import models
from django.contrib.auth.models import User
class BlogPost(models.Model):
title = models.CharField(max_length=255)
body = models.TextField()
slug = models.SlugField()
author = models.ForeignKey(User, related_name='posts')
create_date = models.DateTimeField(auto_now_add=True)
modified_date = models.DateTimeField(auto_now=True)
publish_date = models.DateTimeField(null=True)
from django.db import models
from .behaviors import Authorable, Permalinkable, Timestampable, Publishable
class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
title = models.CharField(max_length=255)
body = models.TextField()
Reusable Behaviors
from django.contrib.auth.models import User
class Authorable(models.Model):
author = models.ForeignKey(User)
class Meta:
abstract = True
class Permalinkable(models.Model):
slug = models.SlugField()
class Meta:
abstract = True
...
Reusable Behaviors (continued)
...
class Publishable(models.Model):
publish_date = models.DateTimeField(null=True)
class Meta:
abstract = True
class Timestampable(models.Model):
create_date = models.DateTimeField(auto_now_add=True)
modified_date = models.DateTimeField(auto_now=True)
class Meta:
abstract = True
Models are more than just Fields
That was just common fields, but what about everything else models encapsulate?
- Properties
- Custom Methods
- Method Overloads (save(), etc...)
- Validation
- Querysets
Traditional Fat Model
class BlogPost(models.Model):
...
@property
def is_published(self):
from django.utils import timezone
return self.publish_date < timezone.now()
@models.permalink
def get_absolute_url(self):
return ('blog-post', (), {
"slug": self.slug,
})
def pre_save(self, instance, add):
from django.utils.text import slugify
if not instance.slug:
instance.slug = slugify(self.title)
Behaviors with Methods
Maintains separation of concerns
class Permalinkable(models.Model):
slug = models.SlugField()
class Meta:
abstract = True
def get_url_kwargs(self, **kwargs):
kwargs.update(getattr(self, 'url_kwargs', {}))
return kwargs
@models.permalink
def get_absolute_url(self):
url_kwargs = self.get_url_kwargs(slug=self.slug)
return (self.url_name, (), url_kwargs)
def pre_save(self, instance, add):
from django.utils.text import slugify
if not instance.slug:
instance.slug = slugify(self.slug_source)
Behaviors with Methods (continued)
Maintains separation of concerns
class Publishable(models.Model):
publish_date = models.DateTimeField(null=True)
class Meta:
abstract = True
objects = PassThroughManager.for_queryset_class(PublishableQuerySet)()
def publish_on(self, date=None):
from django.utils import timezone
if not date:
date = timezone.now()
self.publish_date = date
self.save()
@property
def is_published(self):
from django.utils import timezone
return self.publish_date < timezone.now()
Behavior Based Model
from django.db import models
from .behaviors import Authorable, Permalinkable, Timestampable, Publishable
class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
title = models.CharField(max_length=255)
body = models.TextField()
url_name = "blog-post"
@property
def slug_source(self):
return self.title
Naming Tips
Use "<verb>-able" naming pattern for behaviors
- Readily identifiable as a compositional Mixin and not a standalone Model.
- The word Mixin is already overly-generic...
- (even though the naming gets weird quickly e.g.
OptionallyGenericRelateable)
Custom Queryset Chaining
We all know to chain queryset methods, but what about adding custom manager methods?
Let's Find Posts from a Given Author (username1) that are Published (publish_date in the past)
Typical Tutorial Queries
No Encapsulation
from django.utils import timezone
from .models import BlogPost
>>> BlogPost.objects.filter(author__username='username1') \
.filter(publish_date__lte=timezone.now())
Custom Managers
Let's create methods on a custom Manager to handle the past-publication date and author filters
class BlogPostManager(models.Manager):
def published(self):
from django.utils import timezone
return self.filter(publish_date__lte=timezone.now())
def authored_by(self, author):
return self.filter(author__username=author)
class BlogPost(models.Model):
...
objects = BlogPostManager()
>>> published_posts = BlogPost.objects.published() >>> posts_by_author = BlockPost.objects.authored_by('username1')
Custom Manager
But what about chaining our filters?
>>> BlogPost.objects.authored_by('username1').published() AttributeError: 'QuerySet' object has no attribute 'published'
>>> type(Blogpost.objects.authored_by('username1')) <class 'django.db.models.query.QuerySet'>
Solution: Custom Querysets
Combined with PassthroughManager from django-model-utils
from model_utils.managers import PassThroughManager class PublishableQuerySet(models.query.QuerySet): def published(self): from django.utils import timezone return self.filter(publish_date__lte=timezone.now()) class AuthorableQuerySet(models.query.QuerySet): def authored_by(self, author): return self.filter(author__username=author)
class BlogPostQuerySet(AuthorableQuerySet, PublishableQuerySet):
pass
class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
...
objects = PassThroughManager.for_queryset_class(BlogPostQuerySet)()
Chainable Custom Querysets
Now you can chain custom methods inherited from multiple behaviors
>>> author_public_posts = BlogPost.objects.authored_by('username1').published()
>>> type(Blogpost.objects.authored_by('username1'))
<class 'example.queryset.BlogPostQuerySet'>
Encapsulate the Business Logic
What's more legible and maintainable?
BlogPost.objects.filter(author__username='username1').filter(publish_date__lte=timezone.now())
-or-
BlogPost.objects.authored_by('username1').published()
Testing Behaviors
Create matching Behavior tests to validate our models
Same Benefits as for Models
- DRY
- Readability
- Reusability
- Single Responsibility
Existing Unit Test Example
from django.test import TestCase
from .models import BlogPost
class BlogPostTestCase(TestCase):
def test_published_blogpost(self):
from django.utils import timezone
blogpost = BlogPost.objects.create(publish_date=timezone.now())
self.assertTrue(blogpost.is_published)
self.assertIn(blogpost, BlogPost.objects.published())
Behavior Test Mixin
class BehaviorTestCaseMixin(object):
def get_model(self):
return getattr(self, 'model')
def create_instance(self, **kwargs):
raise NotImplementedError("Implement me")
class PublishableTests(BehaviorTestCaseMixin):
def test_published_blogpost(self):
from django.utils import timezone
obj = self.create_instance(publish_date=timezone.now())
self.assertTrue(obj.is_published)
self.assertIn(obj, self.model.objects.published())
Behavior Based Unit Tests
from django.test import TestCase from .models import BlogPost
from .behaviors.tests import PublishableTests class BlogPostTestCase(PublishableTests, TestCase): model = BlogPost def create_instance(self, **kwargs): return BlogPost.objects.create(**kwargs)
Complete Test Case
class BlogPostTestCase(PublishableTests, AuthorableTests, PermalinkableTests, TimestampableTests, TestCase):
model = BlogPost
def create_instance(self, **kwargs):
return BlogPost.objects.create(**kwargs)
def test_blog_specific_functionality(self):
...
Additional Model Testing Tips
- Use Factory Boy for creating test instances/fixtures
- Use Inherited TestCases to validate different scenarios
class StaffBlogPostTestCase(PublishableTests, AuthorableTests, PermalinkableTests, TimestampableTests, BaseBlogPostTestCase):
det setUp(self):
self.user = StaffUser()
class AuthorizedUserBlogPostTestCase(PublishableTests, AuthorableTests, PermalinkableTests, TimestampableTests, BaseBlogPostTestCase):
det setUp(self):
self.user = AuthorizedUser()
(Same behavior expected for Staff or Authorized User)
Reusability
We eventually build up a Library of Behaviors
- Permalinkable
- Publishable
- Authorable
- Timestampable
Re-usable both across our own Apps and shareable through the Community
More Examples
- Moderatable - BooleanField('approved')
- Scheduleable - (start_date and end_date with range queries)
- GenericRelatable (the triplet of content_type, object_id and GenericForeignKey)
- Orderable - PositiveSmallIntegerField('position')
Reusability Example
from django.db import models
from .behaviors import Authorable, Permalinkable, Timestampable, Publishable
class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
title = models.CharField(max_length=255)
body = models.TextField()
url_name = "blog-post"
@property
def slug_source(self):
return self.title
class BlogComment(Authorable, Permalinkable, Timestampable, models.Model):
post = models.ForeignKey(BlogPost, related_name='comments')
subject = models.CharField(max_length=255)
body = models.TextField()
url_name = 'blog-comment'
def get_url_kwargs(self, **kwargs):
return super(BlogComment, self).get_url_kwargs(post_slug=self.post.slug, **kwargs)
@property
def slug_source(self):
return self.subject
Reusability Enforces Standards
- Common Idioms, esp. in Templates and Template Tags
- Permissions and Security
- Testability
It's ultimately about Separation of Concerns
- Keep the business logic encapsulated in the behavior
- Standardize the interface to shared behaviors for consistency
- Authorable always means obj.author, not obj.user or obj.owner
Recommended App Layout
- querysets.py
- behaviors.py (uses querysets)
- models.py (composition of querysets and behaviors)
- factories.py (uses models)
- tests.py (uses all, split this into a module for larger apps)
I usually have a common app that has the shared behaviors, model and behavior test mixins with no dependencies on other apps.
Limitations/Pitfalls
Basically the challenges of Django Model Inheritance
Leaky Abstractions
- Meta Options don't implicitly inherit (ordering, etc)
- Manager vs Queryset vs Model (some duplication of logic)
- ModelField options (toggling default=True vs default=False)
You often need to handle the composition yourself
(such as merging custom QuerySet classes)
(or combining Meta Options)
3rd Party Helpers
Don't Re-invent the Wheel
- Django Extensions (UUIDField, AutoSlugField, etc)
- Django Model Utils (already mentioned)
- Filters (django-filter)
Test Helpers
- Factories (factory boy)
- Mocking (mock)
Questions?
The End
Example code available at
About the Author:
Kevin Stone is the CTO and Founder of Subblime
Interested in working on these challenges? Subblime is hiring
Django Model Behaviors
By Kevin Stone
Django Model Behaviors
Exploring reusable model behaviors as a pattern to manage complexity in the Django model layer.
- 13,425