migrating thumbnail solution to thumbor
@namngology
Software Developer @ Kogan.com
By: Nam Ngo
Agenda
- Current infrastructure and the problem
- Scalable solution? Microservices?
- More about Thumbor
- Codebase migration
- Go live, feature flags
current setup & The problem
App server
App server
App server
GlusterFS cluster
Volume
Volume
Volume
Issues
- 100,000s of products x 2-4 images x 10-20 thumb sizes
- GlusterFS + small files = :(
- Thumbnailing code lives in main codebase
- Tightly coupled to Ecommerce platform
- Cost
scalable solution? Microservice?
- Thumbnailing service
- Secure
- Extensible
- Large community & active development
- Well documented
- Storage backend: Amazon S3
- Multiple engine support: OpenCV, PIL
- Extra features (face recognition, webp) = plus
- Written in Python = plus
- Reusable across different projects
More about thumbor
http://server/unsafe/300x200/smart/www.mydomain.com/myfile.jpg
Thumbor server
size & filters
url to source image
- Loaders: where to load the thumbnail from?
- Storages: where to store the original image?
- Result storages: where to store the thumbnail?
Codebase migration - Django integration
settings.py
THUMBOR_SECURITY_KEY = 'supersecurekey'
THUMBOR_SERVER 'http://thumbor.kogan.com'
THUMBOR_MEDIA_URL = 'http://www.kogan.com/media'
pip install libthumbor
from libthumbor import CryptoURL
def get_thumbnail(image_path, **kwargs):
crypto = CryptoURL(key=THUMBOR_SECURITY_KEY)
image_url = '%s%s' % (THUMBOR_MEDIA_URL, image_path)
thumbor_url = crypto.generate(image_url=image_url, **kwargs)
return '%s/%s' % (THUMBOR_SERVER, thumbor_url)
sorl-thumbnail template tags
{% load thumbnail %}
{% thumbnail product.image "600x400" as image_file %}
<img src="{{ image_file.url }}" width="{{ image_file.width }}" />
{% endthumbnail %}
django-thumbor library
{% load thumbor_tags %}
<img src="{% thumbor_url product.image.url width=600 height=400 %}"
width="600" />
Backwards compatible templatetags
Context variable
from collections import namedtuple
from core.thumbor import get_thumbnail
Thumbnail = namedtuple('Thumbnail', 'url, width, height')
thumbnail_url = get_thumbnail(url, **options)
thumbnail = Thumbnail(thumbnail_url, **options)
thumbnail.url
thumbnail.width
thumbnail.height
roll-your-own template tags
{% load thumbor %}
{% thumbnail product.image "600x400" as image_file %}
<img src="{{ image_file.url }}" width="{{ image_file.width }}" />
{% endthumbnail %}
FEATURE FLAg
pip install django-waffle
from sorl.thumbnail.templatetags.thumbnail import ThumbnailNode
from thumbor.templatetags import ThumborNode
import waffle
@register.tag
def thumbnail(parser, token):
if waffle.flag_is_active(request, 'use_thumbor'):
return ThumborNode(parser, token)
return ThumbnailNode(parser, token)
Incremental Migration > Big Bang Migration
Kogan.com
is hiring
Q & A
Migrating thumbnail solution to Thumbor
By Nam Ngo
Migrating thumbnail solution to Thumbor
- 1,543