MERY on Docker
[2015/11/10] Fashion Tech meetup #1
Yu Yamanaka (@yuurelx)
DevOps engineer at peroli, Inc.
No.1 curation platform for women in Japan !
MERY is running on Docker
of course in production
How many Docker used?
How do you think about Docker?
Awesome!
(but have a concern about using in production...)
lower performance?
not stabilized yet?
many constraints?
You are right!!
No silver bullet.
This presentation is
our try-and-error story
about migrating to Docker architecture
Topics
- Before
- After
- Troubles
Topics
- Before
- After
- Troubles
Deploy codes & reload server
Clone repository
class ListsController < ApplicationController
require_login = [:new, :create, :update, :destroy, :click]
before_filter :noindex, only: [:search, :click, :favorites]
before_filter :noindex_after_second_page, only: [:index]
SHOW_NUMBER_OF_ITEMS_PER_PAGE_IN_PC = 50
SHOW_NUMBER_OF_ITEMS_PER_PAGE_IN_SP = 20
SHOW_NUMBER_OF_LINKED_LISTS = 5
def index
if request.smart_phone?
side_ranks
else
@ranks = DailyRank.weekly_list_rank_from_cache(5) unless ...
end
@big_lists = BigList.top_lists unless fragment_exist?(...)
@top_lists = TopList.including_lists_by_page_and_per_variables(...)Build app
$ bundle exec cap deploy
Old deployment flow
Create AMI
Manual
It took about an hour per a module...
Issues
- We cannot delivery features to users everyday
- We have risks of human error
- Who ensure sameness between AMI and recipe?
We compared some solutions, and finally chose Docker
Topics
- Before
- After
- Troubles
Concepts
One click deployment
Only merging a PR!
Pull deployment model
DockerHub
Launched by autoscaling job
EC2
EC2
Existing instances
Pull application images from the registry!
Micro Blue-Green deployment
No downtime & Easy rollback
Architecture
Autoscaling flow
Rundeck
Modern
Job Scheduler
Future
- Staging environments for each branch
- Using docker images to build dev env (on Mac)
- Deploy with some orchestration tool
(Amazon ECS, Kubernetes)
Topics
- Before
- After
- Troubles
Performance issue
Native vs Docker
(bridge net. mode)
Apache Bench
container
100%
110%
About 10% slower...
1000 reqs
(by 65 sessions)
DB
Native vs Docker
(host net.)
Apache Bench
container
100%
100%
No overhead!
DB
Sudden death of bridge networking
(From: "http://www.agilegroup.co.jp/technote/docker-network-in-bridge.html")
A bug of Docker?
We decided not to use bridge networking because could not find the cause....
Stabilization of middleware version
Launched by autoscaling job
EC2
EC2
Existing instances
haproxy 1.x
haproxy 1.y
We decided to create an base AMI with required middleware by Packer.
Handling of credentials
DockerHub
source codes without credencials
encrypted credentials with Amazon KMS
(GitHub, Docker, ...)
build an image with fetched credentials
Handling of credentials
...
# Prepare to clone Git repositories
COPY id_rsa /root/.ssh/id_rsa
RUN touch /root/.ssh/known_hosts && \
ssh-keyscan github.com > /root/.ssh/known_hosts && \
chmod -R go-rwx /root/.ssh && \
# build app
rm -f id_rsa /root/.ssh/*
...(A snippet of the Dockefile)
production:
branch: release
commands:
- sudo pip install awscli
- aws s3 cp s3://path/to/id_rsa ./id_rsa
- aws s3 cp s3://path/to/dockercfg $HOME/.dockercfg
- docker build -t example/mery_api:circleci_$CIRCLE_BUILD_NUM .
- docker push example/mery_api:circleci_$CIRCLE_BUILD_NUM
- # kick the rundeck job ...
(A snippet of the circle.yml)
Longer CI time
Each has a cache mechanism.
At default those are incompatible, but you can make that better.
VS
https://circleci.com/docs/docker#caching-docker-layers
1. Before
2. After
3. Troubles
=> You learned why we chose Docker.
=> You learned how we utilize Docker for our infrastructures.
=> You knew some troubles and solutions about Docker in production.
Conclusion
・Migrating to Docker infrastructures is not easy
・But those are running healthy also today
・Since Docker and its ecosystem has great features, they might help your issues
Let's imagine usages of Docker
in your production!
Thank you for your attention!
Yu Yamanaka (@yuurelx)
DevOps engineer at peroli, Inc.
MERY on Docker
By Yu Yamanaka
MERY on Docker
[2015/11/10] Fashion Tech meetup #1
