Damien Metzler
dmetzler@nuxeo.com
https://github.com/dmetzler/
Thierry Delprat
tdelprat@nuxeo.com
https://github.com/tiry/
This is our technical adventure
This is our story but, hopefully, this should also make sense for
2012/2013
2014
now
initial idea and first tentative
first viable version
current development
and future architecture
Multi-Tenants, Cloud & Application Factory
Nuxeo
we provide a Platform that developers can use to
build highly customized Content Applications
we provide components, and the tools to assemble
everything we do is open source
our customers are people building software
in-house -
software vendors -
SaaS
Repository
Services
Workflows, Conversions, Diff, Notifications, Activity ...
Project Management
Digital Assets Management
Case Management
Collaboration
Custom deployment
Industrialization
Customer/Tenant 1
Customer/Tenant 2
Styles
Screen Layouts
Meta-Data
Workflows
Business Logics
Connectors
Styles
Screen Layouts
Meta-Data
Workflows
Business Logics
Connectors
Document Store
Security
Life Cycle
Indexing
Versioning
all clients share the same application
application manages data and configuration partitionning
Shallow isolation
Monolithic
Not even simple
Can not leverage OSGi / Extension Point model
Not "Cloud Native approach"
rely on infrastructure to provide tenants isolation
application does not need to be impacted
Flexible
Unlimited Customization
Full Isolation & Quotas
Create "on demand" application for each customer
Create "on demand" application for each customer
Create "on demand" application for each customer
Create "on demand" application for each customer
Create "on demand" application for each customer
Let's do this ...
... for our own use cases !
We build a platform and customization tools
We want people to be able to taste the full experience
Online demo site
with full per-tenant customization
Early testing with CloudFoundry / OpenShift
Go with Deployment Automation on IaaS
Prospects do not pay
Prospects want to access all features
We want them to have a great experience
free customers with high expectations !
Cost is a major concern
Optimize hosting infrastructure
EC2 Spot instances
Start/Stop instances when needed
Leverage AWS Services
A lot of moving parts
Slow
works but ...
DEAD END ?
We need lighter foot print
We need faster startup
Some developers have started playing
with lightweight containers because
VMs are too fat & slow !
and Docker !
the developers
[dev] Cool new stuff on cloud related techs
...
http://www.docker.io/ :
a command line tool for launching and managing
arbitrary processes using LXC.
Open Source project contributed by the dotcloud guys.
...
03/2013
Build the Application Factory on Docker
First target is to power Nuxeo Trials
need a smooth UX
but also build a generic infrastructure
publish work as opensource
Assign a team of java developers
not system-administrators
Expect results in 3 months !
rebuild / refactor every 3 months
Build Your Own Application
Build Your Own Application
Select
target Platform
Build Your Own Application
Pick additional components
Build Your Own Application
Build Your Own Application
Build new components
Build Your Own Application
Select custom Components
Build Your Own Application
Define Application Model
Build Your Own Application
Build Your Own Application
Choose deployment
environment
Build Your Own Application
Deploy & Run !
Make your own container recipe
Available to the web user
Build Your Own Application
Docker containers !
Leverage AWS infrastructure
Passivation
Dual state orchestration
Use a distributed registry
Containers infrastructure:
Docker + CoreOS
Scheduler:
Fleet
Distributed registry:
etcd
Monitoring:
DataDog
The Containers:
Nuxeo Servers
Manager.io (Nuxeo +AngularJS)
Passivator (Go Service)
ArkenCtl (Go Cmd)
Dynamic reverse proxy Gogeta
(1 per tenant)
Deploy new tenant
Register new tenant
Deploy more tenants
Route request to Customer X
Route request to Customer X
started
1000+ instances/month managed on 4 EC2 VMs (m3.2xlarge)
Production hosting for some Nuxeo based applications
Eventually stable
Docker and Go are really great
The vision is good !
All Good ?
Almost !
Lots of moving parts + Lots of young solutions
Experience a new type of failure every day !
Lot of boring glue code
Storage management is an issue
Troubleshooting is tricky
Command line tools
Still too scary for a customer
the system is complex to setup
Leverage our experience and Docker evolutions
Docker ecosystem evolved
Kubernetes, Swarm, Compose, Rancher ...
We believe in the initial vision is good
Customers starts to like the idea of Containers
We have learned a lot from Nuxeo.io v1
Time for a reboot
Cluster configurations
1 node, 2 nodes, 7 nodes ...
Production hosting
take over Nuxeo-Cloud
Customer compliant
avoid or hide and package
the glue code
X
Remove as much code as we can
Networking in Docker / Rancher / Kubernetes
s/port-mapping/SDN/g
s/Ansible/Swarm/g
s/Scripting/Compose/g
Clustering in Docker-Swarm / Rancher / Kubernetes
Stack templating in Docker-Compose / Rancher / Kubernetes
Free from the Shell !
less glue code to write / debug
focus on application level
Volumes in Docker
storage can now be provisionned as a container
containers can now be statefull
Can provision Storage nodes exactly as Processing nodes
Streamline architecture: everything is container
all tenant resources are provisioned the same way
Nice high level REST API
abstraction on CaaS provider
Swarm/Kubernetes/Cattle
Administration UI
Close to Docker
Application Management
Provide additional features
Volume plugins :
convoy: NFS / GlusterFS
Rancher & Docker
do the heavy lifting
Deploying Arken is easy
One unique API
to deploy new tenants
Application Templating
Package Selection : Wizard + Config + Docker File
Deployment template : Compose + Rancher
Passivation Management
Passivation aware Routing
State management
API & Adapter
Independent
Go Based
OpenSource
Application Level Multi-Tenancy
Container Level Multi-Tenancy - Nuxeo.io v1
Container Level Multi-Tenancy - Nuxeo.io v2
Tenants sharing an Application
Tenants sharing Infrastructure
+ dedicated applications
processing isolation : docker containers
data isolation : docker and volumes
application configuration: Rancher + Arken + Nuxeo
Application Factory: Rancher + Arken
Stack of OSGi Services = MicroServices
Break deployment by tenants
Break deployment by containers
Deploy different service stacks Different containers
Converters, Async Processing, IDE , Account management ...
Leverage Arken architecture
Introduce new granularity level
Thank You !
http://www.nuxeo.com/careers/
We are hiring !
New York, Paris, Lisboa