Cloud level scalability
nuxeo.io, Elasticsearch et MongoDB
ABOUT Cloud Promises
what we expect from the Cloud
Scale to the sky !
- Progressive scaling - economically efficient
- Quick scaling - on demand
- Easy fail-over - design for failure
On demand / SaaS model
- Have an idea / requirement
- Select solution
- Configure solution
- Instanciate solution
- Start working !
Build a custom app on demand !
I need an App for
Case Management !
I need an app for
Content Management
I need an App for
Assets Management !
I need an app for
Document Management
Not that easy !
Nuxeo Platform DOES Help !
Technologies
Architectures
Components
Deployment
Packaging
Management tools
Leverage Cloud Promises
Some basic rules
Leverage the existing infrastructure
you should !
-
Technically
- It can scale
- It is well integrated
- It is maintained
-
Economically
- It is cheaper
- It is easier to track and bill
- It is ready now
+ the technologies names are cooler !
AWS infrastructure
-
Storage
- S3 : Binary storage
-
RDS : SQL Database
-
Monitoring & Scaling
- CloudWatch : Monitoring
- AutoScaling : Automatic Scale out
- ElasticCache : Distributed Caching
-
SNS : Mass notifications
-
Network
- ELB : Loadbalancing
- CloudFront : CDN
Nuxeo is a good Fit
-
We use mainstream technologies
- Java / Redis / SQL
- Java / Redis / SQL
- Platform is easy to control
- JMX monitoring
-
REST API and nuxeoctl
-
The platform is pluggable
- we can change the services backends as needed
- Repository
- Blob Store
- Audit
- Cache
- we can change the services backends as needed
Nuxeo and AWS - now
Nuxeo Cloud tooling
-
Container provisioning
- AWS : Custom AMIs and CloudFormation template
-
Docker images / VM Images
-
System level provisioning
- Debian packages
-
Ansible Playbooks
-
Application level provisioning
- Marketplace packages
-
nuxeoctl
-
Global Automation
- Ansible Playbooks
Nuxeo And AWS - next steps
- Edge caching for large assets
- Leverage CloudFront and S3 Binary Manager
- Leverage CloudFront and S3 Binary Manager
- Multi-devices notification
- Leverage SNS for push notifications
- Leverage SNS for push notifications
- Video transcoding
- Leverage Elastic Transcoder
Distributed architecture
Cloud loves distributed architectures
distributed ?
When Scaling up is no more an option
- technically
- economically
PARADIGM EVOLVE
Centralized & ACID
but overloaded !
Distributed &
Eventually Consistent
but faster
Really fastER !
Distribute the Index !
with Elasticsearch
Shard data on multiple repositories
leverage federated search
It does Scale for ReaL !
-
1 Billions documents
- 10 Repositories (10 PostgreSQL)
- 1 Elasticsearch cluster
- Injection speed :
- bulk import at 6000 docs/s
- indexing at 3500 docs/s
- API Access :
- 3500+ Requests/s
- Response time < 0.2s
Distribute the Database
with Document Based Storage
is the first implementation backend
Distribute Processing
next steps
- Continue leverage Elasticsearch
- Audit
- Activity Stream
- Provide additional backends for DBS
- PGSQL with JSON
-
Cassandra or CouchDB
- Continue pushing transient shared data to Redis
- Transient state manager
Application Factory for the Cloud
Nuxeo Application Factory
WHY AN Application Factory ?
- Provide complete flow in the Cloud
- end to end experience
- end to end experience
- Encourage good practice
- share common components
- continuous deployment
- no excuse to not make tests
-
Build SaaS application
- 2 levels of configuration
Yeh, that's not easy at all !
- Manage a lot of VMs is complex
- Deploy, Update, Monitor, Route is a lot of work
-
find a way to automate this !
-
find a way to automate this !
- Deploy, Update, Monitor, Route is a lot of work
- Pricing and billing is a tricky game
- A lot of VMs all the time is expensive
- Dynamically optimize infrastructure to requirements
- A lot of VMs all the time is expensive
Example use case :
provide an online trial for your SaaS solution
Hopefully, there is a solution ...
Nuxeo.io
- The final step of the Application Factory flow
-
deploy your custom application
-
deploy your custom application
- The technology underlying Nuxeo online trial
-
configure, deploy and test Nuxeo Platform in one click
-
configure, deploy and test Nuxeo Platform in one click
- Best Nuxeo Cloud technologies all together in one service
- Nuxeo / Docker / CoreOS / AWS
- Nuxeo / Docker / CoreOS / AWS
- A generic infrastructure for Application Factory
- Optimize IaaS resources with high density Docker deployment
Nuxeo.io - Architecture
Nuxeo.IO - DEMO
Nuxeo.io - new services to come
-
Nuxeo IDM
- Centralized Identity Management for your application
- Compliant with SAML, SCIM and major IDM providers
-
Nuxeo Converters
- Packages Nuxeo Converters as an online service
- Leverage AWS infrastructure
-
Nuxeo Push
- Provide gateway to Push and Notification services
Q&A
Scaling To The Sky
By Thierry Delprat
Scaling To The Sky
- 3,006