Cloud level scalability

nuxeo.io, Elasticsearch et MongoDB 

ABOUT Cloud Promises

what we expect from the Cloud

Scale to the sky !

  • Progressive scaling - economically  efficient
  • Quick scaling - on  demand
  • Easy fail-over  - design for failure

On demand / SaaS model

  1. Have an idea / requirement
  2. Select solution
  3. Configure solution
  4. Instanciate solution
  5. Start working !

Build a custom app on demand !

I need an App for
Case Management  !

I need an app for
Content Management

I need an App for
Assets Management  !

I need an app for
Document Management

Not that easy !

Nuxeo Platform DOES Help !

Technologies
Architectures
Components

Deployment

Packaging
Management tools

Leverage Cloud Promises

Some basic rules

Leverage the existing infrastructure

you should !

  • Technically
    • It can scale
    • It is well integrated
    • It is maintained
       
  • Economically
    • It is cheaper
    • It is easier to track and bill
    • It is ready now

+ the technologies names are cooler !

AWS infrastructure

  • Storage
    • S3 : Binary storage
    • RDSSQL Database
       
  • Monitoring & Scaling
    • CloudWatch : Monitoring
    • AutoScaling : Automatic Scale out
    • ElasticCache : Distributed Caching
    • SNS : Mass notifications
       
  • Network
    • ELB : Loadbalancing
    • CloudFront : CDN

Nuxeo is a good Fit

  • We use mainstream technologies
    • Java / Redis / SQL
       
  • Platform is easy to control
    • JMX monitoring
    • REST API and nuxeoctl
      
      
  • The platform is pluggable 
    • we can change the services backends as needed
      • Repository
      • Blob Store
      • Audit
      • Cache

Nuxeo and AWS - now

Nuxeo Cloud tooling

  • Container provisioning
    • AWS : Custom AMIs and CloudFormation template
    • Docker images / VM Images
       
  • System level provisioning
    • Debian packages 
    • Ansible Playbooks
       
  • Application level provisioning
    • Marketplace packages
    • nuxeoctl
      
      
  • Global Automation
    • Ansible Playbooks

Nuxeo And AWS - next steps

  • Edge caching for large assets
    • Leverage CloudFront and S3 Binary Manager
       
  • Multi-devices notification
    • Leverage SNS for push notifications
       
  • Video transcoding
    • Leverage Elastic Transcoder

Distributed architecture

Cloud loves distributed architectures

distributed ?

When Scaling up is no more an option

  • technically
  • economically

PARADIGM EVOLVE

Centralized & ACID

but overloaded !

Distributed &
Eventually Consistent

but faster

Really fastER !

Distribute the Index !

with Elasticsearch

Shard data on multiple repositories

leverage federated search

It does Scale for ReaL !

  • 1 Billions documents
    • 10 Repositories (10 PostgreSQL)
    • 1 Elasticsearch cluster
       
  • Injection speed : 
    • bulk import at 6000 docs/s
    • indexing at 3500 docs/s
       
  • API Access :
    • 3500+ Requests/s
    • Response time < 0.2s

Distribute the Database

with Document Based Storage

is the first implementation backend

Distribute Processing

next steps

  • Continue leverage Elasticsearch
    • Audit
    • Activity Stream
       
  • Provide additional backends for DBS
    • PGSQL with JSON
    • Cassandra or CouchDB
       
  • Continue pushing transient shared data to Redis
    • Transient state manager

Application Factory for the Cloud

Nuxeo Application Factory

WHY AN Application Factory ?

  • Provide complete flow in the Cloud
    • end to end experience
       
  • Encourage good practice 
    • share common components
    • continuous deployment
    • no excuse to not make tests
       
  • Build SaaS application
    • 2 levels of configuration

Yeh, that's not easy at all !

  • Manage a lot of VMs is complex
    • Deploy, Update, Monitor, Route is a lot of work
      • find a way to automate this !
         
  • Pricing and billing is a tricky game
    • A lot of VMs all the time is expensive
      • Dynamically optimize infrastructure to requirements

Example use case :

provide an online trial for your SaaS solution

Hopefully, there is a solution ...

Nuxeo.io

  • The final step of the Application Factory flow
    • deploy your custom application
       
  • The technology underlying Nuxeo online trial
    • configure, deploy and test Nuxeo Platform in one click
       
  • Best Nuxeo Cloud technologies all together in one service
    • Nuxeo / Docker / CoreOS / AWS
       
  • A generic infrastructure for Application Factory
    • Optimize IaaS resources with high density Docker deployment

Nuxeo.io - Architecture

Nuxeo.IO - DEMO

Nuxeo.io - new services to come

  • Nuxeo IDM
    • Centralized Identity Management for your application
    • Compliant with SAML, SCIM and major IDM providers
       
  • Nuxeo Converters
    • Packages Nuxeo Converters as an online service
    • Leverage AWS infrastructure
       
  • Nuxeo Push
    • Provide gateway to Push and Notification services

Q&A

Scaling To The Sky

By Thierry Delprat

Scaling To The Sky

  • 3,006