FMM

Overall schema

Database usages

Write

  • 10% of traffic
  • simple requests
  • ACID
  • 100% BO

Read

  • 90% of traffic
  • complex joins
  • high availability
  • 100% FO

CQRS

Command Query Responsibility Segregation

How to?

  • Using RabbitMq to Async commands
    • Scalable
    • Fault tolerance
  • Storing Domain Model in PgSQL
    • consistency
  • Storing Read Model data in ES
    • fast
    • fulltext search
    • cross brand queries

Drawback

  • Data are dupplicated
    • increase storage size
    • possible inconsistency
  • Update an sub-entity => recompute several entities
  • Decrease resilience

Benchmark

Base on a ~19Ko article and 5 Joins

              |  With PgSQL | With ES
--------------+-------------+---------------
1000 articles | 22 secondes | 0,675 seconde

Recommandations

  • Use RabbitMQ to handle async requests
  • Use PgSQL to store relationnal data
  • Store a Ready to fetch view in ElasticSearch

 

Multi brand

  • Isolate environments
    • Security
    • Performances
    • Backup time
    • Allow different application's version

Segmented services

  • 1 database per brand
  • 1 application-server per brand
  • Physically separated on several hosts

Keep the brand in data

  • Specify the brand in queries and responses
  • Allow "shared" service to store several brands

Unique Identifier

  • Use uuid v1 to identify resources
    • Unique in space (based on server's mac adresse)
    • Unique in time (based on server's time)
  • Allow several brands to be merged in the future

Drawback

  • Increase hosting cost
  • Add complexity to expose "cross-brand" API

Go further: Docker

  • Isolate environments
  • Improve deployments with "ready to use" images
  • Hybride infrastructure
    • VM (or BareMetal) to host databases
    • Docker to host applications
             |   VM    | Docker
-------------+---------+---------
Boot time    | ~1 min  | 0.3 sec
Memory usage | ~256 Mo | 1 Mo
Disk usage   | ~1 Go   | 100 Ko

Recommandations

  • Split environments by brand
  • Keep a field "brand" in queries and storage
  • Use docker to easily scale applications

Authentication

needs

  • Restrict access to services
  • Restrict access by client
  • Simple (no ACL, ...)

JWT

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlcyI6WyJyZWFkIiwid3JpdGUiXX0.sScm9scsYDYNASEmwkwCCNm3F5i9Qq6BNdeF2mN-rL0
{
  "alg": "HS256",
  "typ": "JWT"
}
{
  "roles": ["read", "write"]
}
HMACSHA256(
  base64UrlEncode(header)
  + "."
  +  base64UrlEncode(payload),
  "secret"
)

Recommendation

  • Use JWT to store "roles" per client
  • Add a service "authentication"
  • Use the nginx Gateway to
    • decode JWT
    • validate JWT
    • restrict acces by route

JWT Workflow

Recommandations

  • Use JWT to authenticate clients
  • Add an "Authentication" service to create the token
  • Validate security at the gateway level


HTTP Cache

managed at service level

HTTP Cache

avantages

  • autonomous services
  • no dependencies between services
  • services able to warmup cache
  • services able to invalidate cache

Caching strategy

always up to date

  • Use long life cache
  • Cache invalidation managed by application
    • With Nginx: Purge resource's URIs
    • With Varnish: Use Tags

HTTP Cache

benchmark based on an standard API request

              | Without cache | With cache
--------------+---------------+------------
 100 requests |     0m17.162s |   0m0.252s
1000 requests |     2m45.451s |   0m1.160s
  • HTTP call < 2ms
  • Don't use cache at client's level
  • Expose small cachable resources

FMM's Unified cache

  • Keep cache on top of fronts
  • Add a service to purges those caches
    • When a services update a "resource" it trigger an invalidation to the cache
    • How to convert "resource" to URL ?
  • Convert API's unified cache into "Converter"'s cache
    • Same as front's cache
    • How to convert "resource" to URL ?

Recommandations

  • Use only HTTP cache
  • Each service manage his own cache
  • Use cache invalidation instead of expirity
  • Add a "cache" service in charge of invalidating FO caches

 

Several API format

  • Each client has a different need
    • Mobile needs light responses
    • Site's Homepage needs basics informations
    • Site's article needs a full response

GraphQL

  • by Facebook
  • Query language specifications
  • Alternative to REST

GraphQL

{
  user(id: 3500401) {
    id,
    name,
    isViewerFriend,
    profilePicture(size: 50)  {
      uri,
      width,
      height
    }
  }
}

Request

{
  "user" : {
    "id": 3500401,
    "name": "Jing Chen",
    "isViewerFriend": true,
    "profilePicture": {
      "uri": "http://someurl.cdn/pic.jpg",
      "width": 50,
      "height": 50
    }
  }
}

Response

Usage

  • Handled by the "conversion" service
  • Replacement for json, json2, json_mobile
  • Less cachable : new request = new cache

Recommandations

Development environment

  • Reduce Time to market
  • Improve quality
  • Implemented with the coach

Automatize dev stack

  • Ansible (or docker) to build dev environment
  • Closest to the production environment
  • Fast and easy bootstrap

CI

  • Automatized builds with Gitlab CI (or Jenkins)
  • phpmetric to monitor the code quality
  • Sensiolabs Insight to analyse the code
  • Blackfire to spot performance issues

Monolithic repository

  • 1 repository
  • 1 sub repository per application
    • Automaticaly generated from main repo
    • Isolated deployment

Recommandations

  • Industrialized dev environment
  • Monitor tests and quality with the CI
  • Use monolithic repository

Questions ?

Made with Slides.com