Web applications

data persistence:

case study for MXCuBE 3

8th MXCuBE meeting, 27 & 28th June 2016 - EMBL, Hamburg - Germany

Matias Guijarro

Software Engineer @ Beamline Control Unit

European Synchrotron Radiation Facility

Grenoble, France

What is web applications data persistence ?

Capability of remembering a state

 

Tracking state changes across different periods of time

  • within the same instance of the application
  • or multiple instances

 

2 approaches : server-side or client-side

Server-side persistence

Session + authentication cookie

  • data is stored on the server
  • an HTTP cookie makes the link between client and server

 

 

Server-side persistence

Session on server

  • storage size limit is the disk space
  • any kind of data can be saved (binary, strings, etc)
  • storage can be delegated to specialized software

 

Authentication cookie

  • well known mechanism, standard HTTP
  • configurable TTL (time to live)

Client-side persistence

Why storing data on the client ?

  • better performance: caching data so it can be retrieved without additional server requests
  • big amount of client-side-only data: User Interface dump, widgets state
  • to make the application work off-line

 

Client-side persistence

HTML5 Web Storage

  • Local storage: keep data locally within user's web browser, no automatic expiry
  • Session storage: same as above, but data is deleted when tab is closed
  • Indexed DB: structured, transactional, high-performance NoSQL-like data store ; persistent, no automatic expiry

 

Up to 10 MB (only a few KB for cookies)

 

(key, value) pairs, strings only : needs serialization

Client-side persistence

LocalStorage API

  • setItem(key, value)
  • getItem(key)
  • removeItem(key)
  • clear()

 

Storage event

  • fired when a storage area has been modified

Data and state persistence in MXCuBE 3

What do we need to achieve ?

  • user session: authentication and keeping track of connected users - only one client at a time can operate the beamline
  • auto (re)login, if page is reloaded, or closed then accessed again
  • UI state caching: if page is reloaded, user wants to get back the exact same User Interface state, not to start from scratch - beware of consistency problems

 

User session in MXCuBE 3

Flask-Session

  • extension to Flask for server-side sessions
  • uses Redis as data store
  • handles cookie management and storage
from flask import session
...

@mxcube.route("/mxcube/api/v0.1/login", methods=["POST"])
def login():
    content = request.get_json()
    loginID = content['proposal']
    password = content['password']
    
    loginRes = mxcube.db_connection.login(loginID, password)
   
    if loginRes['status']['code'] == 'ok':
        session['loginInfo'] = { 'loginID': loginID, 'password': password, 'loginRes': loginRes } 
    ...

@mxcube.route("/mxcube/api/v0.1/signout")
def signout():
    session.clear()
    ...
from flask import Flask
from flask.ext.session import Session

...

app = Flask(__name__, static_url_path='')
app.config['SESSION_TYPE'] = "redis"
app.config['SESSION_KEY_PREFIX'] = "mxcube:session:"
sess = Session()
sess.init_app(app)

User session in MXCuBE 3, auto login

The session cookie has no expiry date on the client

 

When MXCuBE 3 application loads, login_info route is called: the client sends its cookie (if it exists), MXCuBE server can decide to do auto login or to close the session

mxcube.route("/mxcube/api/v0.1/login_info", methods=["GET"])
def loginInfo():
    loginInfo = session.get("loginInfo")
 
    if loginInfo is not None:
        # for the moment, we always do auto login if a client cookie is set ;
        # of course, it needs more logic before going production-ready
        loginInfo["loginRes"] = mxcube.db_connection.login(loginInfo["loginID"], loginInfo["password"])
        session['loginInfo'] = loginInfo
  
    return jsonify({ "synchrotron_name": mxcube.session.synchrotron_name,
                      "beamline_name": mxcube.session.beamline_name,
                      "loginType": mxcube.db_connection.loginType.title(),
                      "loginRes": convert_to_dict(loginInfo["loginRes"] if loginInfo is not None else {}) })

User Interface state persistence in MXCuBE 3

 

What do we want to persist from UI ?

  • Samples Grid
  • Queue - queue state
  • Sample View - current loaded sample
  • Log messages

What we cannot persist

  • Beamline Setup
  • Actuators positions and state
  • Sample Changer

synchronisation with server needed

User Interface state persistence in MXCuBE 3

 

Proposal: client-side-first persistence

 

Why ?

 

  • better performance
  • thanks to Redux, saving the entire UI state and restoring it comes for free and is 100% accurate
  • code for state (re)construction from the client to the server always exists, whereas the opposite is not true

Client-side persistence in MXCuBE 3 with redux-persist

Store enhancer for redux

  • persists a store
  • can rehydrate it 
import {persistStore, autoRehydrate} from 'redux-persist'
const store = createStore(reducer, undefined, autoRehydrate())
persistStore(store)

Basic usage involves adding 3 lines to the application

Client-side persistence in MXCuBE 3 with redux-persist

Auto-rehydration works

 

Possibility to have a fine-grained control thanks to the REHYDRATE action

  • cache invalidation
  • synchronisation with server
  • other fancy stuff

Do not miraculously solves all persistence problems, but helps a lot

State persistence in MXCuBE 3:

tentative use cases and expected behaviour

1. User reloads page when no queue is running

  • state is restored from cache (local storage)
  • synchronisation for beamline setup, actuators, sample changer

2. User reloads or closes page while a queue is running

  • queue is stopped, then same as (1)

3. User logs out

  • UI is cleared (start from scratch), cache is cleared
  • server-side session is deleted
  • forbidden to log out while queue is running

4. Same user opens another instance of the application from the same computer (e.g from another tab)

  • local storage is shared, both applications will be in sync
  • same as (1) to initialise state

5. Same user opens another instance of the application from another computer

  • Remote Access ? Use of remote access feature (together.js)
  • or ask to close other session

6. Another user opens another instance of the application from another computer

  • Reject login

Thanks for your attention

 

Questions ?

Web applications data persistence for MXCuBE 3

By Matias Guijarro

Web applications data persistence for MXCuBE 3

A case study about web applications data persistence, and what to do for MXCuBE 3

  • 1,124

More from Matias Guijarro