Web applications data persistence for MXCuBE 3

Web applications

data persistence:

case study for MXCuBE 3

8th MXCuBE meeting, 27 & 28th June 2016 - EMBL, Hamburg - Germany

Matias Guijarro

Software Engineer @ Beamline Control Unit

European Synchrotron Radiation Facility

Grenoble, France

What is web applications data persistence ?

Capability of remembering a state

Tracking state changes across different periods of time

within the same instance of the application
or multiple instances

2 approaches : server-side or client-side

Server-side persistence

Session + authentication cookie

data is stored on the server
an HTTP cookie makes the link between client and server

Server-side persistence

Session on server

storage size limit is the disk space
any kind of data can be saved (binary, strings, etc)
storage can be delegated to specialized software

Authentication cookie

well known mechanism, standard HTTP
configurable TTL (time to live)

Client-side persistence

Why storing data on the client ?

better performance: caching data so it can be retrieved without additional server requests
big amount of client-side-only data: User Interface dump, widgets state
to make the application work off-line

Client-side persistence

HTML5 Web Storage

Local storage: keep data locally within user's web browser, no automatic expiry
Session storage: same as above, but data is deleted when tab is closed
Indexed DB: structured, transactional, high-performance NoSQL-like data store ; persistent, no automatic expiry

Up to 10 MB (only a few KB for cookies)

(key, value) pairs, strings only : needs serialization

Client-side persistence

LocalStorage API

```
setItem(key, value)
```
```
getItem(key)
```
```
removeItem(key)
```
```
clear()
```

Storage event

fired when a storage area has been modified

Data and state persistence in MXCuBE 3

What do we need to achieve ?

user session: authentication and keeping track of connected users - only one client at a time can operate the beamline
auto (re)login, if page is reloaded, or closed then accessed again
UI state caching: if page is reloaded, user wants to get back the exact same User Interface state, not to start from scratch - beware of consistency problems

User session in MXCuBE 3

Flask-Session

extension to Flask for server-side sessions
uses Redis as data store
handles cookie management and storage

from flask import session
...

@mxcube.route("/mxcube/api/v0.1/login", methods=["POST"])
def login():
    content = request.get_json()
    loginID = content['proposal']
    password = content['password']
    
    loginRes = mxcube.db_connection.login(loginID, password)
   
    if loginRes['status']['code'] == 'ok':
        session['loginInfo'] = { 'loginID': loginID, 'password': password, 'loginRes': loginRes } 
    ...

@mxcube.route("/mxcube/api/v0.1/signout")
def signout():
    session.clear()
    ...

from flask import Flask
from flask.ext.session import Session

...

app = Flask(__name__, static_url_path='')
app.config['SESSION_TYPE'] = "redis"
app.config['SESSION_KEY_PREFIX'] = "mxcube:session:"
sess = Session()
sess.init_app(app)

User session in MXCuBE 3, auto login

The session cookie has no expiry date on the client

When MXCuBE 3 application loads, login_info route is called: the client sends its cookie (if it exists), MXCuBE server can decide to do auto login or to close the session

mxcube.route("/mxcube/api/v0.1/login_info", methods=["GET"])
def loginInfo():
    loginInfo = session.get("loginInfo")
 
    if loginInfo is not None:
        # for the moment, we always do auto login if a client cookie is set ;
        # of course, it needs more logic before going production-ready
        loginInfo["loginRes"] = mxcube.db_connection.login(loginInfo["loginID"], loginInfo["password"])
        session['loginInfo'] = loginInfo
  
    return jsonify({ "synchrotron_name": mxcube.session.synchrotron_name,
                      "beamline_name": mxcube.session.beamline_name,
                      "loginType": mxcube.db_connection.loginType.title(),
                      "loginRes": convert_to_dict(loginInfo["loginRes"] if loginInfo is not None else {}) })

User Interface state persistence in MXCuBE 3

What do we want to persist from UI ?

Samples Grid
Queue - queue state
Sample View - current loaded sample
Log messages

What we cannot persist

Beamline Setup
Actuators positions and state
Sample Changer

synchronisation with server needed

User Interface state persistence in MXCuBE 3

Proposal: client-side-first persistence

Why ?

better performance
thanks to Redux, saving the entire UI state and restoring it comes for free and is 100% accurate
code for state (re)construction from the client to the server always exists, whereas the opposite is not true

Client-side persistence in MXCuBE 3 with redux-persist

Store enhancer for redux

persists a store
can rehydrate it

import {persistStore, autoRehydrate} from 'redux-persist'
const store = createStore(reducer, undefined, autoRehydrate())
persistStore(store)

Basic usage involves adding 3 lines to the application

Client-side persistence in MXCuBE 3 with redux-persist

Auto-rehydration works

Possibility to have a fine-grained control thanks to the REHYDRATE action

cache invalidation
synchronisation with server
other fancy stuff

Do not miraculously solves all persistence problems, but helps a lot

State persistence in MXCuBE 3:

tentative use cases and expected behaviour

1. User reloads page when no queue is running

state is restored from cache (local storage)
synchronisation for beamline setup, actuators, sample changer

2. User reloads or closes page while a queue is running

queue is stopped, then same as (1)

3. User logs out

UI is cleared (start from scratch), cache is cleared
server-side session is deleted
forbidden to log out while queue is running

4. Same user opens another instance of the application from the same computer (e.g from another tab)

local storage is shared, both applications will be in sync
same as (1) to initialise state

5. Same user opens another instance of the application from another computer

Remote Access ? Use of remote access feature (together.js)
or ask to close other session

6. Another user opens another instance of the application from another computer

Reject login

Thanks for your attention

Questions ?