Logux, a new way of client-server conversation (Chinese)

A new way of client-server conversation

Andrey Sitnik, Evil Martians

Russia is fun

My open source

Autoprefixer

PostCSS

Front-end architecture …

State

Reducers

Components

… is great

State

Reducers

Components

Back-end …

State

Reducers

Components

View

Controller

Model

… is great too

State

Reducers

Components

View

Controller

Model

But the problem lies in between

State

Reducers

Components

View

Controller

Model

Communication

AJAX Requests

Chapter 1

Websites

Static pages

DHTML & jQuery

SPA

Live updates

Offline

New synchronization

AJAX

Two-field form

<form>
  <input name="name"…>
  <input name="color"…>
</form>

Server synchronization code

import { Actions } from 'flummox'
import api from '../api/labels-api'
export default class LabelsActions extends Actions {
  async add(project, data) {
    return await api.add(project, data).then( id => {
      return [id, { ...data, id, project }]
    })
  }
  async update(pr, id, data) {
    return await api.update(pr, id, data).then(() => {
      return [id, { ...data, id, project }]
    })
  }
  async remove(project, id) {
    return await api.remove(project, id).then(() => id )
  }
}

import { OrderedMap } from 'immutable'
import { Store }      from 'flummox'
export default class LabelsStore extends Store {
  constructor(flux, data) {
    super()
    this.state = {
      labels: OrderedMap(data.map(i => [parseInt(i.id), i]))
    }
    let actions = flux.getActionIds('labels')
    this.register(actions.add,    this.update)
    this.register(actions.update, this.update)
    this.register(actions.remove, this.remove)
  }
  remove(id) {
    this.setState({
      labels: this.state.labels.delete(parseInt(id))
    })
  }
  update([id, data]) {
    this.setState({
      labels: this.state.labels.set(parseInt(id), data)
    })
  }
}

fetch('/comments.json').then(r => r.json()).then(list => {
  setComments(list)
})

Simple fetch

showLoader()
fetch('/comments.json').then(r => r.json()).then(list => {
  hideLoader()
  setComments(list)
})

Fetch + loader

showLoader()
fetch('/comments.json').then(r => r.json()).then(list => {
  hideLoader()
  setComments(list)
}).catch(err => {
  if (isNoNetwork(err)) {
    showNetworkError(err)
  } else {
    showServerError(err)
  }
})

Fetch + loader + error handling

Local connection in development

Image author: Nikita Prokopov

Real connection in production

Image author: Nikita Prokopov

Complexity curve with AJAX

Complexity

Features

More complicated problems

Deduplication
Relevance
Protocol version validation
Conflicts

Better world

SPA

Live Updates

Offline

Logux goals

Reduce code size
Support current ecosystems
Live updates out of the box
Optimistic UI & Offline-first

Idea

Chapter 2

Saint Petersburg — best bars in Russia

Conflict

Images author: Nikita Prokopov

{
  title: 'алкоголь',
  likes: 50
}

{
  title: 'водка',
  likes: 51
}

Solution

“Bad programmers worry about the code.
Good programmers worry about
data structures”

— Linus Torvalds

Source: https://lwn.net/Articles/193245/

Event sourcing

Images author: Nikita Prokopov

['post:title', 'алкоголь']
['likes:set', 51]

['likes:remove', 1]

['post:title', 'алкоголь']
['likes:set', 51]

['post:title', 'водка']

CRDT, 2009

hal.upmc.fr/inria-00555588/document

Conflict-free Replicated Data Types

Op-based counter
G-Counter
PN-Counter
Non-negative Counter

LWW-Register
MV-Register

G-Set
2P-Set
LWW-element-Set
PN-Set
OR-Set

Add-only monotonic DAG
Add-remove Partial Order

Replicated Growable Array
Continuous sequence

CRDT in JS

Swarm.js

@gritzko

Event sourcing is everywhere

Redux
DevTools
log

CRDT
log

Backend DB
oplog

Log must be a first-class citizen

Logux inspiration

Redux

Swarm.js

Actions log on client

{
  type: 'CHANGE_NAME',
  user: 13,
  name: 'New name'
}

Images author: Nikita Prokopov

Actions log on server

Images author: Nikita Prokopov

Synchronize logs

Images author: Nikita Prokopov

WebSocket

Synchronization is simple

Images author: Nikita Prokopov

— Give me new actions since “4”

Server actions

Images author: Nikita Prokopov

Between clients

Images author: Nikita Prokopov

Problems and Solutions

Chapter 3

How to write open source

Step 1 Create an idea

Step2 Implement it

Problem 1: order of actions

State: B

State: A

Images author: Nikita Prokopov

Sort by time?

{ action }, { time }

Distributed time is сomplicated

Questions about time: 1

const time1 = Date.now()
const time2 = Date.now()

time2 !== time1

A millisecond is a lot of time

Millisecond

Actions

Questions about time: 2

const time1 = Date.now()
const time2 = Date.now()

time2 >= time1

CloudFlare leap second problem

Source: blog.cloudflare.com/how-and-why-the-leap-second-affected-cloudflare-dns/

Questions about time: 3

const client = await client.askTime()
const server = Date.now()

diff(client, server) < HOUR

Wrong time zone

Source: commons.wikimedia.org/wiki/File:Standard_World_Time_Zones.png

Solution 1: action ID

id: [1489328408677, '10:bfj58s0w', 0]

Milliseconds

Unique Node ID

User ID

Random string

Timestamp always increases

if (now < prevNow) {
  now = prevNow
}
prevNow = now

Sequence number

[1489328408677, '10:…', 0]
[1489328408677, '10:…', 1]
[1489328408678, '10:…', 0]

Same milliseconds

Server sends current time

[
  'connected',
  [0, 0],          // Protocol version
  'server:h4vjdl', // Server unique ID
  [
    1475316481050, // "connect" message was received
    1475316482879  // "connected" message was sent
  ]
]

Client calculates time difference

const roundTrip = now - startTime - authTime
const timeFix = startTime -
                receivedTime +
                roundTrip / 2

Client applies time difference

{ // Received
  id,
  time: id[0] + timeFix
}

{ // Sent
  id,
  time: id[0] - timeFix
}

Chinese Internet

40% packet loss

Problem 2: disconnect detection

ws.onclose = function () {
  // Could be never called
}

Solution 2: ping command

> ['ping', 120]
< ['pong', 130]

Live connection status

if (loguxClient.connected) {
  // Connection was tested
  // at least 5 second ago
}

Problem 3: tabs conflict

Images author: Sebastien Gabriel & Nikita Prokopov

Duplicate

Action

Saved

I hate programming
I hate programming
I hate programming
It works!
I love programming

Solution 3: only the leader tab keeps WS

Images author: Sebastien Gabriel & Nikita Prokopov

leader

follower

Synchronized state between tabs

Modern front-end

Expectation

Reality

Other problems of distributed systems

Problems
Firewalls and WS
Log cleaning

Scaling
Outdated clients
Deduplication

Solutions
Force WSS
Action’s reasons of life

Peer-to-peer protocol
Subprotocol verification
Action ID

Logux API

Chapter 4

API

“Talk is cheap. Show me the code.”

— Linus Torvalds

Source: lkml.org/lkml/2000/8/25/132

Client: we care about client JS size

logux + redux ≈
12 KB

Client: Redux compatible API

import { createLoguxStore } from 'logux-redux'

const store = createLoguxStore(reducers, 
  preloadedState, undefined, {
  url:         'wss://logux.example.com/',
  userId:       user.id,
  subprotocol: '1.1.5',
  credentials:  user.token
})

Client: for “your friend” without Redux

import Client from 'logux-client/client'

const logux = new Client({
  url:         'wss://logux.example.com/',
  userId:       user.id,
  subprotocol: '1.1.5',
  credentials:  user.token
})

Client: local actions

dispatch({ type: 'MENU_OPEN' })

Client: cross-tab actions

add({
  type: 'READ_NOTIFICATION',
  id: notification
}, {
  reasons
})

Client: server actions

add({
  type: 'CHANGE_NAME',
  user, name
}, {
  sync: true,
  reasons
})

Client: instead of AJAX

showLoader()
fetch('/profile', {
  credentials: 'include',
  method: 'POST',
  form
}).then(r => {
  hideLoadaer()
}).catch(e => {
  …
})

add({
  type: 'CHANGE_NAME',
  user, name
}, {
  sync: true,
  reasons
})

Client: Optimistic UI by default

case 'CHANGE_NAME':
  return { name: action.name, ...user }

Optimistic UI

Image author: Denys Mishunov

Client: server could undo action on error

{
  type: 'logux/undo',
  id: [1489328408677, '10:bfj58s0w', 0],
  reason: 'error'
}

Logux Status: widget

Logux Status: favicon

Normal

Offline

Error

Logux Status: warning

Client: action storage

Local

Cached data

Not synchronized yet

Images author: Nikita Prokopov

Client: Pessimistic UI

Client: ask server for changes

add({ type: 'ASK_PAY' })

…

case 'ASK_PAY':
  return { payment: 'waiting' }

Client: wait for server answer

case 'PAID':
  return { payment: 'paid' }

Server: Node.js framework

import { Server } from 'logux-server'

const app = new Server({
  subprotocol: '1.2.0',
  supports: '1.x',
  root: __dirname
})

Server: actions handling

app.type('CHANGE_NAME', {
  access(action, meta, userId) {
    return userId === action.user
  }
  process(action, meta) {
    …
  }
})

Server: use any DB or send any requests

process(action, meta) {
  return db.user.id(action.user).then(u => {
    if (isFirstOlder(u.lastChange, meta)) {
      return u.changeName(action.name)
    }
  })
}

Server: send actions to client at any time

app.log.add({
  type: 'CHANGE_NAME',
  name: 'Looser',
  user: user.id
})

RESTful vs. Logux

Logux

RESTful

URL, HTTP method

action.type

POST form, GET query

Action object

Request happened now

Action could be

created in the past

Wrap legacy back-end (PHP, Ruby)

process({ user, name }) {
  return send('POST',
              `user/${ user }/change.php`,
              { name })
}

Logux Protocol is open

github.com/logux/logux-protocol

Backend languages plan

What is Logux?

Replaces AJAX and RESTful

Synchronizes logs (like Redux actions)
between client and server

Why Logux?

Less code
Live updates out of the box
Optimistic UI & Offline-first
Redux API on the front-end
Similar to RESTful on the back-end

Trade-offs

Chapter 4

Problem 1: Complexity

Complexity

App

Net

Logux

Cleaning

App

Right case for Logux

Complexity

App

Net

Logux

Cleaning

App

Problem 2: under construction

0.1 preview

protocol, core components

end-user API

docs

0.2

autosubscribe, scaling

0.3

Amplifr already uses it in production

>30 contributors

Want to reduce the code now?

Relay

Relay-like API for Logux is planned

Want to be Offline-first now?

Firebase

PouchDB

Gun.js

Want CRDT now?

Swarm.js

Questions?

@logux_io

@andreysitnik