A new way of client-server conversation

Andrey Sitnik, Evil Martians

From Russia with love

The creator of

Autoprefixer

PostCSS

Browserslist

Slides will be in Twitter

sitnikcode

Part 1. The problem.

Image: Invasion Base on the Moon (Apr, 1948)

The 36-Hour War: Life Magazine, 1945

My family lives in China

Chinese Internet

40% packet loss

Websites on Chinese Internet

Waiting for the next page

No stable Internet
in New York subway

Image: Mr. Robot

No stable Internet
on dotConfs in Paris

Image: Nicolas Ravelli

Internet is always unstable

Image: John Stanmeyer

Long offline

Everyday offline

Long ping
Unstable Internet

vs.

Image: Nikita Prokopov

Connection in development

Image: Nikita Prokopov

Real connection

Common problem #1

Infinite loader
on AJAX errors

Universal solution

Have you tried to press “Reload” button?

Common problem #2

Continue to work

Save

Wait

Request statistics

Success

Failure

Common problem #3

Images: Nikita Prokopov

You always have at least 2 users

Part 2. The dream.

Image: O'Neill cylinder

All regular web apps…

… will have real-time updates …

new comment

update comments list

… will sync changes in background …

Image author: Denys Mishunov

… will have offline support …

Read-only offline

Edit in offline

… will not need a lot of code …

… and will use existing ecosystems

Replace DB

Part 3. The analysis.

Complexity curve with AJAX

Complexity

Features

Replace AJAX

Image: Deadpool movie

Current approach

AJAX request

The things
we care about

Don’t care

Distributed systems

Sync

Distributed computing is not easy

Image: J. R. Wharton Eyerman

Frameworks reduce complexity

Framework is the idea

Ruby on Rails: Getting Real

React: retained rendering of components

Redux: newState = reducer(state, action)

Image: Robert McCall

Part 4. The idea.

Saint Petersburg has best bars in Russia

Conflict

Images: Nikita Prokopov

{
  title: '伏特加',
  likes: 50
}

{
  title: '白兰地',
  likes: 51
}

Solution

“Bad programmers worry about the code.
Good programmers worry about
data structures”

— Linus Torvalds

Source: https://lwn.net/Articles/193245/

Event sourcing

Images: Nikita Prokopov

['post:title', '伏特加']
['likes:set', 51]

['likes:remove', 1]

['post:title', '伏特加']
['likes:set', 51]

['post:title', '白兰地']

CRDT , 2009

hal.upmc.fr/inria-00555588/document

Conflict-free Replicated Data Types

Op-based counter
G-Counter
PN-Counter
Non-negative Counter

LWW-Register
MV-Register

G-Set
2P-Set
LWW-element-Set
PN-Set
OR-Set

Add-only monotonic DAG
Add-remove Partial Order

Replicated Growable Array
Continuous sequence

CRDT in JS

Swarm.js

@gritzko

Logs everywhere

Redux actions

GraphQL mutations

CRDT
log

Undo/redo
log

Single log to rule them all

Redux

Sync with back-end

CRDT

Undo/redo

Log

Actions log on client

{
  type: 'CHANGE_NAME',
  user: 13,
  name: 'New name'
}

Images: Nikita Prokopov

Actions log on server

Images: Nikita Prokopov

Send new actions to server

WebSocket

Images: Nikita Prokopov

Send new actions after going offline

— Give me new actions since “4”

Images: Nikita Prokopov

Server could create actions too

Real-time update out of box

Cross-tab sync too

Put action to the log
and forget about it

AJAX shows loader

Rename to New

Name: New

Save to DB

Optimistic UI out of box

Rename to New

Name: New

Save to DB

Common UI to show sync status

Any client action can be undone on error

Name: New

Rename to New

Name: Old

Error

Undo rename

Part 5. Details.

Image: Davis Paul Meltzer

How to write open source

Step 1 Create an idea

Step2 Implement it

Problem 1: order of actions

State: B

State: A

Images: Nikita Prokopov

Sort by time?

[
  [action, { time: Date.now() }]
].sortBy(i => i[1].time)

Distributed time in complicated 1

const time1 = Date.now()
const time2 = Date.now()

time2 !== time1

Distributed time in complicated 2

const time1 = Date.now()
const time2 = Date.now()

time2 >= time1

Distributed time in complicated 3

const client = await client.askTime()
const server = Date.now()

diff(client, server) < HOUR

Vector clock?

const prevTime = 0

time: [
  prevTime++,
  nodeId
]

Vector clock problems

1. Doesn’t work good in offline

2. Doesn’t connected to actual time (no month, weekday)

Logux time

id: [1489328408677, '10:bfj58s0w', 0]



id: [1489328408677, '10:bfj58s0w', 1]
id: [1489328408678, '10:bfj58s0w', 0]

Milliseconds

Unique Node ID

User ID

Random string

Increase on same millisecond

Server sends current time

[
  'connected',
  [0, 0],          // Protocol version
  'server:h4vjdl', // Server unique ID
  [
    1475316481050, // "connect" message was received
    1475316482879  // "connected" message was sent
  ]
]

Client calculates and applies time difference

{ // Received
  id,
  time: id[0] + timeFix
}

{ // Sent
  id,
  time: id[0] - timeFix
}

const roundTrip = now - startTime - authTime
const timeFix = startTime -
                receivedTime +
                roundTrip / 2

Logux time benefits

ID is unique for every node
Сonstantly increasing
Works offline
Return month, weekday, etc
In current node timezone

Problem 2: tabs conflict

Images author: Sebastien Gabriel & Nikita Prokopov

Duplicate

Action

Saved

I hate programming
I hate programming
I hate programming
It works!
I love programming

Only the leader tab keeps WS

leader

follower

Images author: Sebastien Gabriel & Nikita Prokopov

Synchronized state between tabs

Part 6. The reality.

Image: NASA

API

“Talk is cheap. Show me the code.”

— Linus Torvalds

Source: lkml.org/lkml/2000/8/25/132

We care about client JS size

logux + redux ≈
12.6 KB

Thanks to
Size Limit

Redux-compatible API

3 ways to dispatch

```
dispatch
```
```
dispatch.crossTab
```
```
dispatch.sync
```

Cross-tab actions

dispatch.crossTab({
  type: 'READ_NOTIFICATION',
  id: notification.id
})

Sync Redux actions with server

dispatch.sync({
  type: 'CHANGE_NAME',
  userId: user.id,
  name: newName
})

Instead of AJAX, sagas, etc

showLoader()
fetch('/profile', {
  credentials: 'include',
  method: 'POST',
  form
}).then(r => {
  hideLoadaer()
}).catch(e => {
  …
})

dispatch.sync({
  type: 'CHANGE_NAME',
  userId: user.id,
  name: newName
})

Node.js framework for server

import { Server } from 'logux-server'

const app = new Server(
  subprotocol: '1.0.0',
  supports: '1.x',
  root: __dirname
})

Handling actions on server

app.type('CHANGE_NAME', {
  access(action, meta, userId) {
    return userId === action.user
  }
  process(action, meta) {
    …
  }
})

Use any DB or send any requests

app.type('CHANGE_NAME', {
  …
  process (action, meta) {
    return db.user.id(action.user).then(u => {
      if (isFirstOlder(u.lastChange, meta)) {
        return u.changeName(action.name)
      }
    })
  }

Wrap legacy back-end (PHP, Ruby)

app.type('CHANGE_NAME', {
  …
  process(action) {
    return send('POST', 'user/rename.php', {
      name: action.name,
      id: action.user
    })
  }

Send actions to client at any time

app.log.add({
  type: 'CHANGE_NAME',
  name: 'Looser',
  user: user.id
})

Server could undo any action on clients

{
  type: 'logux/undo',
  id: [1489328408677, '10:bfj58s0w', 0],
  reason: 'error'
}

Decorator for subscribing

const subscribe = require('react-logux/subscribe')

@subscribe(props => `users/${ props.id }`)
class User extends React.Component {
  …
}

Subscription on the server

app.channel('user/:id', (params, action, meta, creator) => {
  if (hasAccess(creator.userId, params.id)) return false
  return getUser(params.id).then(user => {
    app.log.add(userStateAction(user), {
      nodeIds: [creator.nodeId]
    })
  })
})

Part 7. Trade-offs.

Image: NASA

VODKA

What Logux is not

Not a CRDT

Not a DB

Framework to implement both

Problem 1: Log cleaning

store.log.on('preadd', (action, meta) => {
  if (action.type === 'READ_NOTIFICATION') {
    meta.reasons.push('notification:' + action.notificationId)
  }
})

onNotificationRemove (id) {
  store.log.removeReason('notification:' + id)
}

Problem 2: Complexity

Complexity

App

Net

Logux

Cleaning

App

Right case for Logux

Complexity

App

Net

Logux