A new way of client-server conversation

Andrey Sitnik, Evil Martians

From Russia with love

The creator of

Autoprefixer

PostCSS

Browserslist

Slides will be in Twitter

Part 1. The problem.

Image: Invasion Base on the Moon (Apr, 1948)

The 36-Hour War: Life Magazine, 1945

My family lives in China

Chinese Internet

40% packet loss

Websites on Chinese Internet

Waiting for the next page

No stable Internet
in New York subway

Image: Mr. Robot

No stable Internet
on dotConfs in Paris

Image: Nicolas Ravelli

Internet is always unstable

Image: John Stanmeyer

Long offline

Everyday offline

Long ping
Unstable Internet

vs.

Image: Nikita Prokopov

Connection in development

Image: Nikita Prokopov

Real connection

Common problem #1

Infinite loader
on AJAX errors

Universal solution

Have you tried to press “Reload” button?

Common problem #2

Continue to work

Save

Wait

Wait

Request statistics

Success

Failure

Common problem #3

Images: Nikita Prokopov

name: A price: A

name: A price: B

name: B price: A

name: B price: A

You always have at least 2 users

1

2

Part 2. The dream.

Image: O'Neill cylinder

All regular web apps…

… will have real-time updates …

new comment

update comments list

… will sync changes in background …

Image author: Denys Mishunov

… will have offline support …

Read-only offline

Edit in offline

… will not need a lot of code …

… and will use existing ecosystems

Replace DB

Part 3. The analysis.

Complexity curve with AJAX

Complexity

Features

Replace AJAX

Image: Deadpool movie

Current approach

AJAX request

The things
we care about

Don’t care

Distributed systems

Sync

Distributed computing is not easy

Image: J. R. Wharton Eyerman

Frameworks reduce complexity

Framework is the idea

Ruby on Rails: Getting Real

React: retained rendering of components

Redux: newState = reducer(state, action)

Image: Robert McCall

Part 4. The idea.

Saint Petersburg has best bars in Russia

Conflict

Images: Nikita Prokopov

{
  title: '伏特加',
  likes: 50
}
{
  title: '白兰地',
  likes: 51
}

?

Solution

“Bad programmers worry about the code.
Good programmers worry about
data structures”

Linus Torvalds

Event sourcing

Images: Nikita Prokopov

['post:title', '伏特加']
['likes:set', 51]

['likes:remove', 1]
['post:title', '伏特加']
['likes:set', 51]

['post:title', '白兰地']

CRDT , 2009

Conflict-free Replicated Data Types

  • Op-based counter
  • G-Counter
  • PN-Counter
  • Non-negative Counter
  • LWW-Register
  • MV-Register
  • G-Set
  • 2P-Set
  • LWW-element-Set
  • PN-Set
  • OR-Set
  • Add-only monotonic DAG
  • Add-remove Partial Order
  • Replicated Growable Array
  • Continuous sequence

CRDT in JS

Swarm.js

@gritzko

Logs everywhere

Redux actions

GraphQL mutations

CRDT
log

Undo/redo
log

Single log to rule them all

Redux

Sync with back-end

CRDT

Undo/redo

Log

Actions log on client

{
  type: 'CHANGE_NAME',
  user: 13,
  name: 'New name'
}

Images: Nikita Prokopov

Actions log on server

Images: Nikita Prokopov

Send new actions to server

WebSocket

Images: Nikita Prokopov

Send new actions after going offline

1

2

3

4

5

— Give me new actions since “4”

Images: Nikita Prokopov

Server could create actions too

Real-time update out of box

Cross-tab sync too

Put action to the log
and forget about it

AJAX shows loader

Rename to New

Name: New

Save to DB

Optimistic UI out of box

Rename to New

Name: New

Save to DB

Common UI to show sync status

Any client action can be undone on error

Name: New

Rename to New

Name: Old

Error

Undo rename

Part 5. Details.

Image: Davis Paul Meltzer

How to write open source

Step 1 Create an idea

Step2 Implement it

Problem 1: order of actions

A

A

B

B

State: B

State: A

Images: Nikita Prokopov

Sort by time?

[
  [action, { time: Date.now() }]
].sortBy(i => i[1].time)

Distributed time in complicated 1

const time1 = Date.now()
const time2 = Date.now()

time2 !== time1

Distributed time in complicated 2

const time1 = Date.now()
const time2 = Date.now()

time2 >= time1

Distributed time in complicated 3

const client = await client.askTime()
const server = Date.now()

diff(client, server) < HOUR

Vector clock?

const prevTime = 0

time: [
  prevTime++,
  nodeId
]

Vector clock problems

1. Doesn’t work good in offline

2. Doesn’t connected to actual time (no month, weekday)

Logux time

id: [1489328408677, '10:bfj58s0w', 0]



id: [1489328408677, '10:bfj58s0w', 1]
id: [1489328408678, '10:bfj58s0w', 0]

Milliseconds

Unique Node ID

User ID

Random string

Increase on same millisecond

Server sends current time

[
  'connected',
  [0, 0],          // Protocol version
  'server:h4vjdl', // Server unique ID
  [
    1475316481050, // "connect" message was received
    1475316482879  // "connected" message was sent
  ]
]

Client calculates and applies time difference

{ // Received
  id,
  time: id[0] + timeFix
}

{ // Sent
  id,
  time: id[0] - timeFix
}
const roundTrip = now - startTime - authTime
const timeFix = startTime -
                receivedTime +
                roundTrip / 2

Logux time benefits

  1. ID is unique for every node
  2. Сonstantly increasing
  3. Works offline
  4. Return month, weekday, etc
  5. In current node timezone

Problem 2: tabs conflict

Images author: Sebastien Gabriel & Nikita Prokopov

Duplicate

Action

Saved

I hate programming
I hate programming
I hate program
ming
It works!
I love programming

Only the leader tab keeps WS

leader

follower

Images author: Sebastien Gabriel & Nikita Prokopov

Synchronized state between tabs

Part 6. The reality.

Image: NASA

API

“Talk is cheap. Show me the code.”

Linus Torvalds

We care about client JS size

logux + redux ≈
12.6 KB

Thanks to
Size Limit

Redux-compatible API

3 ways to dispatch

  1. dispatch
  2. dispatch.crossTab
  3. dispatch.sync

Cross-tab actions

dispatch.crossTab({
  type: 'READ_NOTIFICATION',
  id: notification.id
})

Sync Redux actions with server

dispatch.sync({
  type: 'CHANGE_NAME',
  userId: user.id,
  name: newName
})

Instead of AJAX, sagas, etc

showLoader()
fetch('/profile', {
  credentials: 'include',
  method: 'POST',
  form
}).then(r => {
  hideLoadaer()
}).catch(e => {
  …
})
dispatch.sync({
  type: 'CHANGE_NAME',
  userId: user.id,
  name: newName
})

Node.js framework for server

import { Server } from 'logux-server'

const app = new Server(
  subprotocol: '1.0.0',
  supports: '1.x',
  root: __dirname
})

Handling actions on server

app.type('CHANGE_NAME', {
  access(action, meta, userId) {
    return userId === action.user
  }
  process(action, meta) {
    …
  }
})

Use any DB or send any requests

app.type('CHANGE_NAME', {
  …
  process (action, meta) {
    return db.user.id(action.user).then(u => {
      if (isFirstOlder(u.lastChange, meta)) {
        return u.changeName(action.name)
      }
    })
  }

Wrap legacy back-end (PHP, Ruby)

app.type('CHANGE_NAME', {
  …
  process(action) {
    return send('POST', 'user/rename.php', {
      name: action.name,
      id: action.user
    })
  }

Send actions to client at any time

app.log.add({
  type: 'CHANGE_NAME',
  name: 'Looser',
  user: user.id
})

Server could undo any action on clients

{
  type: 'logux/undo',
  id: [1489328408677, '10:bfj58s0w', 0],
  reason: 'error'
}

Decorator for subscribing

const subscribe = require('react-logux/subscribe')

@subscribe(props => `users/${ props.id }`)
class User extends React.Component {
  …
}

Subscription on the server

app.channel('user/:id', (params, action, meta, creator) => {
  if (hasAccess(creator.userId, params.id)) return false
  return getUser(params.id).then(user => {
    app.log.add(userStateAction(user), {
      nodeIds: [creator.nodeId]
    })
  })
})

Part 7. Trade-offs.

Image: NASA

VODKA

What Logux is not

Not a CRDT

Not a DB

Framework to implement both

Problem 1: Log cleaning

store.log.on('preadd', (action, meta) => {
  if (action.type === 'READ_NOTIFICATION') {
    meta.reasons.push('notification:' + action.notificationId)
  }
})

onNotificationRemove (id) {
  store.log.removeReason('notification:' + id)
}

Problem 2: Complexity

Complexity

App

Net

Logux

Cleaning

App

Right case for Logux

Complexity

App

Net

Logux

Cleaning

App

Problem 3: not for large production sites​

Version: 0.2

Amplifr already uses it in production

Brave managers of Amplifr

30+ contributors

Image: Paolo Nespoli

Part 8. The result.

What is Logux?

Replaces AJAX and RESTful


Synchronizes logs (Redux actions)
between client and server

Where to use Logux

Any webapp
where users edit data a lot

Why Logux?

  • No need for loaders, sagas for AJAX
  • Faster UI with Optimistic UI
  • Live updates & Offline-first
  • Redux API and any DB

Questions?