and other new ideas

Andrey Sitnik
Evil Martians

CRDT

for client-server communication

evilmartians.com

I am working at

@sitnikcode

Autoprefixer

PostCSS

Browserslist

The creator of

@sitnikcode

Part 1. What is next?

GraphQL is cool

@sitnikcode

What’s the next big idea?

RPC

SOAP

REST

GraphQL

?

@sitnikcode

And then I saw CRDT talk by Victor Grishenko

Swarm.js

What I believe in

RPC

SOAP

REST

GraphQL

CRDT

¯\_(ツ)_/¯
not 100% sure

@sitnikcode

Part 2. Non-standard problem

A very standard problem

Buy a milk

0

Buy a break

0

Buy a whiskey

1

New to-do

Buy a whiskey

1

Leave a comment

You were prepared for this

@sitnikcode

You finished the project

@sitnikcode

Small detail from client

I forgot a small detail.
Multiple users should edit the same task at the same moment.

Should not be that hard to add?

@sitnikcode

Should we ask user to merge conflict?

@sitnikcode

Buy a whiskey

Buy a vodka

Another user change the same task.
Please choose the right revision:

We need automaic edit conflict resolution

Buy a whiskey

Buy a vodka

Another user change the same task.
Please choose the right revision:

People shouldn’t suffer. Machines should suffer

We can add hacks around GraphQL …

@sitnikcode

… or you can go to the dungeons of CS

@sitnikcode

Distributed systems, 1982

@sitnikcode

Part 3. Distributed systems

Conflicts resolution

{
  name: '伏特加',
  likes: 50
}
{
  name: '白兰地',
  likes: 51
}

?

@sitnikcode

“Bad programmers worry about the code.
Good programmers worry about
data structures”

— Linus Torvalds

Source: https://lwn.net/Articles/193245/

Sending the whole state is a problem

@sitnikcode

Event sourcing

set likes=50
set name='白兰地'
set likes=50
set name='白兰地'

@sitnikcode

Event = Redux action

set likes=50
{ type: 'SET_LIKES', value: 50 }

@sitnikcode

Important part

No AJAX requests

showLoader()
fetch('/profile', {
  credentials: 'include',
  method: 'POST',
  form
}).then(r => {
  hideLoadaer()
}).catch(e => {
  …
})

@sitnikcode

No mutations or sagas

const ADD_TODO = gql`
  mutation addTodo($type: String!) {
    addTodo(type: $type) {
      id
      type
    }
  }
`;

const AddTodo = () => {
  return (
    <Mutation mutation={ADD_TODO}>
      {addTodo => (
        <button onClick={() => {
          addTodo({ variables })
        }}>
      )}
    </Mutation>
  )
}
const createTask = write.bind(
  null, taskList, taskList.push,
  createTaskFailed)

function* watchCreateTask() {
  while (true) {
    let { payload } = yield take(CREATE_TASK);
    yield fork(createTask, payload.task);
  }
}

@sitnikcode

No loaders

Continue to work

Save

@sitnikcode

Keep WebSockets

WS

@sitnikcode

Send Redux actions to server in background

Buy a break

Buy a break

dispatch({ type: 'DONE', taskId: 12 })
{ type: 'DONE', taskId: 12 }

WS

Optimistic UI, 2010

Continue to work

Save

Continue to work

Save

First mention: Google Wave Operational Transformation

@sitnikcode

Server re-send actions to other clients

{ type: 'DONE', taskId: 12 }

WS

WS

@sitnikcode

Server could create Redux actions too

{ type: 'TRIAL_END' }

WS

@sitnikcode

What if you need a loader?

Pessimistic UI

@sitnikcode

Use two actions for Pessimistic UI

{ type: 'PAY', … }
{ type: 'PAID' }

@sitnikcode

DB error?

@sitnikcode

Redux already has action undo in DevTools

Server can ask a client to undo wrong action

Name A

Rename to B

Name B

Rename to B

Undo

Name A

Part 4. The time

Distributed systems are not so easy

Step 1 Create an idea

Step2 Implement it

@sitnikcode

Every client could create action

Action

Client 1

Client 2

Action

Action

Action

@sitnikcode

Actions order problem

Rename to A

Final: B

Rename to B

Rename to B

Final: A

Rename to A

@sitnikcode

 Time to maintain the same action order?

dispatch({ …, time: Date.now() })
actions.sortBy(i => i.time)

Time: no synchroniziation across clients

13:15:00 UTC

12:10:00 UTC

1970-01-01 01:15:00

@sitnikcode

Return to distributed systems dungeons

@sitnikcode

No time = no problem

@sitnikcode

Logical clock, 1978

let counter = 0

function now () {
  counter += 1
  return counter + ':' + uniqueClientId
}

@sitnikcode

Part 5. Edit conflicts

Easy to merge changes in different fields

{
  type: 'SET_LIKES',
  taskId: 1,
  likes: 50
}
{
  type: 'CHANGE_NAME',
  taskId: 1,
  name: 'Buy a bread'
}

But how to merge changes of the same field?

{
  type: 'SET_LIKES',
  taskId: 1,
  likes: 50
}
{
  type: 'SET_LIKES',
  taskId: 1,
  likes: 50
}

?

Collaborative text editing?

@sitnikcode

How to make actions atomic?

{
  ? // What put here
}

@sitnikcode

Return to distributed systems dungeons

@sitnikcode

CRDT, 2009

hal.upmc.fr/inria-00555588/document

@sitnikcode

Conflict-free replicated data types

  • Op-based counter
  • G-Counter
  • PN-Counter
  • Non-negative Counter
  • LWW-Register
  • MV-Register
  • G-Set
  • 2P-Set
  • LWW-element-Set
  • PN-Set
  • OR-Set
  • Add-only monotonic DAG
  • Add-remove Partial Order
  • Replicated Growable Array
  • Continuous sequence

…

@sitnikcode

P2P: every node could create event

Name = A, time: 2

Name = B, time: 1

@sitnikcode

No master

Same final state after synchronization

Name = A

Name = A

Name = A

Name = A

Name = A

Name = A

Name = A

Sorry, last write wins

@sitnikcode

Op-based counter

{
  type: 'SET_LIKES',
  taskId: 1,
  likes: 51
}
{
  type: 'ADD_LIKE',
  taskId: 1
},
{
  type: 'REMOVE_LIKE',
  taskId: 1
}

@sitnikcode

Action for changing every symbol in text?

c a t
1 2 3

delete symbol at index 1
cat → at

add symbol h after index 1
cat → chat

String:

Indexes:

Operations change symbol indexes

c a t
1 2 3

delete symbol at index 1

String:

Indexes:

a t
1 2

String:

Indexes:

@sitnikcode

Conflicts after merging

delete 1

add h after 1
cat → aht

delete symbol at index 1
cat → at

add symbol h after index 1
cat → chat

@sitnikcode

+

Persistent indexes

delete symbol at index 1

a t
2 3

String:

Indexes:

@sitnikcode

c a t
1 2 3

String:

Indexes:

Persistent indexes

add symbol h after index 1

c  h  a  t
1 1.5 2  3

String:

Indexes:

@sitnikcode

c a t
1 2 3

String:

Indexes:

Better merging

delete 1

add h to 1.5
cat → hat

delete symbol at index 1
cat → at

add symbol h to index 1.5
cat → chat

@sitnikcode

CRDT is not a silver bullet

CRDT

Real apps

@sitnikcode

CRDT is inspiration for your custom types

Part 6. The result

Overview

  1. Redux, Vuex, etc
  2. action.time = distributedTime()
  3. Sync actions between server and clients
  4. Time travel for undo and inserting in the middle of the history
  5. CRDT to make actions atomic

@sitnikcode

We made a collaborative ToDo app

Buy a milk

0

Buy a break

0

Buy a whiskey

1

New to-do

Buy a whiskey

1

Leave a comment

@sitnikcode

But maybe we made something bigger?

New non-standard problem

Bring back old forgotten theory

Find it fitting for other things

@sitnikcode

AJAX & GraphQL were designed for

Image:  Nikita Prokopov

@sitnikcode

CRDT was designed for

Image:  Nikita Prokopov

@sitnikcode

40 % packet lost in China

@sitnikcode

No stable Internet in NY subway

Image: Mr. Robot

Next billion users will not have good Internet

Image: John Stanmeyer

Bad connection support because CRDT is P2P

Option: Multi-master servers

@sitnikcode

Optimistic UI and live updates by design

@sitnikcode

CRDT features

@sitnikcode

  1. Works on bad Internet → more users → more profit
  2. P2P by design → easy to scale
  3. Optimistic UI → faster UI → more profit

CRDT for regular apps

What I believe in

RPC

SOAP

REST

GraphQL

CRDT

¯\_(ツ)_/¯
not 100% sure

@sitnikcode

Thanks