and other new ideas

Andrey Sitnik
Evil Martians

CRDT

for client-server communication

evilmartians.com

I am working at

@sitnikcode

Autoprefixer

PostCSS

Browserslist

The creator of

@sitnikcode

Part 1. What is next?

GraphQL is cool

@sitnikcode

What’s the next big idea?

RPC

SOAP

REST

GraphQL

?

@sitnikcode

And then I saw CRDT talk by Victor Grishenko

Swarm.js

What I believe in

RPC

SOAP

REST

GraphQL

CRDT

¯\_(ツ)_/¯
not 100% sure

@sitnikcode

Part 2. Non-standard problem

A very standard problem

Buy a milk

0

Buy a break

0

Buy a whiskey

1

New to-do

Buy a whiskey

1

Leave a comment

You were prepared for this

@sitnikcode

You finished the project

@sitnikcode

Small detail from client

I forgot a small detail.
Multiple users should edit the same task at the same moment.

Should not be that hard to add?

@sitnikcode

Should we ask user to merge conflict?

@sitnikcode

Buy a whiskey

Buy a vodka

Another user change the same task.
Please choose the right revision:

We need automaic edit conflict resolution

Buy a whiskey

Buy a vodka

Another user change the same task.
Please choose the right revision:

People shouldn’t suffer. Machines should suffer

We can add hacks around GraphQL …

@sitnikcode

… or you can go to the dungeons of CS

@sitnikcode

Distributed systems, 1982

@sitnikcode

Part 3. Distributed systems

Conflicts resolution

{
  name: '伏特加',
  likes: 50
}
{
  name: '白兰地',
  likes: 51
}

?

@sitnikcode

“Bad programmers worry about the code.
Good programmers worry about
data structures”

— Linus Torvalds

Source: https://lwn.net/Articles/193245/

Sending the whole state is a problem

@sitnikcode

Event sourcing

set likes=50
set name='白兰地'
set likes=50
set name='白兰地'

@sitnikcode

Event = Redux action

set likes=50
{ type: 'SET_LIKES', value: 50 }

@sitnikcode

Important part

No AJAX requests

showLoader()
fetch('/profile', {
  credentials: 'include',
  method: 'POST',
  form
}).then(r => {
  hideLoadaer()
}).catch(e => {
  …
})

@sitnikcode

No mutations or sagas

const ADD_TODO = gql`
  mutation addTodo($type: String!) {
    addTodo(type: $type) {
      id
      type
    }
  }
`;

const AddTodo = () => {
  return (
    <Mutation mutation={ADD_TODO}>
      {addTodo => (
        <button onClick={() => {
          addTodo({ variables })
        }}>
      )}
    </Mutation>
  )
}
const createTask = write.bind(
  null, taskList, taskList.push,
  createTaskFailed)

function* watchCreateTask() {
  while (true) {
    let { payload } = yield take(CREATE_TASK);
    yield fork(createTask, payload.task);
  }
}

@sitnikcode

No loaders

Continue to work

Save

@sitnikcode

Keep WebSockets

WS

@sitnikcode

Send Redux actions to server in background

Buy a break

Buy a break

dispatch({ type: 'DONE', taskId: 12 })
{ type: 'DONE', taskId: 12 }

WS

Optimistic UI, 2010

Continue to work

Save

Continue to work

Save

First mention: Google Wave Operational Transformation

@sitnikcode

Server re-send actions to other clients

{ type: 'DONE', taskId: 12 }

WS

WS

@sitnikcode

Server could create Redux actions too

{ type: 'TRIAL_END' }

WS

@sitnikcode

What if you need a loader?

Pessimistic UI

@sitnikcode

Use two actions for Pessimistic UI

{ type: 'PAY', … }
{ type: 'PAID' }

@sitnikcode

DB error?

@sitnikcode

Redux already has action undo in DevTools

Server can ask a client to undo wrong action

Name A

Rename to B

Name B

Rename to B

Undo

Name A

Part 4. The time

Distributed systems are not so easy

Step 1 Create an idea

Step2 Implement it

@sitnikcode

Every client could create action

Action

Client 1

Client 2

Action

Action

Action

@sitnikcode

Actions order problem

Rename to A

Final: B

Rename to B

Rename to B

Final: A

Rename to A

@sitnikcode

 Time to maintain the same action order?

dispatch({ …, time: Date.now() })
actions.sortBy(i => i.time)

Time: no synchroniziation across clients

13:15:00 UTC

12:10:00 UTC

1970-01-01 01:15:00

@sitnikcode

Return to distributed systems dungeons

@sitnikcode

No time = no problem

@sitnikcode

Logical clock, 1978

let counter = 0

function now () {
  counter += 1
  return counter + ':' + uniqueClientId
}

@sitnikcode

Part 5. Edit conflicts

Easy to merge changes in different fields

{
  type: 'SET_LIKES',
  taskId: 1,
  likes: 50
}
{
  type: 'CHANGE_NAME',
  taskId: 1,
  name: 'Buy a bread'
}

But how to merge changes of the same field?

{
  type: 'SET_LIKES',
  taskId: 1,
  likes: 50
}
{
  type: 'SET_LIKES',
  taskId: 1,
  likes: 50
}

?

Collaborative text editing?

@sitnikcode

How to make actions atomic?

{
  ? // What put here
}

@sitnikcode

Return to distributed systems dungeons

@sitnikcode

CRDT, 2009

hal.upmc.fr/inria-00555588/document

@sitnikcode

Conflict-free replicated data types

  • Op-based counter
  • G-Counter
  • PN-Counter
  • Non-negative Counter
  • LWW-Register
  • MV-Register
  • G-Set
  • 2P-Set
  • LWW-element-Set
  • PN-Set
  • OR-Set
  • Add-only monotonic DAG
  • Add-remove Partial Order
  • Replicated Growable Array
  • Continuous sequence

…

@sitnikcode

P2P: every node could create event

Name = A, time: 2

Name = B, time: 1

@sitnikcode

No master

Same final state after synchronization

Name = A

Name = A

Name = A

Name = A

Name = A

Name = A

Name = A

Sorry, last write wins

@sitnikcode

Op-based counter

{
  type: 'SET_LIKES',
  taskId: 1,
  likes: 51
}
{
  type: 'ADD_LIKE',
  taskId: 1
},
{
  type: 'REMOVE_LIKE',
  taskId: 1
}

@sitnikcode

Action for changing every symbol in text?

c a t
1 2 3

delete symbol at index 1
cat → at

add symbol h after index 1
cat → chat

String:

Indexes:

Operations change symbol indexes

c a t
1 2 3

delete symbol at index 1

String:

Indexes:

a t
1 2

String:

Indexes:

@sitnikcode

Conflicts after merging

delete 1

add h after 1
cat → aht

delete symbol at index 1
cat → at

add symbol h after index 1
cat → chat

@sitnikcode

+

Persistent indexes

delete symbol at index 1

a t
2 3

String:

Indexes:

@sitnikcode

c a t
1 2 3

String:

Indexes:

Persistent indexes

add symbol h after index 1

c  h  a  t
1 1.5 2  3

String:

Indexes:

@sitnikcode

c a t
1 2 3

String:

Indexes:

Better merging

delete 1

add h to 1.5
cat → hat

delete symbol at index 1
cat → at

add symbol h to index 1.5
cat → chat

@sitnikcode

CRDT is not a silver bullet

CRDT

Real apps

@sitnikcode

CRDT is inspiration for your custom types

Part 6. The result

Overview

  1. Redux, Vuex, etc
  2. action.time = distributedTime()
  3. Sync actions between server and clients
  4. Time travel for undo and inserting in the middle of the history
  5. CRDT to make actions atomic

@sitnikcode

We made a collaborative ToDo app

Buy a milk

0

Buy a break

0

Buy a whiskey

1

New to-do

Buy a whiskey

1

Leave a comment

@sitnikcode

But maybe we made something bigger?

New non-standard problem

Bring back old forgotten theory

Find it fitting for other things

@sitnikcode

AJAX & GraphQL were designed for

Image:  Nikita Prokopov

@sitnikcode

CRDT was designed for

Image:  Nikita Prokopov

@sitnikcode

40 % packet lost in China

@sitnikcode

No stable Internet in NY subway

Image: Mr. Robot

Next billion users will not have good Internet

Image: John Stanmeyer

Bad connection support because CRDT is P2P

Option: Multi-master servers

@sitnikcode

Optimistic UI and live updates by design

@sitnikcode

CRDT features

@sitnikcode

  1. Works on bad Internet → more users → more profit
  2. P2P by design → easy to scale
  3. Optimistic UI → faster UI → more profit

CRDT for regular apps

What I believe in

RPC

SOAP

REST

GraphQL

CRDT

¯\_(ツ)_/¯
not 100% sure

@sitnikcode

Thanks

CRDT and other new ideas for client-server communication

By Andrey Sitnik

CRDT and other new ideas for client-server communication

Right now we have great frameworks and technologies to create websites and mobile apps. We have great languages and databases for server back-end. But there are no many changes in a connection between client and server. Many application stops to work with a slow or unstable connection. Loaders block our UX on every small step. Most of the application doesn’t have offline support and live updates. We are not ready for next billion users with slow Internet. But also we are not ready for the unstable Internet in metro or low-signal LTE. Andrey Sitnik, the creator of PostCSS and Autoprefixer, will speak about new ideas in client-server communications. How the mix of ideas behind CRDT, distributed computing, and Elm/Redux could change the UX in most of applications.

  • 3,132

More from Andrey Sitnik