and other new ideas
Andrey Sitnik
Evil Martians
CRDT
for client-server communication
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/10590164/Man_Writing_a_Letter_by_Gabri%C3%ABl_Metsu.jpg)
I am working at
@sitnikcode
Consultancy boutique working with
@sitnikcode
Autoprefixer
PostCSS
Browserslist
Our known DevTools
@sitnikcode
Logux
AnyCable
Our new DevTools
@sitnikcode
WebSockets
Distributed
CRDT
Why CRDT is cool?
@sitnikcode
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/10590340/26xp-painting-01-superJumbo.jpg)
Why CRDT is not so scary?
@sitnikcode
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/10590342/Gustave_Courbet_-_Le_D%C3%A9sesp%C3%A9r%C3%A9__1843_.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5127760/DVCXOEVVwAAHQwD.jpg)
Part 1. Good technology?
A way to revolution
- New non-standard problem
- Old forgotten theory
- Find it fitting for other things
@sitnikcode
React
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582125/Cg36ZhG.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128423/b_w-1988.jpg)
React
Functional programming
Facebook notifications
@sitnikcode
- New non-standard problem
- Old forgotten theory
- Find it fitting for other things
Redux
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3298561/logo.png)
React Hot Loader
Elm
Redux
@sitnikcode
- New non-standard problem
- Old forgotten theory
- Find it fitting for other things
Part 2. Non-standard problem
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5130912/rowboat_ship_dock_painting_barrels_horse_boxes_classic_art-199712.jpg)
A very standard problem
Buy milk
0
Buy break
0
Buy whiskey
1
New to-do
Buy whiskey
1
Leave a comment
You were prepared for this
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128705/5bNYlqP.png)
@sitnikcode
Redux
const action = {
type: 'ADD_TASK',
name: 'Buy sugar',
likes: 0,
comment: ''
}
store.dispatch(action)
const state2 = { tasks: [X, Y] }
const state3 = reducers(state2, action)
state3 //=> { tasks: [X, Y, Z] }
@sitnikcode
dispatch({ type: 'ADD_STARTED' })
fetch('/tasks', opts)
.then(() => {
dispatch({ type: 'ADD_COMPLETE' })
})
.catch(() => {
dispatch({ type: 'ADD_FAILED' })
})
AJAX and loaders
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4219187/512x512.png3b853521-e78a-43a5-b4e3-aac827518645Large.jpg)
@sitnikcode
<Mutation mutation={ADD_TODO}>
{(addTodo, { loading, error }) => (
!loading && <Form onSubmit={addTodo} />
{ loading && <Loading /> }
{ error && <Error /> }
)}
</Mutation>
GraphQL and loaders
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4219187/512x512.png3b853521-e78a-43a5-b4e3-aac827518645Large.jpg)
@sitnikcode
You finished the project
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128756/7087challenge_accepted.jpg)
@sitnikcode
Small detail from client
I forgot a small detail.
Multiple users should edit the same task at the same moment.
Would that be so hard to add?
@sitnikcode
Collaboration is a popular feature
@sitnikcode
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/10590369/64760069e93084646c9eeea3_6263b255f76a3c2ef0f05a8f_appel-conversation-figma.png)
Especially with
remote work
Should we ask user to resolve conflict?
@sitnikcode
Buy whiskey
Buy vodka
Another user edited the same task.
Please choose the right revision:
Automatic edit conflict resolution
@sitnikcode
Buy whiskey
Buy vodka
Another user change the same task.
Please choose the right revision:
People shouldn’t suffer
Machines should suffer
@sitnikcode
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4059039/f129011be0142f4d0da04b8ee530b172.jpg)
We can add hacks around GraphQL…
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128819/Без_названия.jpeg)
@sitnikcode
… or you can go to the dungeons of CS
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128841/562MaZ4aJiU.jpg)
@sitnikcode
Distributed systems, 1982
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128843/enchiridion_farm_world_by_philelmago-db6tecg.png)
@sitnikcode
Part 3. Distributed systems
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5130913/Raid-on-New-York.jpg)
Requests
GraphQL/AJAX request
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4224739/ancient-maps-with-sea-monsters-2.jpg)
The things
we care about
Don’t care
@sitnikcode
Distributed system
Sync
@sitnikcode
Conflicts resolution
{
name: '伏特加',
likes: 50
}
{
name: '白兰地',
likes: 51
}
?
@sitnikcode
“Bad programmers worry about the code.
Good programmers worry about
data structures”
— Linus Torvalds
Source: https://lwn.net/Articles/193245/
Sending the whole state is a problem
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5131042/linus.faces22052.web_.jpg)
@sitnikcode
Event sourcing
set likes=50
set name='白兰地'
set likes=50
set name='白兰地'
@sitnikcode
Event = Redux action
set likes=50
{ type: 'SET_LIKES', value: 50 }
@sitnikcode
Important part
No AJAX requests
showLoader()
fetch('/profile', {
credentials: 'include',
method: 'POST',
form
}).then(r => {
hideLoadaer()
}).catch(e => {
…
})
@sitnikcode
No mutations or sagas
const ADD_TODO = gql`
mutation addTodo($type: String!) {
addTodo(type: $type) {
id
type
}
}
`;
const AddTodo = () => {
return (
<Mutation mutation={ADD_TODO}>
{addTodo => (
<button onClick={() => {
addTodo({ variables })
}}>
)}
</Mutation>
)
}
const createTask = write.bind(
null, taskList, taskList.push,
createTaskFailed)
function* watchCreateTask() {
while (true) {
let { payload } = yield take(CREATE_TASK);
yield fork(createTask, payload.task);
}
}
@sitnikcode
No loaders
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4219187/512x512.png3b853521-e78a-43a5-b4e3-aac827518645Large.jpg)
Continue to work
Save
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4221837/bouncer1.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4221839/bouncer2.jpg)
@sitnikcode
Keep WebSockets
WS
@sitnikcode
Send actions to server in background
Buy break
Buy break
dispatch({ type: 'DONE', taskId: 12 })
{ type: 'DONE', taskId: 12 }
WS
@sitnikcode
Optimistic UI, 2010
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4219187/512x512.png3b853521-e78a-43a5-b4e3-aac827518645Large.jpg)
Continue to work
Save
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4221837/bouncer1.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4221839/bouncer2.jpg)
Continue to work
Save
First mention: Google Wave Operational Transformation
@sitnikcode
Server re-sends actions to other clients
{ type: 'DONE', taskId: 12 }
WS
WS
@sitnikcode
Server could create Redux actions too
{ type: 'TRIAL_END' }
WS
@sitnikcode
Offline?
@sitnikcode
Show global warning
Buy milk
0
Buy bread
0
Buy whiskey
1
Offline
@sitnikcode
Redux already keeps actions in memory
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5131165/GarreauFaurot_DRA_01.png)
@sitnikcode
DB error?
@sitnikcode
Redux already has action undo
@sitnikcode
Server can ask client to undo a bad action
Name A
Rename to B
Name B
Rename to B
Undo
Name A
Part 4. The time
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5131191/Schepen_aan_lager_wal_-_Ships_running_aground_-_The__Ridderschap__and__Hollandia__in_trouble_in_the_Street_of_Gibraltar_1-3_March_1694__Ludolf_Backhuysen__1708_.jpg)
Distributed systems are not so easy
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3617202/how-to-draw-an-owl.jpg)
Step 1 Create an idea
Step2 Implement it
Every client creates actions
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5132865/63982f7d9a7da4331ffcebef09830384.jpg)
Action
Client 1
Client 2
Action
Action
Action
@sitnikcode
Actions order problem
Rename to A
Final: B
Rename to B
Rename to B
Final: A
Rename to A
@sitnikcode
Can we use time to order actions?
dispatch({ …, time: Date.now() })
actions.sortBy(i => i.time)
@sitnikcode
Time: not unique
Date.now() //=> 153247098000
Date.now() //=> 153247098000
@sitnikcode
Time: no consistency growth
Date.now() //=> 153247000011
// sudo ntpdate -s time.nist.gov
Date.now() //=> 153247000005
@sitnikcode
Time: no synchroniziation across clients
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128756/7087challenge_accepted.jpg)
13:15:00 UTC
12:10:00 UTC
1970-01-01 01:15:00
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5132893/45119094_0b62d0583950196052f526056f557c66_800.jpg)
@sitnikcode
Back to distributed systems dungeons
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128841/562MaZ4aJiU.jpg)
@sitnikcode
No time = no problem
@sitnikcode
Logical clock, 1978
let counter = 0
function now () {
counter += 1
return counter + ':' + uniqueClientId
}
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5133250/Lamport_5.jpg)
Part 5. Edit conflicts
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5132927/1-the-battle-of-trafalgar-english-school.jpg)
Merge changes in different fields
{
type: 'SET_LIKES',
taskId: 1,
likes: 50
}
{
type: 'CHANGE_NAME',
taskId: 1,
name: 'Buy bread'
}
@sitnikcode
But how to merge the same field?
{
type: 'SET_LIKES',
taskId: 1,
likes: 50
}
{
type: 'SET_LIKES',
taskId: 1,
likes: 50
}
?
@sitnikcode
Collaborative text editing?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5133333/googlecollab.png)
@sitnikcode
How to make actions atomic?
{
? // What put here
}
@sitnikcode
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128841/562MaZ4aJiU.jpg)
@sitnikcode
Back to distributed systems dungeons
CRDT, 2009
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3295643/RdoCb5i.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5133263/Marc_Shapiro.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5133264/Carlos_Baquero.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5133267/nmp155.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5133268/lDhiwLTf.jpg)
@sitnikcode
One of them after my talk in Porto
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5142752/IMG_20180518_103408__1_.jpg)
Conflict-free replicated data types
- Op-based counter
- G-Counter
- PN-Counter
- Non-negative Counter
- LWW-Register
- MV-Register
- G-Set
- 2P-Set
- LWW-element-Set
- PN-Set
- OR-Set
- Add-only monotonic DAG
- Add-remove Partial Order
- Replicated Growable Array
- Continuous sequence
…
@sitnikcode
Every type has
- Limits (last write wins)
- Operations (set key, remove key)
@sitnikcode
P2P: every node could create event
Name = A, time: 2
Name = B, time: 1
@sitnikcode
No master
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5133290/Eugène_Delacroix_-_Le_28_Juillet._La_Liberté_guidant_le_peuple.jpg)
Same final state after synchronization
Name = A
Sorry, last write wins
@sitnikcode
Name = A
Name = A
Name = A
Name = A
Name = A
Name = A
Op-based counter
{
type: 'SET_LIKES',
taskId: 1,
likes: 51
}
{
type: 'ADD_LIKE',
taskId: 1
},
{
type: 'REMOVE_LIKE',
taskId: 1
}
@sitnikcode
Conflicts after merging
delete 1
add h after 1
cat → aht
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4920458/aw7e9Lx_700b.jpg)
delete symbol at index 1
cat → at
add symbol h after index 1
cat → chat
c a t 1 2 3
String:
Indexes:
@sitnikcode
Linked list
@sitnikcode
'cat' → 'c'—'a'—'t'
↓
{ type: 'ADD', id: 1, symbol: 'c', after: null },
{ type: 'ADD', id: 2, symbol: 'a', after: 1 },
{ type: 'ADD', id: 3, symbol: 't', after: 2 }
Better merging
delete symbol
cat → at
add symbol h to
cat → chat
@sitnikcode
{ type: 'ADD', id: 1, symbol: 'c', after: null },
{ type: 'ADD', id: 2, symbol: 'a', after: 1 },
{ type: 'ADD', id: 3, symbol: 't', after: 2 }
{ type: 'HIDE', id: A }
{ type: 'ADD', id: A, symbol: 'h', after: A }
Better merging
cat → hat
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128756/7087challenge_accepted.jpg)
delete symbol
cat → at
add symbol h to
cat → chat
@sitnikcode
{ type: 'ADD', id: 1, symbol: 'c', after: null },
{ type: 'ADD', id: 2, symbol: 'a', after: 1 },
{ type: 'ADD', id: 3, symbol: 't', after: 2 },
{ type: 'HIDE', id: A },
{ type: 'ADD', id: A, symbol: 'h', after: A }
CRDT is not a silver bullet
CRDT
Real apps
@sitnikcode
CRDT is an inspiration for your custom types
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5133395/1200px-The_Inspiration_of_Saint_Matthew-Caravaggio__1602_.jpg)
Part 6. The result
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5133408/landscape-painting-people-mountains-ship-boat-28145-wallhere.com.jpg)
Overview
- Change state by actions
- action.time = distributedTime()
- Sync actions log between server and clients
- Time travel for undo and inserting in the middle of the history
- CRDT to make actions atomic
@sitnikcode
We made a collaborative ToDo app
Buy milk
0
Buy bread
0
Buy whiskey
1
New to-do
Buy whiskey
1
Leave a comment
But maybe we made something bigger?
New non-standard problem
Bring back old forgotten theory
Find it fitting for other things
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4219187/512x512.png3b853521-e78a-43a5-b4e3-aac827518645Large.jpg)
@sitnikcode
Click to turn off
Click to turn on
Final state on
Save on to DB
Save off to DB
Use case: AJAX, GraphQL
Wrong order because of the network
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4920458/aw7e9Lx_700b.jpg)
@sitnikcode
Save on to DB
Ignore
Turn off, time: 1
Final state on
Use case: CRDT
Turn on, time: 2
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/5128756/7087challenge_accepted.jpg)
@sitnikcode
AJAX & GraphQL were designed for
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3575227/mPV8Lo8.png)
Image: Nikita Prokopov
@sitnikcode
CRDT was designed for
Image: Nikita Prokopov
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3575245/P0Y2RR6.png)
@sitnikcode
40 % packet loss in China
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3580974/NjFlSJnLzC8.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4221711/Flag_of_China.png)
@sitnikcode
No stable Internet in NYC subway
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4219222/2015-07-23-mr-robot.jpg)
Image: Mr. Robot
Next billion users will not have good Internet
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/4219287/1398243193465_01_John_Stanmeyer.jpg)
Image: John Stanmeyer
Feature 1
- Works with poor connection
- More clients
- More money
@sitnikcode
CRDT supports also P2P
@sitnikcode
Option: Multi-master servers
@sitnikcode
Option: Mesh
@sitnikcode
Feature 2
- P2P-protocol by design
- More options
- Better scalability
@sitnikcode
Optimistic UI
Optimistic UI
Appling changes before server response
Offline support
Predictable conflict resolution
CRDT
@sitnikcode
Optimistic UI and live updates by design
@sitnikcode
“Every 100ms of additional latency costs Amazon 1% in profit”
More cases: pwastats.com
Optimistic UI → Performance → Money
@sitnikcode
Feature 3
- Optimistic UI by design
- Faster UI
- More money
@sitnikcode
Visual Studio
Collaborative editing is a trend
@sitnikcode
Figma
Trello
Local First
@sitnikcode
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/10590876/Captura_desde_2023-07-01_17-24-14.png)
You own your data
@sitnikcode
Thanks
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/10032507/01_Evil-Martians_Logo_v2.1_RGB_on-White.png)