A new way of client-server conversation
Andrey Sitnik, Evil Martians
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/2583932/Iv5oal8.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3325758/logux-text.png)
Russia is fun
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3586528/image001.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3586538/tolstoy1.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3586542/840full-crime-and-punishment-cover.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3586546/War-and-Peace-LARGER-ALTERNATIVE2.jpg)
My open source
Autoprefixer
PostCSS
Front-end architecture …
State
Reducers
Components
… is great
State
Reducers
Components
Back-end …
State
Reducers
Components
View
Controller
Model
… is great too
State
Reducers
Components
View
Controller
Model
But the problem lies in between
State
Reducers
Components
View
Controller
Model
Communication
AJAX Requests
Chapter 1
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3322392/tumblr_nxp68zDTq71ukmr73o1_500.gif)
Websites
Static pages
DHTML & jQuery
SPA
Live updates
Offline
New synchronization
AJAX
Two-field form
<form>
<input name="name"…>
<input name="color"…>
</form>
Server synchronization code
import { Actions } from 'flummox'
import api from '../api/labels-api'
export default class LabelsActions extends Actions {
async add(project, data) {
return await api.add(project, data).then( id => {
return [id, { ...data, id, project }]
})
}
async update(pr, id, data) {
return await api.update(pr, id, data).then(() => {
return [id, { ...data, id, project }]
})
}
async remove(project, id) {
return await api.remove(project, id).then(() => id )
}
}
import { OrderedMap } from 'immutable'
import { Store } from 'flummox'
export default class LabelsStore extends Store {
constructor(flux, data) {
super()
this.state = {
labels: OrderedMap(data.map(i => [parseInt(i.id), i]))
}
let actions = flux.getActionIds('labels')
this.register(actions.add, this.update)
this.register(actions.update, this.update)
this.register(actions.remove, this.remove)
}
remove(id) {
this.setState({
labels: this.state.labels.delete(parseInt(id))
})
}
update([id, data]) {
this.setState({
labels: this.state.labels.set(parseInt(id), data)
})
}
}
fetch('/comments.json').then(r => r.json()).then(list => {
setComments(list)
})
Simple fetch
showLoader()
fetch('/comments.json').then(r => r.json()).then(list => {
hideLoader()
setComments(list)
})
Fetch + loader
showLoader()
fetch('/comments.json').then(r => r.json()).then(list => {
hideLoader()
setComments(list)
}).catch(err => {
if (isNoNetwork(err)) {
showNetworkError(err)
} else {
showServerError(err)
}
})
Fetch + loader + error handling
Local connection in development
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3575227/mPV8Lo8.png)
Image author: Nikita Prokopov
Real connection in production
Image author: Nikita Prokopov
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3575245/P0Y2RR6.png)
Complexity curve with AJAX
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3590614/1-qqrhwnLRxCq0a34vTSVLQQ.png)
Complexity
Features
More complicated problems
- Deduplication
- Relevance
- Protocol version validation
- Conflicts
Better world
SPA
Live Updates
Offline
Logux goals
- Reduce code size
- Support current ecosystems
- Live updates out of the box
- Optimistic UI & Offline-first
Idea
Chapter 2
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3322154/tumblr_n5gd4iwekS1tzixowo1_500.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577234/8r1uwJmWI6xkYOZ3P2rdPA-wide.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577239/DHaVv.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577259/andrey-sitnik.jpg)
Saint Petersburg — best bars in Russia
Conflict
Images author: Nikita Prokopov
{
title: 'алкоголь',
likes: 50
}
{
title: 'водка',
likes: 51
}
?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577306/client1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577308/client2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577310/server.png)
Solution
“Bad programmers worry about the code.
Good programmers worry about
data structures”
— Linus Torvalds
Source: https://lwn.net/Articles/193245/
Event sourcing
Images author: Nikita Prokopov
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577306/client1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577308/client2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577310/server.png)
['post:title', 'алкоголь']
['likes:set', 51]
['likes:remove', 1]
['post:title', 'алкоголь']
['likes:set', 51]
['post:title', 'водка']
CRDT, 2009
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3295643/RdoCb5i.png)
Conflict-free Replicated Data Types
- Op-based counter
- G-Counter
- PN-Counter
- Non-negative Counter
- LWW-Register
- MV-Register
- G-Set
- 2P-Set
- LWW-element-Set
- PN-Set
- OR-Set
- Add-only monotonic DAG
- Add-remove Partial Order
- Replicated Growable Array
- Continuous sequence
CRDT in JS
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3295699/site-logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3295701/ade720cb7a20a6edb4d694fa43ac3a77.jpg)
Swarm.js
@gritzko
Event sourcing is everywhere
Redux
DevTools
log
CRDT
log
Backend DB
oplog
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3580904/giphy.gif)
Log must be a first-class citizen
Logux inspiration
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3298561/logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3295699/site-logo.png)
Redux
Swarm.js
+
Actions log on client
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577308/client2.png)
{
type: 'CHANGE_NAME',
user: 13,
name: 'New name'
}
Images author: Nikita Prokopov
Actions log on server
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577310/server.png)
Images author: Nikita Prokopov
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3580948/5818635fa29df1581f442db2.jpg)
Synchronize logs
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577308/client2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577310/server.png)
Images author: Nikita Prokopov
WebSocket
Synchronization is simple
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577308/client2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577310/server.png)
1
2
3
4
5
Images author: Nikita Prokopov
— Give me new actions since “4”
Server actions
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577308/client2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577310/server.png)
Images author: Nikita Prokopov
Between clients
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577308/client2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577310/server.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577306/client1.png)
Images author: Nikita Prokopov
Problems and Solutions
Chapter 3
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3580964/tumblr_noxf76RRlP1rnvb0co1_500.gif)
How to write open source
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3617202/how-to-draw-an-owl.jpg)
Step 1 Create an idea
Step2 Implement it
Problem 1: order of actions
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577308/client2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577306/client1.png)
A
A
B
B
State: B
State: A
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3580991/t6LRJBB.png)
Images author: Nikita Prokopov
Sort by time?
{ action }, { time }
Distributed time is сomplicated
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3581131/QuantumBioPhysics_html_m316aad5f.png)
Questions about time: 1
const time1 = Date.now()
const time2 = Date.now()
time2 !== time1
A millisecond is a lot of time
Millisecond
Millisecond
Actions
Questions about time: 2
const time1 = Date.now()
const time2 = Date.now()
time2 >= time1
CloudFlare leap second problem
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582712/XmdddHL.png)
Questions about time: 3
const client = await client.askTime()
const server = Date.now()
diff(client, server) < HOUR
Wrong time zone
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3581724/zone.jpg)
Solution 1: action ID
id: [1489328408677, '10:bfj58s0w', 0]
Milliseconds
Unique Node ID
User ID
Random string
Timestamp always increases
if (now < prevNow) {
now = prevNow
}
prevNow = now
Sequence number
[1489328408677, '10:…', 0]
[1489328408677, '10:…', 1]
[1489328408678, '10:…', 0]
Same milliseconds
Server sends current time
[
'connected',
[0, 0], // Protocol version
'server:h4vjdl', // Server unique ID
[
1475316481050, // "connect" message was received
1475316482879 // "connected" message was sent
]
]
Client calculates time difference
const roundTrip = now - startTime - authTime
const timeFix = startTime -
receivedTime +
roundTrip / 2
Client applies time difference
{ // Received
id,
time: id[0] + timeFix
}
{ // Sent
id,
time: id[0] - timeFix
}
Chinese Internet
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3580974/NjFlSJnLzC8.jpg)
40% packet loss
Problem 2: disconnect detection
ws.onclose = function () {
// Could be never called
}
Solution 2: ping command
> ['ping', 120]
< ['pong', 130]
Live connection status
if (loguxClient.connected) {
// Connection was tested
// at least 5 second ago
}
Problem 3: tabs conflict
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577310/server.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582060/tabs.png)
Images author: Sebastien Gabriel & Nikita Prokopov
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582072/m4xpTE7.png)
Duplicate
Action
Saved
I hate programming
I hate programming
I hate programming
It works!
I love programming
Solution 3: only the leader tab keeps WS
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577310/server.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582060/tabs.png)
Images author: Sebastien Gabriel & Nikita Prokopov
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582072/m4xpTE7.png)
leader
follower
Synchronized state between tabs
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582060/tabs.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582125/Cg36ZhG.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582125/Cg36ZhG.png)
Modern front-end
Expectation
Reality
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3617217/math.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3617234/c8d53a293c05a8ccf65def966a6de913.jpg)
Other problems of distributed systems
Problems
Firewalls and WS
Log cleaning
Scaling
Outdated clients
Deduplication
Solutions
Force WSS
Action’s reasons of life
Peer-to-peer protocol
Subprotocol verification
Action ID
Logux API
Chapter 4
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3322028/ac2eaa8c06712db692ecb071b8b6ebcf.gif)
API
“Talk is cheap. Show me the code.”
— Linus Torvalds
Source: lkml.org/lkml/2000/8/25/132
Client: we care about client JS size
logux + redux ≈
12 KB
Client: Redux compatible API
import { createLoguxStore } from 'logux-redux'
const store = createLoguxStore(reducers,
preloadedState, undefined, {
url: 'wss://logux.example.com/',
userId: user.id,
subprotocol: '1.1.5',
credentials: user.token
})
Client: for “your friend” without Redux
import Client from 'logux-client/client'
const logux = new Client({
url: 'wss://logux.example.com/',
userId: user.id,
subprotocol: '1.1.5',
credentials: user.token
})
Client: local actions
dispatch({ type: 'MENU_OPEN' })
Client: cross-tab actions
add({
type: 'READ_NOTIFICATION',
id: notification
}, {
reasons
})
Client: server actions
add({
type: 'CHANGE_NAME',
user, name
}, {
sync: true,
reasons
})
Client: instead of AJAX
showLoader()
fetch('/profile', {
credentials: 'include',
method: 'POST',
form
}).then(r => {
hideLoadaer()
}).catch(e => {
…
})
add({
type: 'CHANGE_NAME',
user, name
}, {
sync: true,
reasons
})
Client: Optimistic UI by default
case 'CHANGE_NAME':
return { name: action.name, ...user }
Optimistic UI
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3299063/optimistic-ui2-large-opt.png)
Image author: Denys Mishunov
Client: server could undo action on error
{
type: 'logux/undo',
id: [1489328408677, '10:bfj58s0w', 0],
reason: 'error'
}
Logux Status: widget
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3321815/screen_shot_2016-12-09_at_09.55.50_1024.png)
Logux Status: favicon
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3299120/favicon.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3299121/offline.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3299123/error.png)
Normal
Offline
Error
Logux Status: warning
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3298757/68747470733a2f2f692e696d6775722e636f6d2f6730414f31494a2e706e67.png)
Client: action storage
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3577308/client2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582072/m4xpTE7.png)
Local
Cached data
Not synchronized yet
Images author: Nikita Prokopov
Client: Pessimistic UI
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582556/14317911768789.jpg)
Client: ask server for changes
add({ type: 'ASK_PAY' })
…
case 'ASK_PAY':
return { payment: 'waiting' }
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3617160/512x512.png9365038d-7059-49a3-a28e-258c054cf528Large.jpg)
Client: wait for server answer
case 'PAID':
return { payment: 'paid' }
Server: Node.js framework
import { Server } from 'logux-server'
const app = new Server({
subprotocol: '1.2.0',
supports: '1.x',
root: __dirname
})
Server: actions handling
app.type('CHANGE_NAME', {
access(action, meta, userId) {
return userId === action.user
}
process(action, meta) {
…
}
})
Server: use any DB or send any requests
process(action, meta) {
return db.user.id(action.user).then(u => {
if (isFirstOlder(u.lastChange, meta)) {
return u.changeName(action.name)
}
})
}
Server: send actions to client at any time
app.log.add({
type: 'CHANGE_NAME',
name: 'Looser',
user: user.id
})
RESTful vs. Logux
Logux
RESTful
URL, HTTP method
action.type
POST form, GET query
Action object
Request happened now
Action could be
created in the past
Wrap legacy back-end (PHP, Ruby)
process({ user, name }) {
return send('POST',
`user/${ user }/change.php`,
{ name })
}
Logux Protocol is open
Backend languages plan
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3298673/comingsoon.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3298673/comingsoon.png)
What is Logux?
Replaces AJAX and RESTful
Synchronizes logs (like Redux actions)
between client and server
Why Logux?
- Less code
- Live updates out of the box
- Optimistic UI & Offline-first
- Redux API on the front-end
- Similar to RESTful on the back-end
Trade-offs
Chapter 4
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3582686/giphy.gif)
Problem 1: Complexity
Complexity
App
Net
Logux
Cleaning
App
Right case for Logux
Complexity
App
Net
Logux
Cleaning
App
Problem 2: under construction
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3584832/dog.jpg)
0.1 preview
protocol, core components
end-user API
docs
0.2
autosubscribe, scaling
0.3
Amplifr already uses it in production
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3299632/amplifr.jpg)
>30 contributors
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585003/3617409.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3584970/11071.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585014/13414205.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3584991/29658.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3584993/63065.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3584996/95453.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3584999/141334.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585001/191113.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3584985/19343.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585016/775692.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585019/1193817.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585020/1282980.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585021/1452165.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585023/1506905.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585024/1592680.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585025/2734841.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585027/2822264.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585028/3505878.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585030/3762784.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585031/3992528.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585032/4654180.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585034/4983850.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585035/6660939.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3585037/20017097.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3584960/745.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3590621/4834604.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3599604/9166883.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3614830/1516722.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3635886/16529522.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3635888/22623917.jpg)
Want to reduce the code now?
Relay
Relay-like API for Logux is planned
Want to be Offline-first now?
Firebase
PouchDB
Gun.js
Want CRDT now?
Swarm.js
![](https://s3.amazonaws.com/media-p.slid.es/uploads/467124/images/3295699/site-logo.png)
Questions?
Logux, a new way of client-server conversation (Chinese)
By Andrey Sitnik
Logux, a new way of client-server conversation (Chinese)
- 3,987