A new way of client-server conversation
Andrey Sitnik, Evil Martians
Russia is fun
Evil Martians front-end open source
Animakit
Autoprefixer
PostCSS
AnyCable
My first pull request …
… was not that successful
Revenge from Autoprefixer
An issue for Rework …
… was not successful either
Rework is dead
Open source advice
Maintainer — be polite
Contributor — don’t be afraid to fail
Front-end architecture …
State
Reducers
Components
… is great
State
Reducers
Components
Back-end …
State
Reducers
Components
View
Controller
Model
… is great too
State
Reducers
Components
View
Controller
Model
But the problem lies in between
State
Reducers
Components
View
Controller
Model
Communication
AJAX Requests
Chapter 1
Websites
Static pages
DHTML & jQuery
SPA
Live updates
Offline
New synchronization
AJAX
Two-field form
<form>
<input name="name"…>
<input name="color"…>
</form>
Server synchronization code
import { Actions } from 'flummox'
import api from '../api/labels-api'
export default class LabelsActions extends Actions {
async add(project, data) {
return await api.add(project, data).then( id => {
return [id, { ...data, id, project }]
})
}
async update(pr, id, data) {
return await api.update(pr, id, data).then(() => {
return [id, { ...data, id, project }]
})
}
async remove(project, id) {
return await api.remove(project, id).then(() => id )
}
}
import { OrderedMap } from 'immutable'
import { Store } from 'flummox'
export default class LabelsStore extends Store {
constructor(flux, data) {
super()
this.state = {
labels: OrderedMap(data.map(i => [parseInt(i.id), i]))
}
let actions = flux.getActionIds('labels')
this.register(actions.add, this.update)
this.register(actions.update, this.update)
this.register(actions.remove, this.remove)
}
remove(id) {
this.setState({
labels: this.state.labels.delete(parseInt(id))
})
}
update([id, data]) {
this.setState({
labels: this.state.labels.set(parseInt(id), data)
})
}
}
fetch('/comments.json').then(r => r.json()).then(list => {
setComments(list)
})
Simple fetch
showLoader()
fetch('/comments.json').then(r => r.json()).then(list => {
hideLoader()
setComments(list)
})
Fetch + loader
showLoader()
fetch('/comments.json').then(r => r.json()).then(list => {
hideLoader()
setComments(list)
}).catch(err => {
if (isNoNetwork(err)) {
showNetworkError(err)
} else {
showServerError(err)
}
})
Fetch + loader + error handling
Local connection in development
Image author: Nikita Prokopov
Real connection in production
Image author: Nikita Prokopov
Complexity curve with AJAX
Complexity
Features
More complicated problems
- Deduplication
- Relevance
- Protocol version validation
- Conflicts
Better world
SPA
Live Updates
Offline
Logux goals
- Reduce code size
- Support current ecosystems
- Live updates out of the box
- Optimistic UI & Offline-first
Idea
Chapter 2
Saint Petersburg — best bars in Russia
Conflict
Images author: Nikita Prokopov
{
title: '伏特加',
likes: 50
}
{
title: '白兰地',
likes: 51
}
?
Solution
“Bad programmers worry about the code.
Good programmers worry about
data structures”
— Linus Torvalds
Source: https://lwn.net/Articles/193245/
Event sourcing
Images author: Nikita Prokopov
['post:title', '伏特加']
['likes:set', 51]
['likes:remove', 1]
['post:title', '伏特加']
['likes:set', 51]
['post:title', '白兰地']
CRDT, 2009
Conflict-free Replicated Data Types
- Op-based counter
- G-Counter
- PN-Counter
- Non-negative Counter
- LWW-Register
- MV-Register
- G-Set
- 2P-Set
- LWW-element-Set
- PN-Set
- OR-Set
- Add-only monotonic DAG
- Add-remove Partial Order
- Replicated Growable Array
- Continuous sequence
CRDT in JS
Swarm.js
@gritzko
Event sourcing is everywhere
Redux
DevTools
log
CRDT
log
Backend DB
oplog
Log must be a first-class citizen
Logux inspiration
Redux
Swarm.js
+
Actions log on client
{
type: 'CHANGE_NAME',
user: 13,
name: 'New name'
}
Images author: Nikita Prokopov
Actions log on server
Images author: Nikita Prokopov
Synchronize logs
Images author: Nikita Prokopov
WebSocket
Synchronization is simple
1
2
3
4
5
Images author: Nikita Prokopov
— Give me new actions since “4”
Server actions
Images author: Nikita Prokopov
Between clients
Images author: Nikita Prokopov
Problems and Solutions
Chapter 3
How to write open source
Step 1 Create an idea
Step2 Implement it
Problem 1: order of actions
A
A
B
B
State: B
State: A
Images author: Nikita Prokopov
Sort by time?
{ action }, { time }
Distributed time is сomplicated
Questions about time: 1
const time1 = Date.now()
const time2 = Date.now()
time2 !== time1
A millisecond is a lot of time
Millisecond
Millisecond
Actions
Questions about time: 2
const time1 = Date.now()
const time2 = Date.now()
time2 >= time1
CloudFlare leap second problem
Questions about time: 3
const client = await client.askTime()
const server = Date.now()
diff(client, server) < HOUR
Wrong time zone
Solution 1: action ID
id: [1489328408677, '10:bfj58s0w', 0]
Milliseconds
Unique Node ID
User ID
Random string
Timestamp always increases
if (now < prevNow) {
now = prevNow
}
prevNow = now
Sequence number
[1489328408677, '10:…', 0]
[1489328408677, '10:…', 1]
[1489328408678, '10:…', 0]
Same milliseconds
Server sends current time
[
'connected',
[0, 0], // Protocol version
'server:h4vjdl', // Server unique ID
[
1475316481050, // "connect" message was received
1475316482879 // "connected" message was sent
]
]
Client calculates time difference
const roundTrip = now - startTime - authTime
const timeFix = startTime -
receivedTime +
roundTrip / 2
Client applies time difference
{ // Received
id,
time: id[0] + timeFix
}
{ // Sent
id,
time: id[0] - timeFix
}
Chinese Internet
40% packet loss
Problem 2: disconnect detection
ws.onclose = function () {
// Could be never called
}
Solution 2: ping command
> ['ping', 120]
< ['pong', 130]
Live connection status
if (loguxClient.connected) {
// Connection was tested
// at least 5 second ago
}
Problem 3: tabs conflict
Images author: Sebastien Gabriel & Nikita Prokopov
Duplicate
Action
Saved
I hate programming
I hate programming
I hate programming
It works!
I love programming
Solution 3: only the leader tab keeps WS
Images author: Sebastien Gabriel & Nikita Prokopov
leader
follower
Synchronized state between tabs
Modern front-end
Expectation
Reality
Other problems of distributed systems
Problems
Firewalls and WS
Log cleaning
Scaling
Outdated clients
Deduplication
Solutions
Force WSS
Action’s reasons of life
Peer-to-peer protocol
Subprotocol verification
Action ID
Logux API
Chapter 4
API
“Talk is cheap. Show me the code.”
— Linus Torvalds
Source: lkml.org/lkml/2000/8/25/132
Client: we care about client JS size
logux + redux ≈
12 KB
Client: Redux compatible API
import { createLoguxStore } from 'logux-redux'
const store = createLoguxStore(reducers,
preloadedState, undefined, {
url: 'wss://logux.example.com/',
userId: user.id,
subprotocol: '1.1.5',
credentials: user.token
})
Client: for “your friend” without Redux
import Client from 'logux-client/client'
const logux = new Client({
url: 'wss://logux.example.com/',
userId: user.id,
subprotocol: '1.1.5',
credentials: user.token
})
Client: local actions
dispatch({ type: 'MENU_OPEN' })
Client: cross-tab actions
add({
type: 'READ_NOTIFICATION',
id: notification
}, {
reasons
})
Client: server actions
add({
type: 'CHANGE_NAME',
user, name
}, {
sync: true,
reasons
})
Client: instead of AJAX
showLoader()
fetch('/profile', {
credentials: 'include',
method: 'POST',
form
}).then(r => {
hideLoadaer()
}).catch(e => {
…
})
add({
type: 'CHANGE_NAME',
user, name
}, {
sync: true,
reasons
})
Client: Optimistic UI by default
case 'CHANGE_NAME':
return { name: action.name, ...user }
Optimistic UI
Image author: Denys Mishunov
Client: server could undo action on error
{
type: 'logux/undo',
id: [1489328408677, '10:bfj58s0w', 0],
reason: 'error'
}
Logux Status: widget
Logux Status: favicon
Normal
Offline
Error
Logux Status: warning
Client: action storage
Local
Cached data
Not synchronized yet
Images author: Nikita Prokopov
Client: Pessimistic UI
Client: ask server for changes
add({ type: 'ASK_PAY' })
…
case 'ASK_PAY':
return { payment: 'waiting' }
Client: wait for server answer
case 'PAID':
return { payment: 'paid' }
Server: Node.js framework
import { Server } from 'logux-server'
const app = new Server({
subprotocol: '1.2.0',
supports: '1.x',
root: __dirname
})
Server: actions handling
app.type('CHANGE_NAME', {
access(action, meta, userId) {
return userId === action.user
}
process(action, meta) {
…
}
})
Server: use any DB or send any requests
process(action, meta) {
return db.user.id(action.user).then(u => {
if (isFirstOlder(u.lastChange, meta)) {
return u.changeName(action.name)
}
})
}
Server: send actions to client at any time
app.log.add({
type: 'CHANGE_NAME',
name: 'Looser',
user: user.id
})
RESTful vs. Logux
Logux
RESTful
URL, HTTP method
action.type
POST form, GET query
Action object
Request happened now
Action could be
created in the past
Wrap legacy back-end (PHP, Ruby)
process({ user, name }) {
return send('POST',
`user/${ user }/change.php`,
{ name })
}
Logux Protocol is open
Backend languages plan
What is Logux?
Replaces AJAX and RESTful
Synchronizes logs (like Redux actions)
between client and server
Why Logux?
- Less code
- Live updates out of the box
- Optimistic UI & Offline-first
- Redux API on the front-end
- Similar to RESTful on the back-end
Trade-offs
Chapter 4
Problem 1: Complexity
Complexity
App
Net
Logux
Cleaning
App
Right case for Logux
Complexity
App
Net
Logux
Cleaning
App
Problem 2: under construction
0.1 preview
protocol, core components
end-user API
channels, docs
0.2
autosubscribe, scaling
0.3
Amplifr already uses it in production
30 contributors
Want to reduce the code now?
Relay
Relay-like API for Logux is planned
Want to be Offline-first now?
Firebase
PouchDB
Gun.js
Want CRDT now?
Swarm.js
Questions?