Taming Complexity
with
Data-Oriented Programming
Yehonathan Sharvit
Oct 9, 2023
Huawei Research Center
Hong Kong
Complexity
... and to change.
What makes a system hard to understand...
Accidental
Essential
Software
system
State
Input
Output
State mutation
Side effects
Can it be sent over the wire?
Data
Abstract entitiy
No
Yes
Abstract entities vs. Data
Objects
Encapsulation
Mutation
Validation
Some state together with a set of procedures for accessing and manipulating that state.
public class PerformanceTest {
public void insert10Times() {
Document doc = new Document()
.append("hello", "world");
for (int i = 0; i < 10; i++) {
collection.insertOne(doc);
}
}
}
// Somewhere inside the code of `insertOne` method
if(doc.get("_id") == null) {
doc.put("_id", new ObjectId());
}
Mutation in the way
Execution error (MongoWriteException) at com.mongodb.client.internal.MongoCollectionImpl/executeSingleWriteRequest
(MongoCollectionImpl.java:1032).
Write operation error on server localhost:27017. Write error:
WriteError{code=11000,
message='E11000 duplicate key error collection: db_for_test.40b37bce-142d-4f7d-b6a6-4580bdc55ca2
index: _id_ dup key: { _id: ObjectId('65188e6c1cd8c84dfe5af4ab') }', details={}}.
public class PerformanceTest {
public void insert10Times() {
Document doc = new Document()
.append("hello", "world");
for (int i = 0; i < 10; i++) {
collection.insertOne(doc);
doc.remove("_id");
}
}
}
function moveToPreviousMonth(date) {
date.setMonth(date.getMonth() - 1);
// Set the day to either 28. 29, 30, or 31
date.setDate(date.daysInMonth());
}
function moveToNextMonth(date) {
// Set the day first, since the next month
// might not have 31 days
date.setDate(1);
date.setMonth(date.getMonth() + 1);
}
Compulsive Validation
function moveToNextMonth(date) {
date.setMonth(date.getMonth() + 1);
date.setDate(1);
}
- Objective
- Relate to the system
- Boring?
Simple
Tools
Easy
- Subjective
- Relate to the person
- Cool
Simple
Tools
Easy
function easiness(levelOfExpertise) {
if (logEnabled) {
console.log("easiness called");
}
if (levelOfExpertise >= SENIOR) {
return "easy";
} else {
return "hard";
}
}
function easiness(levelOfExpertise) {
// if (logEnabled) {
// console.log("easiness called:");
// }
if (levelOfExpertise >= SENIOR) {
return "easy";
} else {
return "hard";
}
}
(defn simplicity [system]
(if log-enabled
(println "simplicity called"))
(if (stateful? system)
:complex
:simple))
(defn simplicity [system]
#_(if log-enabled
(println "simplicity called"))
(if (stateful? system)
:complex
:simple))
Code generators
Autocompletion
Rigidity
Code duplication
Class diagrams
Type checking
Spaghetti code
Dependency hell
Wildcard versioning
TOOLS
SYSTEM
It works,
don't touch it!
Data-Oriented Programming
1
Separate between code and data
2
Never mutate data
3
Separate between data representation and data validation
Treat data as first-class citizen
Business logic
in DOP
State
State snapshot
Updated state snaphsot
State mutation
Side effects
Input
Output
You cannot step into the same river twice
– Heraclitus
Performance of immutable data structures
Correctness
Culture
Programming Language
Tooling
DOP in practice
Case studies
Case | Benefits | Challenges |
---|---|---|
Audyx (50K lines of frontend, 10 devs) | - Natural for juniors - Data manipulation for free |
- Sometimes, it's harder to write code without mutation - Not cool enough |
Cycognito (microservice architecture, 50 devs) | - Central schema repository - Unit testing in isolation - Speed of development |
- Mistyping fields - State is passed to internal functions instead of being accessed globally - Lack of documentation |
Clojure community (50,000 devs) | - Simplicity - Backward compatability |
- Use DOP for abstract entities |
Love is blind
- Accidental vs. Essential Complexity
- Abstract vs. Real Entities
- Managing State
- Simple vs. Easy
- Hiding Complexity with Tools
- Treat Data as Data
- To type or not to type?
Wrapping up
Can it be sent over the wire?
It's data!
Use DOP
It's abstract!
Use OOP
No
Yes
Thank You!
Questions?
viebel@gmail.com
Taming Complexity with Data-Oriented Programming
By Yehonathan Sharvit
Taming Complexity with Data-Oriented Programming
- 167