Taming Complexity

with

Data-Oriented Programming

Yehonathan Sharvit

Oct 9, 2023

Huawei Research Center

Hong Kong

Complexity

... and to change.

What makes a system hard to understand...

Accidental

Essential

Software

system

State

Input

Output

State mutation

Side effects

Can it be sent over the wire?

Data

Abstract entitiy

No

Yes

Abstract entities vs. Data

Objects

Encapsulation

Mutation

Validation

Some state together with a set of procedures for accessing and manipulating that state.

public class PerformanceTest {
    
    public void insert10Times() {
        Document doc = new Document()
        .append("hello", "world");
        
        for (int i = 0; i < 10; i++) {
            collection.insertOne(doc);
        }
    }
}
// Somewhere inside the code of `insertOne` method
if(doc.get("_id") == null) {
    doc.put("_id", new ObjectId());
}

Mutation in the way

Execution error (MongoWriteException) at com.mongodb.client.internal.MongoCollectionImpl/executeSingleWriteRequest 
(MongoCollectionImpl.java:1032).
Write operation error on server localhost:27017. Write error: 
WriteError{code=11000, 
message='E11000 duplicate key error collection: db_for_test.40b37bce-142d-4f7d-b6a6-4580bdc55ca2 
index: _id_ dup key: { _id: ObjectId('65188e6c1cd8c84dfe5af4ab') }', details={}}.
public class PerformanceTest {
    
    public void insert10Times() {
        Document doc = new Document()
        .append("hello", "world");
        
        for (int i = 0; i < 10; i++) {
            collection.insertOne(doc);
            doc.remove("_id");
        }
    }
}
function moveToPreviousMonth(date) {
    date.setMonth(date.getMonth() - 1);
    // Set the day to either 28. 29, 30, or 31
    date.setDate(date.daysInMonth());
}
function moveToNextMonth(date) {
    // Set the day first, since the next month
    // might not have 31 days
    date.setDate(1);
    date.setMonth(date.getMonth() + 1);
}

Compulsive Validation

function moveToNextMonth(date) {
    date.setMonth(date.getMonth() + 1);
    date.setDate(1);
}
  • Objective
  • Relate to the system
  • Boring?

Simple

Tools

Easy

  • Subjective
  • Relate to the person
  • Cool

Simple

Tools

Easy

function easiness(levelOfExpertise) {
    if (logEnabled) {
        console.log("easiness called");
    }
    if (levelOfExpertise >= SENIOR) {
        return "easy";
    } else {
        return "hard";
    }
}
function easiness(levelOfExpertise) {
    // if (logEnabled) {
    //     console.log("easiness called:");
    // }
    if (levelOfExpertise >= SENIOR) {
        return "easy";
    } else {
        return "hard";
    }
}
(defn simplicity [system]
  (if log-enabled
    (println "simplicity called"))
  (if (stateful? system)
    :complex
    :simple))
(defn simplicity [system]
  #_(if log-enabled
      (println "simplicity called"))
  (if (stateful? system)
    :complex
    :simple))

Code generators

Autocompletion

Rigidity

Code duplication

Class diagrams

Type checking

Spaghetti code

Dependency hell

Wildcard versioning

TOOLS

SYSTEM

It works,

don't touch it!

Data-Oriented Programming

1

Separate between code and data

2

Never mutate data

3

Separate between data representation and data validation

Treat data as first-class citizen

Business logic

in DOP

State

State snapshot

Updated state snaphsot

State mutation

Side effects

Input

Output

You cannot step into the same river twice

– Heraclitus

Performance of immutable data structures

Correctness

Culture

Programming Language

Tooling

DOP in practice

Case studies

Case Benefits Challenges
Audyx (50K lines of frontend, 10 devs) - Natural for juniors
- Data manipulation for free
- Sometimes, it's harder to write code without mutation
- Not cool enough
Cycognito (microservice architecture, 50 devs) - Central schema repository
- Unit testing in isolation
- Speed of development
- Mistyping fields
- State is passed to internal functions instead of being accessed globally
- Lack of documentation
Clojure community (50,000 devs) - Simplicity
- Backward compatability
- Use DOP for abstract entities

Love is blind

  • Accidental vs. Essential Complexity
  • Abstract vs. Real Entities
  • Managing State
  • Simple vs. Easy
  • Hiding Complexity with Tools
  • Treat Data as Data
  • To type or not to type?

Wrapping up

Can it be sent over the wire?

It's data!

Use DOP

It's abstract!

Use OOP

No

Yes

Thank You!

Questions?

viebel@gmail.com

Taming Complexity with Data-Oriented Programming

By Yehonathan Sharvit

Taming Complexity with Data-Oriented Programming

  • 167