What is Endatabas?

wat?

Endatabas is...

• A database startup, in a land where — and age when — the first rule of database startups is "don't"
• Sells Open Source
• Based on papers from 1970, 1974, 1980...
• Doubles down on SQL, a language everyone ... loves?
• Written in Common Lisp, despite all advice to the contrary

Ethos

(Advance warning: these are all about life, business, technology, and breakfast.)

Pain
is

Bad.

Honesty
is
Good.

Safety
is
Good.

Commodity
Thinking.

commodity

1. an economic good
2. something useful or valued, convenience, advantage
3. a good or service whose wide availability ...
4. subject to ready exchange or exploitation
5. obsolete : quantity, lot

com-modus

1. comfortable, commodious, suitable, useful, convenient, becoming
2. opportune, timely, tidy
3. pleasant, friendly

Commodities

Immutable
by
Default.

Empowering
is

Good.

Plastic
is
Powerful.

Design
should
Disappear.

Building
should be

Joyful.

Have
Fun.
 

Business

Third Wave Commercial Open Source

3rd Wave:

SaaS, mostly

Community

Control

Central

Control

endb

Proton

Minio

Neon

Metabase

Firefox

Cassandra

Spark

Arrow

Debian

Emacs

Clojure

3WCO = Silly?

It may seem naïve to assume we can make money building a SaaS database (DBaaS) in a world of Amazons, Googles, and Microsofts ... but there's big money thrown at much less adventurous products.

$30 million, eh

What we're hearing from CTOs

"I don't want to babysit a database."

"If you can give me a developer back, your DB is worth at least $100,000 / yr"

"Just solve a small, boring problem ... audits, reconciliation, ML model replay ..."

Product

Immutable Databases

A still more glorious dawn is ... uh, dawning.

What endb is not

• Blockchain ... anything
• Trustless, proof-of-something
• "Git for data" (branching, merging)
• Event Sourcing
• A new, disruptive query language
• NoSQL
• Graph-First

Endatabas aspires to be...?

Create a Commodity

Safety, Honesty

immutable

Painless, Plastic, Powerful

{ "first_name": "Ravi Chandra",

  "last_name": "Padmala",

  "email_address": "n@neena.ca",
  "is_alive": true,

  "age": 30,

  "height_cm": 185.2,

  "billing_address": {

    "address": "902 Main Street, Juiceville",

    "city": "Hyderabad",

    "state": "Telangana",

    "pin": "500002"

  },

  "shipping_address": {

    "address": "Nilenso Research, A-4 Research Drive",

    "city": "Hyderabad",

    "state": "Telangana",

    "pin": 500001"

  },

  ...

}

We've been here before

Okay, so

Yay Standards*

SQL:2023

...

SQL:2016

SQL:99

SQL:2011

Some
Examples
Please

Schemaless

Time Travel

SELECT name, system_time_start, system_time_end
  FROM users
  FOR SYSTEM_TIME AS OF 2018-01-02T00:00:00Z;

Erasure

ERASE FROM users WHERE username = 'shivam';
 

Documents

SQL:2023 (inspired)

INSERT INTO channels {id: 654,
                      name: "#on-grid",
                      access:
                        {mods: [100, 200, 300]
                         ...},
                      ...};
 

SELECT u.access.mods[2] FROM users u;

SELECT u FROM users u; -- select a document(s)

Sugar

SQL:2016 SQL/JSON

+

ECMAScript

LET c = [1, 2, 3];
LET d = 41;
INSERT INTO users {a: 1,          -- shorthand
                   b: [...c, 4],  -- spreads
                   c[d+1]: [3, 4] -- computed
                  }
 

SELECT u.a, u.b, u.c42 FROM users u;

Maybe: Paths

SQL:2016 SQL/JSON

+

ECMAScript

-- get an array of names:
SELECT c.events[*].name FROM channels c;

But Remember

schema migration

audits

reconciliation

GDPR compliance

replication

OLAP pipelines

failover

sharding

etc.

Internals

Scale and Scope

SQL parser

SQL compiler

execution

engine

columnar

storage

time travel

storage

vs.

compute

plastic SQL

(UX)

adaptive

indexes

bitemporal

indexes

Sidebar: Storage v Compute

storage

vs.

compute

Sidebar: Adaptive Indexes

"Light and Adaptive Indexing for Immutable Databases" by Håkan Råberg (Strange Loop 2022)

Sidebar: Temporality

(uni-temporal)

time travel

bitemporal

indexes

read-only

read-write

Decision-Making

• Two people
• History (3 years, 5 years)
• Computer Science research
• Market research
• Trial-and-error

3 years

5 years

CS Research

Yes, some of these papers are very old. (But not all.)

People were trying things back in 1985 that hardware has only caught up to now ... similar to LISP in the 1950s.

SBCL, you say.

Common Lisp gives us the plasticity to iterate quickly, throw things out, experiment. It's nice not to write a dynamic runtime from scratch.

SBCL is fast. C bindings are clunky, but they work.

Moving code to Rust "calcifies" it, so that's a late-bound activity.

Retro

Copeland's DSM paper was published in 1980.

Moderno

Apache Arrow is a modern avatar of the DSM.

Roadmap

Roadmap

• Powerful UX (think JavaScript, Python, Ruby, Clojure)
• Time Travel (SELECT ... AS OF ...)
• Querying Time ("when did X happen?")
• Erasure (GDPR and PIPEDA compliance)
• Kubernetes (open source users run clusters)
• DBaaS (you know, like, a product)
• HTAP (flex those columnar muscles)
• Storage vs. Compute (S3 becomes the disk)
• Assertions, Schema restrictions, Consistent Views...
• ...
• Graph Query? (SQL:2023, PGQ)
• Bitemporality? (SQL:2011 Application Time)