Bringing the Ancient tech back into the foreground

• ~20 years old stuff.
• The network is unreliable, planet-scale systems are not for the network. They are based on rendezvous.
• Service discovery should be conform to rendezvous.

ZEROCONF

ZEROCONF

ZEROCONF

• Used from HPC to WSNs
• We are not using that much in the cloud. (K8S, *DNS etc. handles these)
  • Still want full-blown zeroconf? use k8s with host network and deploy your app as a stateful/affine set or daemon set.
• Share network topology without intermediate 3rd party server or funky daemonized web apps.

ZEROCONF

• Used from HPC to WSNs
• We are not using that much in the cloud. (K8S, *DNS etc. handles these)
  • Still want full-blown zeroconf? use k8s with host network and deploy your app as a stateful/affine set or daemon set.
• Share network topology without intermediate 3rd party server or funky daemonized web apps.

At this point, we have established a network at the link-local level.

ZEROCONF

Want to go cross DC discovery?

ZEROCONF

Either zeroconf or another way you will discover services.

Artillery comes with built-in Zeroconf with both pure broadcast and UDP-anycast based methods.

NEXT UP: Forming a Cluster

FORMING CLUSTERS

REQUIREMENTS

• Forming a cluster and receiving membership events is needed to make a communication medium for nodes.
• Needed for database systems to look for replicas or status of the database instances.
• The protocol should be flexible, writing a library means that the user can have flexibility.
• The network shouldn't be congested with packets.
• Broadcast storm prevention, congestion, and all other things should be considered during the design.
• I need less packages and more work (bounded upper case traffic)

FORMING CLUSTERS

Guess which protocol suites suit to it?

FORMING CLUSTERS

Guess which protocol suites suit to it?

FORMING CLUSTERS

EPIDEMIC protocol (aka Gossip)

FORMING CLUSTERS

• SWIM-like FULLY custom message delivery based membership. I call them UDE(User-Defined Events). Serialize anything to bincode, or JSON or whatever...
• Limitation?
  • your router's frag scheme, or that(x, y, z) orchestration software.
• Back it up with Bastion. You have an undying a fully reliable AP distributed cluster with the custom message send events.
• No need to call someone when it fails.

• When I write it I should make it well tested.
• Do more checking with model checkers for the possible worst-case scenarios.
• I check worst-case scenarios, not best-case scenarios, that's why stochastic methods are the best.
• Meanwhile released Chaos Testing Harness: KAOS
• KAOS enabled Artillery to be tested.

FORMING CLUSTERS

KAOS: Chaotic Testing Harness

https://github.com/vertexclique/kaos

FORMING CLUSTERS

~~~ IN ACTION ~~~

FORMING CLUSTERS

Artillery comes with the mentioned membership protocol.

In the future for distributed actors, we can use gossip to exchange messages.

NEXT UP: Data Replication

DATA REPLICATION

DATA REPLICATION

REQUIREMENTS

• Now nodes are aware of each other and can send each other messages. (aka rumor-mongering)
• Network is in place.
• Need to replicate data between nodes.
  • Strong Consistency please
  • Don't use consensus that doesn't meant for data replication but agreement.
  • Faul tolerance strategy should be one-or-all style.
• Lightweight replicated state machines
• Log should be ordered.

DATA REPLICATION

RAFT?

DATA REPLICATION

RAFT?

PAXOS?

DATA REPLICATION

RAFT?

PAXOS?

Actual mixed workloads are like: 60% read, 25% update, 10% insert, and 5% scan

* YCSB mixed workload definition

DATA REPLICATION

*

* Wu, Y., Arulraj, J., Lin, J., Xian, R., & Pavlo, A. (2150). An Empirical Evaluation of In-Memory Multi-Version Concurrency Control.

It is hard to make "%60 read" successful with low latency in these circumstances with geo distribution.

Workloads are read mostly.

DATA REPLICATION

Solution:

CHAIN REPLICATION

DATA REPLICATION

How it works?

* Ordinary chain replication runs with tail queried reads

* Advanced mode (apportioned queries) runs with all nodes enabled with reads

DATA REPLICATION

• Implemented CR and implemented CRAQ to increase the read throughput!
• Artillery right now has lock free (except one that I need to refactor) CR implementation.
• Rust safety + concurrency + speed . The best combo.
• Tested (but not on cloud yet, I am on it)

DONE

DATA REPLICATION

• GC for replicated log
• Replicated state machine backup (aka checkpointing)

TODO

DATA REPLICATION

BENCHMARKS (RUST)

Even in synchronous mode it is

16000 ops/sec

Rust is my comrade in arms. Can't imagine how fast this will be with async code & executors.

TO SUM UP ARTILLERY IS

• A state-of-the-art distributed data library!
• Not a blockchain library or networking library, less deps.
• Exploits latest research techniques and other modern approaches.
• Designed for database systems & query engines
• YOU CAN NOW MAKE PURE DISTRIBUTED DATABASES WITH THE TOOLING OUT THERE IN RUST!
  • at planet scale
  • with code optimized for resources
  • blazingly fast
  • interested? read Artillery's wip doc:
    • https://artillery.bastion.rs/
  • Contribute to development:
    • https://github.com/bastion-rs/artillery
  • Sponsor me, please sustain OSS development:
    • https://github.com/sponsors/vertexclique

Q & A

github: @vertexclique

twitter: @vertexclique

site: vertexclique.com