An Introduction of NewSQL 

Story of TiDB

Agenda

  • Why another database?
  • What kind of database we want to use?
  • What is TiDB?
  • Let's talk about how to design, the principles, architecture and decisions
  • Introduce the core technologies for TiKV
  • Consensus Algorithm: Raft
  • New language: Go & Rust 
  • Hmm... let's go back to our real life

Why another database?

  • Relational Database is hard to provider better performance, hard to scale, although we have a lot of shading solutions (MySQL proxy) but distributed transactions and cross node join are not supported
  • NoSQL is easy to scale, but don't support SQL well and consistent transactions
  • Papers about Google Spanner and F1 describe a new database:  NoSQL like and maintains ACID transcations
  • TiDB and CockroachDB are NewSQL implementations

What kind of database we want to use?

  • It should support SQL of course
  • Must easy to scale, support fail-over and load balance
  • ACID transactions, we want strong consistency guarantee to help us write code easily 
  • We want the schema changes, secondary index and migrations are easy to do
  • Highly available always is important to us 

What is TiDB?

  • An open source NewSQL distributed database from PingCAP
    • BTW I love this name
  • Support MySQL protocol and support different persistence engine
  • Inspired by Google Spanner & F1
  • Based on K-V and support Facebook etcd
  • Created by Go & Rust
  • Distributed, consistent, scalable, SQL database

The principles or the philosophy

  • No data lost and system can automatically recover
  • Easy to use, easy to migrate data
  • On premise, cloud or container
  • Easy to dev and maintain - loose coupling
    • SQL layer
    • K-V layer 

The logical architecture

The logical architecture

Transcations

  • Inspired by Google's Percolator
  • Two-phase commit protocol with timestamp allocator
  • 3 columns:
    • lock: uncommitted trans write this cell and contains primary lock
    • write: stores the timestamp
    • data: store the data

Transcations

Transcations

MVCC - Multiversion concurrency control

  • Each transaction sees a snapshot of the database at the beginning time of this transaction. Any changes made by this transaction will not be seen by other transactions until the transaction is committed.
  • Data is tagged with versions in the following format: Key_version: value.
  • MVCC also ensures Lock-free snapshot reads.

Raft

  • Byzantine Generals Problem
  • We send data to server1, server2 and server 3 ...
  • If we can make server1 as the port and server2, server 3 just follow it
  • Leader, Candidate, Follower
  • Election
    • type and term
    • election timeout
    • forward log
Made with Slides.com