Lesson Learned

Benchmarking NoSQL 

on the AWS Cloud

 

@Lynn Langit

Unstructured Data

Alternatives to Relational


Use case?

Title Text

Some
Solutions

AerospikeDB or Redis

Aerospike

  • Multi-threaded
  • Tuned for
    • In-memory
    • Persistence (SSD)
  • Auto Re-balancing
    • Clustering 
    • Sharding

Redis

  • Single-threaded
  • Tuned for
    • In-memory
  • Can be persisted
    • AOF
    • Snapshots
  • Manual Re-balancing

Redis Architecture

Aerospike Architecture

Demo

Hello Aerospike AMC

Benchmarking

using the AWS Cloud

1,000,000
TPS

Published Benchmark

Benchmark Details

  • Config - 10GB Network, 4-node CentOS
  • Data - 500GB, 50M records
  • Record Size - 100 bytes, 23 byte key, 10 fields
  • Used Aerospike benchmarking tool

TL;DR
Results
Validated

Link

Challenges

  • Disk IO
  • Memory
  • Network
  • Cloud

Solutions

Disk IO & Memory

Detail

Solutions

Network & Cloud (VPC)

 

  • Select best OS/AMI
  • Enhanced networking / HVM
  • AWS Placement groups
  • Add ENIs, 256k TPS/NIC
  • Pin process to NIC

Benchmarking Tools

Text

How
To
Get Started

Download Free Edition

Demo

SSH Aerospike tools

Concepts

Cluster

Node

Data

Namespace = Database

Set = Table

Key / Record = Row

Bin = Column

Data Types

  • Integer
  • String
  • List
  • BLOB
  • Map
  • LDT

How
To
Interact

Programming Model

  • Key-Value - put, get, scan...
  • Queries - select...
  • Aggregations - map, filter...
  • User-defined Functions

Data Types

  • Integer
  • String
  • List
  • BLOB
  • Map
	private static void writeSingleValue(AerospikeClient client,
			WritePolicy writePolicy, Key key) {
		Bin bin = new Bin("mybin", "myReadModifyWriteValue");
		client.put(writePolicy, key, bin);
		System.out.println("Wrote this new value (or bin): "+ key);
	}
	
	private static void addSingleValue(AerospikeClient client,
			WritePolicy writePolicy, Key key) {
		Key newKey = new Key("test","myAddSet","myAddKey");
		Bin counter = new Bin("mybin", 1);
		client.add(writePolicy, newKey, counter);
		System.out.println("Wrote this additional value (or bin):  "+ newKey);
	}

Demo

Aerospike Java client

Aerospike Tools for Eclipse

Conclusion

Use the right tool right

Aerospike Details

Benchmarking NoSQL on the AWS Cloud

By Lynn Langit

Benchmarking NoSQL on the AWS Cloud

Presentation given at NDCOslo on Benchmarking NoSQL databases on the AWS Cloud - for AerospikeDB and Redis

  • 1,962

More from Lynn Langit