Lesson Learned

Benchmarking NoSQL 

on the AWS Cloud

 

@Lynn Langit

Unstructured Data

Alternatives to Relational


Use case?

Title Text

Some
Solutions

AerospikeDB or Redis

Aerospike

  • Multi-threaded
  • Tuned for
    • In-memory
    • Persistence (SSD)
  • Auto Re-balancing
    • Clustering 
    • Sharding

Redis

  • Single-threaded
  • Tuned for
    • In-memory
  • Can be persisted
    • AOF
    • Snapshots
  • Manual Re-balancing

Redis Architecture

Aerospike Architecture

Demo

Hello Aerospike AMC

Benchmarking

using the AWS Cloud

1,000,000
TPS

Published Benchmark

Benchmark Details

  • Config - 10GB Network, 4-node CentOS
  • Data - 500GB, 50M records
  • Record Size - 100 bytes, 23 byte key, 10 fields
  • Used Aerospike benchmarking tool

TL;DR
Results
Validated

Link

Challenges

  • Disk IO
  • Memory
  • Network
  • Cloud

Solutions

Disk IO & Memory

Detail

Solutions

Network & Cloud (VPC)

 

  • Select best OS/AMI
  • Enhanced networking / HVM
  • AWS Placement groups
  • Add ENIs, 256k TPS/NIC
  • Pin process to NIC

Benchmarking Tools

Text

How
To
Get Started

Download Free Edition

Demo

SSH Aerospike tools

Concepts

Cluster

Node

Data

Namespace = Database

Set = Table

Key / Record = Row

Bin = Column

Data Types

  • Integer
  • String
  • List
  • BLOB
  • Map
  • LDT

How
To
Interact

Programming Model

  • Key-Value - put, get, scan...
  • Queries - select...
  • Aggregations - map, filter...
  • User-defined Functions

Data Types

  • Integer
  • String
  • List
  • BLOB
  • Map
	private static void writeSingleValue(AerospikeClient client,
			WritePolicy writePolicy, Key key) {
		Bin bin = new Bin("mybin", "myReadModifyWriteValue");
		client.put(writePolicy, key, bin);
		System.out.println("Wrote this new value (or bin): "+ key);
	}
	
	private static void addSingleValue(AerospikeClient client,
			WritePolicy writePolicy, Key key) {
		Key newKey = new Key("test","myAddSet","myAddKey");
		Bin counter = new Bin("mybin", 1);
		client.add(writePolicy, newKey, counter);
		System.out.println("Wrote this additional value (or bin):  "+ newKey);
	}

Demo

Aerospike Java client

Aerospike Tools for Eclipse

Conclusion

Use the right tool right

Aerospike Details