Lynn Langit
Cloud Architect who codes
Lesson Learned
Benchmarking NoSQL
on the AWS Cloud
@Lynn Langit
Unstructured Data
Alternatives to Relational
Use case?
Title Text
Some
Solutions
AerospikeDB or Redis
Aerospike
- Multi-threaded
- Tuned for
- In-memory
- Persistence (SSD)
- Auto Re-balancing
- Clustering
- Sharding
Redis
- Single-threaded
- Tuned for
- In-memory
- Can be persisted
- AOF
- Snapshots
- Manual Re-balancing
Redis Architecture
Aerospike Architecture
Demo
Hello Aerospike AMC
Benchmarking
using the AWS Cloud
1,000,000
TPS
Published Benchmark
Benchmark Details
- Config - 10GB Network, 4-node CentOS
- Data - 500GB, 50M records
- Record Size - 100 bytes, 23 byte key, 10 fields
- Used Aerospike benchmarking tool
TL;DR
Results
Validated
Challenges
- Disk IO
- Memory
- Network
- Cloud
Solutions
Disk IO & Memory
Solutions
Network & Cloud (VPC)
- Select best OS/AMI
- Enhanced networking / HVM
- AWS Placement groups
- Add ENIs, 256k TPS/NIC
- Pin process to NIC
Benchmarking Tools
Text
How
To
Get Started
Demo
SSH Aerospike tools
Concepts
Cluster
Node
Data
Namespace = Database
Set = Table
Key / Record = Row
Bin = Column
Data Types
- Integer
- String
- List
- BLOB
- Map
- LDT
How
To
Interact
Programming Model
- Key-Value - put, get, scan...
- Queries - select...
- Aggregations - map, filter...
- User-defined Functions
Data Types
- Integer
- String
- List
- BLOB
- Map
private static void writeSingleValue(AerospikeClient client,
WritePolicy writePolicy, Key key) {
Bin bin = new Bin("mybin", "myReadModifyWriteValue");
client.put(writePolicy, key, bin);
System.out.println("Wrote this new value (or bin): "+ key);
}
private static void addSingleValue(AerospikeClient client,
WritePolicy writePolicy, Key key) {
Key newKey = new Key("test","myAddSet","myAddKey");
Bin counter = new Bin("mybin", 1);
client.add(writePolicy, newKey, counter);
System.out.println("Wrote this additional value (or bin): "+ newKey);
}
Demo
Aerospike Java client
Conclusion
Use the right tool right
Benchmarking NoSQL on the AWS Cloud
By Lynn Langit
Benchmarking NoSQL on the AWS Cloud
Presentation given at NDCOslo on Benchmarking NoSQL databases on the AWS Cloud - for AerospikeDB and Redis
