DynamoDB
NoSQL in the cloud
What is a DynamoDB?
Scalable
Fast, Predictable Performance
Easy Administration
Built-in Fault Tolerance
Flexible
Efficient Indexing
Cost Effective
Integrated Monitoring
Is a Key - Value storage
with document support
DynamoDB Data Model Concept
Table
Data Model Concept
Item
Item
Item
Table
Data Model Concept
Item
Item
Item
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
DynamoDB is schema-less
each attribute defines it
max size 400kB
Data Model Concept
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Hash key
Hash key
Hash key
Hash keys:
- mandatory for all items in table
- defines data partitioning
- represents key-value pattern
Data Model Concept
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Hash key
Range keys:
- With them 1:N relation is possible
- Enables rich queries like ==, <, >, begins, etc.
- Together with hash key creates primary key composition
Range key
Indexes
- Local Secondary Index (LSI)
- Global Secondary Index (GSI)
Data Model Concept
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Hash key
Local secondary index:
- Alternative range key with same hash
- Total size of ale LSI cannot extend 10 GB
- Query will fetch automatically all attributes from table
- Read & write capacity consumed from a table
Range key
Data Model Concept
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Attribute
Hash key
Global secondary index:
- New hash and/or range key
- No size limit
- Query over a table across all partitions
- Query can only request projected attributes
Range key
DynamoDB data types
Scalar
- Number
- String
- Binary
- Boolean
- Null
Multi-value
- String set
- Number set
- Binary set
Document
- list
- map
{
"day": "Monday",
"unreadEmails": 42,
"dayPhoto" : "dGhpcyB0ZXh0IGlzIGJhc2U2NC1lbmNvZGVk",
"itemsOnMyDesk": [
"coffee Cup",
"telephone",
{
"pens": { "Quantity" : 3},
"pencils": { "Quantity" : 2},
"erasers": { "Quantity" : 1}
}
]
}
Provisioning
===
Throughput
Read capacity
- One strongly consistency read for items up to 4KB
- For eventual consistency up to 8KB
- How to:
Number of items read x 4KB
Write capcity
- One write per second for item up to 1KB
- How to:
Number of items per write x 1KB
Example
498 * 20 bytes / 1024 bytes / 4kB ~ 2,43 = 3 Read capacity units
498 * 20 bytes / 1024 bytes / 1kB ~ 9,73 = 10 Write capacity units
Partitioning
Partition 1
Partition 2
Partition N
Table
DynamoDB automatically partition data using hash key
How to count/predict partitioning
considering size only
considering table read/write throughput
tableSizeInBytes / 10GB
readCapacity / 3000 + writeCapacity / 1000
Example
1) RCU: 2000 and WCU: 1000
2000 / 3000 + 1000 / 1000 = 1.(3) ==> 2 partitions
2) Table size = 1TB
1000 GB / 10 GB = 10 partitions,
then throughput based on example 1:
200 RCU and 100 WCU per partition
Example
How hash key may interfere
Development
How to?
Many SDK provided by AWS!
- Java
- Java script
- .Net
- Python
- Ruby
- iOS
- Android
Table creation
There are 2 possibilities to create a DynamoDB table
via AWS Web console
via API
DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
new ProfileCredentialsProvider()));
CreateTableRequest request = new CreateTableRequest()
.withTableName(tableName)
.withKeySchema(keySchema)
.withAttributeDefinitions(attributeDefinitions)
.withProvisionedThroughput(new ProvisionedThroughput()
.withReadCapacityUnits(5L)
.withWriteCapacityUnits(6L));
Table table = dynamoDB.createTable(request);
Demo
Table API
- createTable
- describeTable
- updateTable
- deleteTable
- listTables
- waitFor
Current API calls
Item API
- putItem
- updateItem
- deleteItem
- getItem
- batchGetItem
- batchWriteItem
- query
- scan
Demo
Transactions
ACID in DynamoDB world
- Available via external lib https://github.com/awslabs/dynamodb-transactions
-
Atomicity is being performed with 2 additional tables
- TxRecords
Store information about transaction: status, items, updateCommands
- TxRecords
-
- TxImages
Store information about item before transaction and a ref to TxRecord
- TxImages
- Isolation:
- Read and ignore lock, ignore pending transactions (weakest)
- Read only commited items, ignore locks
- Use read locks the same as implementing transactions
Some conclusion
Maybe?
Q & A
Links
- SDK: http://tinyurl.com/mk3unhr
- DynamoDB Developer Guide: http://tinyurl.com/k27j7ad
- DynamoDB Local: http://tinyurl.com/mha4ca3
- Best Practices: http://tinyurl.com/o639oa2
- Limits cheatsheet: http://tinyurl.com/kcds9jj
- My sample java project: http://tinyurl.com/ntbx9lh
- Another java project: http://tinyurl.com/k2zdljh
Thanks!
Krzysztof Folwarczny
ImmobilienScout24 Tribe
pokój 5.56
DynamoDB - NoSQL in the cloud
By Krzysztof Folwarczny
DynamoDB - NoSQL in the cloud
Short introduction to Amazon DynamoDB
- 573