DynamoDB

NoSQL in the cloud

What is a DynamoDB?

Scalable 

Fast, Predictable Performance

Easy Administration

Built-in Fault Tolerance

Flexible 

Efficient Indexing 

Cost Effective

Integrated Monitoring

Is a Key - Value storage

with document support

DynamoDB Data Model Concept

Table

Data Model Concept

Item

Item

Item

Table

Data Model Concept

Item

Item

Item

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

DynamoDB is schema-less

each attribute defines it

max size 400kB

Data Model Concept

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Hash key

Hash key

Hash key

Hash keys:

  • mandatory for all items in table
  • defines data partitioning
  • represents key-value pattern

Data Model Concept

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Hash key

Range keys:

  • With them 1:N relation is possible
  • Enables rich queries like ==, <, >, begins, etc.
  • Together with hash key creates primary key composition

Range key

Indexes

  • Local Secondary Index (LSI)
  • Global Secondary Index (GSI)

Data Model Concept

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Hash key

Local secondary index:

  • Alternative range key with same hash
  • Total size of ale LSI cannot extend 10 GB
  • Query will fetch automatically all attributes from table
  • Read & write capacity consumed from a table

Range key

Data Model Concept

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Attribute

Hash key

Global secondary index:

  • New hash and/or range key
  • No size limit
  • Query over a table across all partitions
  • Query can only request projected attributes

Range key

DynamoDB data types

Scalar

  • Number
  • String
  • Binary
  • Boolean
  • Null

Multi-value

  • String set
  • Number set
  • Binary set

Document

  • list
  • map
{
    "day": "Monday",
    "unreadEmails": 42,
    "dayPhoto" : "dGhpcyB0ZXh0IGlzIGJhc2U2NC1lbmNvZGVk",
    "itemsOnMyDesk": [
        "coffee Cup",
        "telephone",
        {
            "pens": { "Quantity" : 3},
            "pencils": { "Quantity" : 2},
            "erasers": { "Quantity" : 1}
        }
    ]
}

Provisioning

===

Throughput

Read capacity

  • One strongly consistency read for items up to 4KB
  • For eventual consistency up to 8KB
  • How to:
    Number of items read x 4KB

Write capcity

  • One write per second for item up to 1KB

 

  • How to:

  Number of items per write x 1KB 

Example

498 * 20 bytes / 1024 bytes / 4kB ~ 2,43 = 3 Read capacity units 

498 * 20 bytes / 1024 bytes / 1kB ~ 9,73 = 10 Write capacity units 

Partitioning

Partition 1

Partition 2

Partition N

Table

DynamoDB automatically partition data using hash key

How to count/predict partitioning

considering size only

considering table read/write throughput

tableSizeInBytes / 10GB

readCapacity / 3000 + writeCapacity / 1000

Example

1) RCU: 2000 and WCU: 1000

 

2000 / 3000 + 1000 / 1000 = 1.(3) ==> 2 partitions

 

2) Table size = 1TB

 

1000 GB / 10 GB = 10 partitions,

then throughput based on example 1:

200 RCU and 100 WCU per partition

Example

How hash key may interfere

Development

How to?

Many SDK provided by AWS!

  • Java
  • Java script
  • .Net
  • Python
  • Ruby
  • iOS
  • Android

Table creation

There are 2 possibilities to create a DynamoDB table

via AWS Web console

via API

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient(
	new ProfileCredentialsProvider()));

CreateTableRequest request = new CreateTableRequest()
		.withTableName(tableName)
		.withKeySchema(keySchema)
		.withAttributeDefinitions(attributeDefinitions)
		.withProvisionedThroughput(new ProvisionedThroughput()
		    .withReadCapacityUnits(5L)
			.withWriteCapacityUnits(6L));

Table table = dynamoDB.createTable(request);

Demo

Table API

  • createTable
  • describeTable
  • updateTable
  • deleteTable
  • listTables
  • waitFor

Current API calls

Item API

  • putItem
  • updateItem
  • deleteItem
  • getItem
  • batchGetItem
  • batchWriteItem
  • query
  • scan

Demo

Transactions

ACID in DynamoDB world

  • Available via external lib https://github.com/awslabs/dynamodb-transactions
  • Atomicity is being performed with 2 additional tables
    • TxRecords
      Store information about transaction: status, items, updateCommands
    • TxImages
      Store information about item before transaction and a ref to TxRecord
  • Isolation:
    • Read and ignore lock, ignore pending transactions (weakest)
    • Read only commited items, ignore locks
    • Use read locks the same as implementing transactions

Some conclusion

Maybe?

Q & A

Links

Thanks!

Krzysztof Folwarczny

ImmobilienScout24 Tribe

pokój 5.56

DynamoDB - NoSQL in the cloud

By Krzysztof Folwarczny

DynamoDB - NoSQL in the cloud

Short introduction to Amazon DynamoDB

  • 573