Amazon Kinesis Data Streams - Hands On Demo

Use the following command to list available options and services:

aws help
aws kinesis help

You will be using the Kinesis Data Streams service, so you can review the AWS CLI subcommands related to Kinesis Data Streams using the following command:

Step 1: Create a Stream

Your first step is to create a stream and verify that it was successfully created. Use the following command to create a stream named "Foo":

aws kinesis create-stream --stream-name Foo
aws kinesis describe-stream-summary --stream-name Foo
aws kinesis list-streams

Next, issue the following command to check on the stream's creation progress:

Initially the status will be CREATING, check back again in a few moments and you should see status change to ACTIVE.

You can also verify the existence of your new stream by using the list-streams command, as shown here:

Step 2: Put a Record

Now that you have an active stream, you are ready to put some data. For this tutorial, you will use the simplest possible command, put-record, which puts a single data record containing the text "testdata" into the stream:

aws kinesis put-record --stream-name Foo --partition-key 123 --data testdata
{
    "ShardId": "shardId-000000000000",
    "SequenceNumber": "49546986683135544286507457936321625675700192471156785154"
}

This command, if successful, will result in output similar to the following example:

Step 3: Get the Record

 

Before you can get data from the stream you need to obtain the shard iterator for the shard you are interested in. A shard iterator represents the position of the stream and shard from which the consumer (get-record command in this case) will read. You'll use the get-shard-iterator command, as follows:

 

aws kinesis get-shard-iterator --shard-id shardId-000000000000 --shard-iterator-type TRIM_HORIZON --stream-name Foo
aws kinesis get-records --shard-iterator

After you have got the ShardIterator run the below command replacing the Shard Iterator value

 

Step 4: Clean Up

Finally, you'll want to delete your stream to free up resources and avoid unintended charges to your account, as previously noted. Do this in practice any time you have created a stream and will not be using it because charges accrue per stream whether you are putting and getting data with it or not.

aws kinesis delete-stream --stream-name Foo
aws kinesis describe-stream-summary --stream-name Foo

Use describe-stream to check on deletion progress:

 

If the stream is deleted you will see some error like below.

An error occurred (ResourceNotFoundException) when calling the DescribeStreamSummary operation: Stream Foo under account not found.

Thanks

For

Watching

Amazon Kinesis Data Streams - Hands On Demo

By Deepak Dubey

Amazon Kinesis Data Streams - Hands On Demo

  • 335

More from Deepak Dubey