Streams in DynamoDB

Hello Everyone

Welcome to CloudAffaire and this is Debjeet

In the last blog post, we have discussed Secondary Indexes in DynamoDB

https://cloudaffaire.com/secondary-indexes-in-dynamodb/

In this blog post, we are going to discuss streams in DynamoDB.

Streams in DynamoDB:

DynamoDB Streams is an optional feature that captures data modification events in DynamoDB tables. The data about these events appear in the stream in near real time, and in the order that the events occurred. Each event is represented by a stream record.

If you enable a stream on a table, DynamoDB Streams writes a stream record whenever one of the following events occurs:

• A new item is added to the table: The stream captures an image of the entire item, including all of its attributes.
• An item is updated: The stream captures the “before” and “after” image of any attributes that were modified in the item.
• An item is deleted from the table: The stream captures an image of the entire item before it was deleted.

Each stream record also contains the name of the table, the event timestamp, and other metadata. Stream records have a lifetime of 24 hours; after that, they are automatically removed from the stream.

Next, we are going to enable stream on a table in our local DynamoDB instance.

Step 1: Create a table named CloudAffaire

Step 2: Insert a single record

Step 3: Enable stream on the table

Note down the LatestStreamARN from the output which will be required to query the stream data.

You can also get the LatestStreamARN from the output of describe-table

LatestStreamARN: arn:aws:dynamodb:ddblocal:000000000000:table/CloudAffaire/stream/2018-12-31T06:33:56.877

You will also need shardId apart from LatestStreamARN to get stream data. To get the shardid, execute describe-stream with –stream-arn as LatestStreamARN

shardId : shardId-00000001546238036879-b81e7a3e

Step 4: Insert another record in the CloudAffaire table. This will be captured in stream.

To get the stream data using get-record, we need the shard iterator id. Next, we are going to get the shard iterator id using get-shard-iterator AWS CLI

Step 5: Get shard iterator id for last put-item operation

Note: Replace stream-arn and shard-id that you get from previous steps.

ShardIterator: 000|arn:aws:dynamodb:ddblocal:000000000000:table/CloudAffaire/stream/2018-12-31T06:33:56.877|c2hhcmRJZC0wMDAwMDAwMTU0NjIzODAzNjg3OS1iODFlN2EzZXwwMDAwMDAwMDAwMDAwMDAwMDAwMjd8MDAwMDAwMDAwMDAwMDAwMDAxNTQ2MjQ0Njg1NjA1

Step 6: Get stream data using get-record

Note: Stream are captured for any new item, update item and delete item operations.

You can also get the list of streams using list-stream AWS CLI

Delete the table:

Hope you have enjoyed this article. In the next blog post, we will explore the DynamoDB AWS console.

To get more details on DynamoDB, please refer below AWS documentation

https://docs.aws.amazon.com/dynamodb/index.html