Inventory in S3

Hello Everyone

Welcome to CloudAffaire and this is Debjeet

In the last blog post, we have discussed encryption in S3.

https://cloudaffaire.com/encryption-in-s3/

In this blog post, we are going to discuss inventory in S3.

Inventory in S3:

You can use AWS S3 inventory feature to keep a track of your S3 bucket contents. The inventory provides you the list of objects in your bucket on a daily or weekly basis. AWS S3 inventory comes in the form of CSV (comma-separated values) or ORC (Apache optimized row columnar) files. AWS inventory contains inventory list files, manifest files and other metadata about your inventory.

Prerequisite for setting up S3 inventory:

1. Source bucket (S3 bucket for which you want to enable inventory)
2. Destination bucket (S3 bucket where the inventory files will be delivered, can use the same bucket as source)
3. Frequency (Daily or Weekly)
4. Bucket policy (Grant permission to S3 to write inventory file in destination bucket)

Next, we are going to set up inventory for an S3 bucket.

Prerequisite for this demo:

• A source S3 bucket.
• A destination S3 bucket.

Step 1: Login to AWS console and click ‘S3’ located under Storage.

Step 2: Click on the source bucket.

Note: Upload some files in your source bucket in order to list them in your inventory listing. We have uploaded 5 files inside our source bucket.

Warning: Additional charges apply depending upon the file size and bucket region.

Step 3: Navigate to ‘Inventory’ located under ‘Management’ and click ‘Add new’.

Step 4: Provide inventory name, destination bucket, frequency, file type, and metadata. Click ‘Save’.

Note: You can also encrypt your inventory files.

Inventory successfully saved and bucket policy also created to write inventory files in the destination bucket.

Note: Inventory provides eventual consistency and may not list all objects in your bucket properly. Also, the inventory files are not generated in real time and will be delivered as per frequency that you have given while setting up the inventory. In our case, we have to wait minimum 1 day before we can see the inventory files.

Step 5: Navigate to the folder with your inventory name in your destination bucket

Note: The data folder contains your inventory list files in .CSV format. The other folders contain manifest files and symlink files that have metadata about your inventory.

Inventory List files:

Manifest Files:

Symlink File:

For more details on the inventory metadata, please follow AWS S3 documentation

https://docs.aws.amazon.com/s3/index.html?id=docs_gateway#lang/en_us

To stop acquiring any cost, delete inventory and delete the buckets once the demo is completed.

Hope you have enjoyed this article, in the next blog post, we are going to discuss metrics in S3.