How to list contents of an S3 bucket?
You can list contents of an S3 bucket using AWS CLI, boto3 or any other SDK provided by AWS.
List content of an S3 bucket using AWS CLI:
List all files (objects) and folders (keys) in an S3 bucket using AWS CLI
|
1 2 3 4 5 |
## List contents of an entire S3 bcuket aws s3 ls s3:// --recursive \ --human-readable \ --summarize |
List all files (objects) and folders (keys) in a specific folder (key)in S3 bucket using AWS CLI
|
1 2 3 4 5 |
## List contents of a specific directory in S3 bucket aws s3 ls s3:// --recursive \ --human-readable \ --summarize |
List all files (objects) with specific extension in an S3 bucket using AWS CLI
|
1 2 3 4 |
## List files with specific extension in S3 bucket aws s3 ls s3:// --recursive \ --human-readable | grep -e "\.txt$" |
List content of an S3 bucket using boto3:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
import boto3 s3_client = boto3.client('s3') bucket = ' prefix = ' paginator = s3_client.get_paginator('list_objects_v2') response_iterator = paginator.paginate(Bucket=bucket, Prefix=prefix) file_names = [] for response in response_iterator: for object_data in response['Contents']: key = object_data['Key'] if key.endswith('.json'): print(key) |
Keep <YOUR_FOLDER_PATH> blank if you want to list the entire S3 bucket, change the extension from .json to any other as per your requirements.
You can use below links to install and configure AWS CLI and boto3
https://cloudaffaire.com/how-to-install-aws-cli/
https://cloudaffaire.com/how-to-configure-aws-cli/
https://cloudaffaire.com/how-to-install-python-boto3-sdk-for-aws/
https://cloudaffaire.com/how-to-configure-python-boto3-sdk-for-aws/
Hope this solves your issue, keep learning and keep sharing.