Question:
You can download a file via boto3 from a RequesterPays S3 bucket, as follows:
|
1 2 |
s3_client.download_file('aws-naip', 'md/2013/1m/rgbir/38077/{}'.format(filename), full_path, {'RequestPayer':'requester'}) |
What I can’t figure out is how to list the objects in the bucket… I get an authentication error when I try and call objects.all() on the bucket.
How can I use boto3 to enumerate the contents of a RequesterPays bucket? Please note this is a particular kind of bucket where the requester pays the S3 charges.
Answer:
From boto3, we can see that there is a #S3.Client.list_objects method. This can be used to enumerate objects:
|
1 2 3 4 5 6 7 8 |
import boto3 s3_client = boto3.client('s3') resp = s3_client.list_objects(Bucket='RequesterPays') # print names of all objects for obj in resp['Contents']: print 'Object Name: %s' % obj['Key'] |
Output:
|
1 2 3 4 |
Object Name: pic.gif Object Name: doc.txt Object Name: page.html |
If you are getting a 401 then make sure that IAM user calling the API has s3:GetObject permissions on the bucket.