How to retrieve bucket prefixes in a filesystem style using boto3


Doing something like the following:

Will return all the objects under ‘a_dummy_bucket’ bucket, like:

Is there any way of getting, in this case, ‘test1′,’test2’, ‘test3’, etc… without paginating over all results?
For reaching ‘test2’ I need 3 paginated calls, each one with 1000 keys to know that there is a ‘test2’, and then other 3 with 1000 keys to reach ‘test3’, and so on.

How can I get all these prefixes without paginating over all results?



I believe getting the Common Prefixes is what you are possibly looking for. Which can be done using this example:

AWS Documentation#Bucket.Get says the following regarding Common Prefixes:

A response can contain CommonPrefixes only if you specify a delimiter. When you do, CommonPrefixes contains all (if there are any) keys between Prefix and the next occurrence of the string specified by delimiter. In effect, CommonPrefixes lists keys that act like subdirectories in the directory specified by Prefix. For example, if prefix is notes/ and delimiter is a slash (/), in notes/summer/july, the common prefix is notes/summer/. All of the keys rolled up in a common prefix count as a single return when calculating the number of returns. See MaxKeys.

Type: String

Ancestor: ListBucketResult

Leave a Reply