Question:
I want to secure files located in S3 buckets, and ensure that no sensitive files are being shared.
I am aware of various ways (in the console and using scripts) to view which buckets have public permissions.
However, objects can be granted anonymous read permissions even when they are in a private bucket.
It can be hard to track/audit files/objects which are publicly readable because I cannot see any way to find them other than browsing through every single path in the AWS console.
Is there a way to list all the objects which have anonymous (read) permissions in a bucket? Any method would be fine, including AWSCLI, Boto, REST etc.
I considered trying to use an anonymous AWSCLI profile but this would not allow listing bucket contents so it could only be used to test files individually.
I suppose I could create a script using Boto (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#objectacl). Is that the only way, or is there an official method such as an AWSCLI command?
Answer:
What you’re talking about is Object ACLs. In the spirit of job-to-be-done, I want to point out that you can configure a bucket to deny public object ACLs. This is probably among the best enterprise practices for prevention. Among the best enterprise practices for auditing and verifying continuously is described here.
Update: If you’re interested in monitoring & auditing bucket-level ACLs, take a look at this managed AWS Config solution.
However, if you’re looking for a bash script/tool using the aws-cli (which is the tag for this question), this will do the trick:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
bucket="my-bucket-name" search_term="AllUsers" for key in $(aws s3 ls --recursive s3://$bucket/ | awk '{$1=$2=$3=""; print $0}' | sed 's/^[ \t]*//'); do acl=$(aws s3api get-object-acl --bucket $bucket --key $key) && result_found=$(echo $acl | grep -q $search_term) && if $result_found; then echo $key; echo $acl; fi; done |
Here’s what it does:
- Recursively list all objects in your bucket
- Iterate through those object keys
- Ask S3 what the ACL for that object is
- If that ACL contains the string “AllUsers” (the global s3 permission group), it will echo that ACL to stdout.
I generalized this problem to “echo all keys and their ACLs within a bucket iff that ACL contains a given $search_term“, so if others are stumbling across a similar, but subtly different problem, this solution will still be helpful, insofar as they change the $search_term to something that suits their problem.
Ideally (assuming you want no public objects) if you run this… nothing should show up.
Keep in mind, this solution won’t scale well for massive buckets with tons and tons of objects.