Question:
I’m storing midi files in an S3 bucket and am trying to download them into the SageMake jupyter notebook. I am using this code
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
import os import boto3 # Python library for Amazon API import botocore from botocore.exceptions import ClientError def download_from_s3(url): """ex: url = s3://sagemakerbucketname/data/validation.tfrecords""" url_parts = url.split("/") # => ['s3:', '', 'sagemakerbucketname', 'data', ... bucket_name = url_parts[2] key = os.path.join(*url_parts[3:]) filename = url_parts[-1] if not os.path.exists(filename): try: # Create an S3 client s3 = boto3.resource('s3') print('Downloading {} to {}'.format(url, filename)) s3.Bucket(bucket_name).download_file(key, filename) except botocore.exceptions.ClientError as e: if e.response['Error']['Code'] == "404": print('The object {} does not exist in bucket {}'.format( key, bucket_name)) else: raise |
however I am getting An error occurred (403) when calling the HeadObject operation: Forbidden
Here are the permissions attached for the S3:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::*" ] } ] } |
Answer:
The S3 bucket sagemakerbucketname you are using should be in the same region as the Sagemaker Notebook Instance.
The IAM role associated with the notebook instance should be given permission to access the S3 bucket.
Run below command in the sagemaker notebook to get the IAM role
role = get_execution_role()
Verify the role used to launch the notebook has permissions to access the S3 bucket. These are the permissions you are expected to have
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::sagemakerbucketname"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::sagemakerbucketname/*"
]
}