How to configure python boto3 SDK for AWS
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last blog post, we have discussed how to install Python Boto3 SDK for AWS.
https://cloudaffaire.com/how-to-install-python-boto3-sdk-for-aws/
In this blog post, we will discuss how to configure python boto3 SDK for AWS.
Boto3 configuration:
There are two types of configuration data in boto3: credentials and non-credentials. Credentials include items such as aws_access_key_id, aws_secret_access_key, and aws_session_token. Non-credential configuration includes items such as which region to use or which addressing style to use for Amazon S3. The distinction between credentials and non-credentials configuration is important because the lookup process is slightly different. Boto3 will look in several additional locations when searching for credentials that do not apply when searching for non-credential configuration.
The mechanism in which boto3 looks for credentials is to search through a list of possible locations and stop as soon as it finds credentials. The order in which Boto3 searches for credentials is:
- Passing credentials as parameters in the boto.client() or boto.Session().
- Passing credentials as Environment variables
- Passing credentials as Shared credential file (~/.aws/credentials)
- Passing credentials using AWS config file (~/.aws/config)
- Passing credentials as by assume Role provider
- Passing credentials using Boto2 config file (/etc/boto.cfg and ~/.boto)
- Passing credentials through Instance metadata service on an Amazon EC2 instance that has an IAM role configured.
Next, we are going to explain some of the configuration options using examples.
Prerequisites for this demo:
- One EC2 AWS Linux 2 instance.
- One IAM user with programmatic admin access.
Demo:
Passing credentials as parameters:
You can directly pass your AWS_ACCESS_KEY and AWS_SECRET_KEY as parameter in your python code using boto.client() or boto.Session().
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
##------------------------------------- ## Passing credentials as parameters ## ##------------------------------------- ## Login to your EC2 instance ## Passing credentials as parameters to Session() ## Declared variables for your access_key and secret_key export AWS_ACCESS_KEY= export AWS_SECRET_KEY= ## Create a file named list_vpc_id.py vi list_vpc_id.py -------------------------- #!/usr/bin/env python #print vpc id import os import json import boto3 session = boto3.Session(region_name='ap-south-1',aws_access_key_id=os.environ['AWS_ACCESS_KEY'],aws_secret_access_key=os.environ['AWS_SECRET_KEY']) ec2 = session.client('ec2') response = ec2.describe_vpcs() print(json.dumps(response['Vpcs'][0]['VpcId'], indent=4, sort_keys=True)) -------------------------- :wq ## Execute the file python list_vpc_id.py ## Passing credentials as parameters to client() ## Declared variables for your access_key and secret_key export AWS_ACCESS_KEY= export AWS_SECRET_KEY= ## Create a file named list_vpc_id.py vi list_vpc_id.py -------------------------- #!/usr/bin/env python #print vpc id import os import json import boto3 ec2 = boto3.client('ec2',region_name='ap-south-1',aws_access_key_id=os.environ['AWS_ACCESS_KEY'],aws_secret_access_key=os.environ['AWS_SECRET_KEY']) response = ec2.describe_vpcs() print(json.dumps(response['Vpcs'][0]['VpcId'], indent=4, sort_keys=True)) -------------------------- :wq ## Execute the file python list_vpc_id.py |
Passing credentials as environment variables:
You can declared your aws access and secret key’s in the form of AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables and boto3 will automatically detect this environment variable and will use them to connect to your AWS infrastructure.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
##------------------------------------------------ ## Passing credentials as environment variables ## ##------------------------------------------------ ## Login to your EC2 instance ## Declared environment variables for your access_key and secret_key export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= ## Create a file named list_vpc_id.py vi list_vpc_id.py -------------------------- #!/usr/bin/env python #print vpc id import json import boto3 ec2 = boto3.client('ec2',region_name='ap-south-1') response = ec2.describe_vpcs() print(json.dumps(response['Vpcs'][0]['VpcId'], indent=4, sort_keys=True)) -------------------------- :wq ## Execute the file python list_vpc_id.py |
Passing credentials using AWS CLI config\credentials files:
If you have AWS CLI installed in the system, you can pass credentials using ~/.aws/config and ~/.aws/credentials files. Boto3 will automatically detect the credential files and use the credential to interact with your AWS infrastructure.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
##---------------------------------------------------------- ## Passing credentials using aws cli config\credentials file ##---------------------------------------------------------- ## Login to your EC2 instance ## Install AWS CLI, if not installed already ## Configure AWS CLI aws configure --profile admin #AWS Access Key ID [None]: #AWS Secret Access Key [None]: #Default region name [None]: #Default output format [None]: ## Create a file named list_vpc_id.py vi list_vpc_id.py -------------------------- #!/usr/bin/env python #print vpc id import json import boto3 session = boto3.Session(profile_name='admin') ec2 = session.client('ec2') response = ec2.describe_vpcs() print(json.dumps(response['Vpcs'][0]['VpcId'], indent=4, sort_keys=True)) -------------------------- :wq ## Execute the file python list_vpc_id.py |
Hope you have enjoyed this article. In the next blog post, we will discuss boto3 resources and will create our 1st AWS resource using Boto3 SDK.
To get more details on Python Boto3, please refer below AWS documentation
https://boto3.amazonaws.com/v1/documentation/api/latest/index.html