Boto3 resources

Hello Everyone

Welcome to CloudAffaire and this is Debjeet.

In the last blog post, we have discussed how to configure Python Boto3 SDK for AWS.

https://cloudaffaire.com/how-to-configure-python-boto3-sdk-for-aws/

In this blog post, we will discuss boto3 resources.

What are resources?

Resources represent an object-oriented interface to Amazon Web Services (AWS). They provide a higher-level abstraction than the raw, low-level calls made by service clients. Every resource instance has a number of attributes and methods. These can conceptually be split up into identifiers, attributes, actions, references, sub-resources, and collections.

Resources themselves can also be conceptually split into service resources (like sqs, s3, ec2, etc) and individual resources (like sqs.Queue or s3.Bucket). Service resources do not have identifiers or attributes. The two share the same components otherwise.

Resource components:

Identifier: An identifier is a unique value that is used to call actions on the resource. Resources must have at least one identifier, except for the top-level service resources (e.g. ec2 or s3). An identifier is set at instance creation-time, and failing to provide all necessary identifiers during instantiation will result in an exception.
Attributes: Resources may also have attributes, which are lazy-loaded properties on the instance. They may be set at creation time from the response of an action on another resource, or they may be set when accessed or via an explicit call to the load or reload action.
Actions: An action is a method which makes a call to the service. Actions may return a low-level response, a new resource instance or a list of new resource instances. Actions automatically set the resource identifiers as parameters, but allow you to pass additional parameters via keyword arguments.
References: A reference is an attribute which may be None or a related resource instance. The resource instance does not share identifiers with its reference resource, that is, it is not a strict parent to child relationship. In relational terms, these can be considered many-to-one or one-to-one.
Sub-resources: A sub-resource is similar to a reference, but is a related class rather than an instance. Sub-resources, when instantiated, share identifiers with their parent. It is a strict parent-child relationship. In relational terms, these can be considered one-to-many.
Waiters: A waiter is similar to an action. A waiter will poll the status of a resource and suspend execution until the resource reaches the state that is being polled for or a failure occurs while polling. Waiters automatically set the resource identifiers as parameters, but allow you to pass additional parameters via keyword arguments.
Collections: A collection provides an iterable interface to a group of resources. Collections behave similarly to Django QuerySets and expose a similar API. A collection seamlessly handles pagination for you, making it possible to easily iterate over all items from all pages of data.

Let us explain the different component of a resource with a demo.

Prerequisites for this demo:

One EC2 AWS Linux 2 instance with boto3 installed and configured.

Demo:

## Login to your EC2 instance
## Create a file named create_vpc.py
vi create_vpc.py
---------------------------
#creates a vpc and two subnets
#!/usr/bin/env python
#import python boto3
import boto3

#variables
aws_region = "ap-south-1"
aws_az1 = "ap-south-1a"
aws_az2 = "ap-south-1b"
vpc_cidr = "10.0.0.0/16"
subnet1_cidr = "10.0.1.0/24"
subnet2_cidr = "10.0.2.0/24"

#top-level service resource ec2 called
ec2 = boto3.resource('ec2', aws_region) 

#individual resource vpc created using ec2 resource action create_vpc() and attribute CidrBlock
#individual resource vpc will be created with identifier vpc id
#individual resource vpc has reference dhcp_options which is taken autometically if not provided.
vpc = ec2.create_vpc(CidrBlock=vpc_cidr)

#waiter is called to poll the status of the resource vpc
vpc.wait_until_available()

#individual resource subnet created using ec2 resource action create_subnet() and attributes AvailabilityZone, CidrBlock
#individual resource subnet will be created with identifier subnet id
subnet1 = vpc.create_subnet(AvailabilityZone=aws_az1,CidrBlock=subnet1_cidr)
subnet2 = vpc.create_subnet(AvailabilityZone=aws_az2,CidrBlock=subnet2_cidr)

#create_tags() is an action supported by most of the resources. 
vpc.create_tags(Tags=[{"Key": "Name", "Value": "myvpc"}])
subnet1.create_tags(Tags=[{"Key": "Name", "Value": "myvpc_subnet1"}])
subnet2.create_tags(Tags=[{"Key": "Name", "Value": "myvpc_subnet2"}])

#vpc.id is a sub-resource of resource subnet, subnet will always have an vpc assosiated. 
print('vpc id:')
print(subnet1.vpc.id)

#subnets is a collection of resource ec2 and all() creates an iterable of all Subnet resources in the collection.
print('all subnet ids of all vpc in the region:')
subnet_iterator = ec2.subnets.all()
for sbnt in subnet_iterator:
    print(sbnt.id)

#You can limit a collection using limit()
print('1st 3 subnet ids:')
subnet_iterator1 = ec2.subnets.limit(3)
for sbnt1 in subnet_iterator1:
    print(sbnt1.id)

#collection also supports filters
print('subnet id where tag{Name:myvpc_subnet2}:')
subnet_iterator2 = ec2.subnets.filter(Filters=[{'Name': 'tag:Name','Values': ['myvpc_subnet2']}])
for sbnt2 in subnet_iterator2:
    print(sbnt2.id)

#cleanup
subnet1.delete()
subnet2.delete()
vpc.delete()
---------------------------
:wq

## Execute the script
python create_vpc.py

## Login to your EC2 instance

## Create a file named create_vpc.py

vi create_vpc.py

---------------------------

#creates a vpc and two subnets

#!/usr/bin/env python

#import python boto3

import boto3

#variables

aws_region = "ap-south-1"

aws_az1 = "ap-south-1a"

aws_az2 = "ap-south-1b"

vpc_cidr = "10.0.0.0/16"

subnet1_cidr = "10.0.1.0/24"

subnet2_cidr = "10.0.2.0/24"

#top-level service resource ec2 called

ec2 = boto3.resource('ec2', aws_region)

#individual resource vpc created using ec2 resource action create_vpc() and attribute CidrBlock

#individual resource vpc will be created with identifier vpc id

#individual resource vpc has reference dhcp_options which is taken autometically if not provided.

vpc = ec2.create_vpc(CidrBlock=vpc_cidr)

#waiter is called to poll the status of the resource vpc

vpc.wait_until_available()

#individual resource subnet created using ec2 resource action create_subnet() and attributes AvailabilityZone, CidrBlock

#individual resource subnet will be created with identifier subnet id

subnet1 = vpc.create_subnet(AvailabilityZone=aws_az1,CidrBlock=subnet1_cidr)

subnet2 = vpc.create_subnet(AvailabilityZone=aws_az2,CidrBlock=subnet2_cidr)

#create_tags() is an action supported by most of the resources.

vpc.create_tags(Tags=[{"Key": "Name", "Value": "myvpc"}])

subnet1.create_tags(Tags=[{"Key": "Name", "Value": "myvpc_subnet1"}])

subnet2.create_tags(Tags=[{"Key": "Name", "Value": "myvpc_subnet2"}])

#vpc.id is a sub-resource of resource subnet, subnet will always have an vpc assosiated.

print('vpc id:')

print(subnet1.vpc.id)

#subnets is a collection of resource ec2 and all() creates an iterable of all Subnet resources in the collection.

print('all subnet ids of all vpc in the region:')

subnet_iterator = ec2.subnets.all()

for sbnt in subnet_iterator:

print(sbnt.id)

#You can limit a collection using limit()

print('1st 3 subnet ids:')

subnet_iterator1 = ec2.subnets.limit(3)

for sbnt1 in subnet_iterator1:

print(sbnt1.id)

#collection also supports filters

print('subnet id where tag{Name:myvpc_subnet2}:')

subnet_iterator2 = ec2.subnets.filter(Filters=[{'Name': 'tag:Name','Values': ['myvpc_subnet2']}])

for sbnt2 in subnet_iterator2:

print(sbnt2.id)

#cleanup

subnet1.delete()

subnet2.delete()

vpc.delete()

---------------------------

:wq

## Execute the script

python create_vpc.py

Resource components with respect to the code:

Service Resource: ec2
Individual Resources: vpc, subnet
Identifier: vpc (id) and subnet (id).
Attributes: vpc (CidrBlock=’10.0.0.0/16′) and subnet (AvailabilityZone=’ap-south-1a’,CidrBlock=’10.0.1.0/24′)
Actions: ec2 (create_vpc()), vpc (create_subnet(), create_tags()), subnet (create_tags())
References: vpc (dhcp_options), subnet (vpc)
Sub-resources: subnet (print(subnet.vpc.id))
Waiters: vpc (wait_until_available())

Hope you have enjoyed this article. In the next blog post, we will discuss boto3 clients.

To get more details on Python Boto3, please refer below AWS documentation

https://boto3.amazonaws.com/v1/documentation/api/latest/index.html

Boto3 Resources