How To Group Resources In AWS Using Tags
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last blog post, we have discussed the Parameter Store in AWS SSM.
https://cloudaffaire.com/how-to-create-parameter-store-in-aws-ssm/
In this blog post, we will discuss how to group resources in AWS using tags and AWS Resource Group service.
What Is AWS Resource Group:
You can use resource groups to organize your AWS resources. Resource groups make it easier to manage, monitor, and automate tasks on large numbers of resources at one time in the same AWS region. AWS Resource Groups provides two general methods for defining a resource group. Both methods involve using a query to identify the members of a group. The first method relies on tags applied to AWS resources to add resources to a group. The second method is based on resources available in an individual AWS CloudFormation stack.
What Can You Do With AWS Resource Group:
- You can integrate AWS Resource Group with AWS System Manager (SSM) to run commands, create automation workflows, create maintenance windows, create state manager, perform patching etc. on the resources under resource group.
- You can integrate AWS Resource Group with AWS CloudWatch to create a central monitoring dashboard for all your resources under the resource group.
- You can integrate AWS Resource Group with AWS Resource Access Manager (RAM) to share all the resources under your resource group at once with different accounts.
How To Group Resources In AWS Using Tags:
Requirements:
AWS CLI installed and configured. You can follow the below blog post to install and configure AWS CLI.
https://cloudaffaire.com/how-to-install-aws-cli/
https://cloudaffaire.com/how-to-configure-aws-cli/
Step 1: Create some AWS resources to add to the resources group.
Note: I am adding an EC2 instance with lots of custom configuration (Custom networking, LAMP stack, instance profile for SSM etc.) which is an overkill for this demo but will be required in upcoming demos if you want you can create few simple resources like an S3 bucket.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 |
############################################## ## How To Group Resources In AWS Using Tags ## ############################################## ## I am using a Linux host system to execute below commands ## --------------------- ## Create some resources ## --------------------- ## Create a directory for this demo mkdir myapp && cd myapp ## Create a VPC AWS_VPC_ID=$(aws ec2 create-vpc \ --cidr-block 10.0.0.0/16 \ --query 'Vpc.{VpcId:VpcId}' \ --output text) ## Enable DNS hostname for your VPC aws ec2 modify-vpc-attribute \ --vpc-id $AWS_VPC_ID \ --enable-dns-hostnames "{\"Value\":true}" ## Create a public subnet AWS_SUBNET_PUBLIC_ID=$(aws ec2 create-subnet \ --vpc-id $AWS_VPC_ID --cidr-block 10.0.1.0/24 \ --availability-zone ap-south-1a --query 'Subnet.{SubnetId:SubnetId}' \ --output text) ## Enable Auto-assign Public IP on Public Subnet aws ec2 modify-subnet-attribute \ --subnet-id $AWS_SUBNET_PUBLIC_ID \ --map-public-ip-on-launch ## Create an Internet Gateway AWS_INTERNET_GATEWAY_ID=$(aws ec2 create-internet-gateway \ --query 'InternetGateway.{InternetGatewayId:InternetGatewayId}' \ --output text) ## Attach Internet gateway to your VPC aws ec2 attach-internet-gateway \ --vpc-id $AWS_VPC_ID \ --internet-gateway-id $AWS_INTERNET_GATEWAY_ID ## Create a route table AWS_CUSTOM_ROUTE_TABLE_ID=$(aws ec2 create-route-table \ --vpc-id $AWS_VPC_ID \ --query 'RouteTable.{RouteTableId:RouteTableId}' \ --output text ) ## Create route to Internet Gateway aws ec2 create-route \ --route-table-id $AWS_CUSTOM_ROUTE_TABLE_ID \ --destination-cidr-block 0.0.0.0/0 \ --gateway-id $AWS_INTERNET_GATEWAY_ID ## Associate the public subnet with route table AWS_ROUTE_TABLE_ASSOID=$(aws ec2 associate-route-table \ --subnet-id $AWS_SUBNET_PUBLIC_ID \ --route-table-id $AWS_CUSTOM_ROUTE_TABLE_ID \ --output text) ## Create a security group aws ec2 create-security-group \ --vpc-id $AWS_VPC_ID \ --group-name myvpc-security-group \ --description 'My VPC non default security group' ## Get security group ID's AWS_DEFAULT_SECURITY_GROUP_ID=$(aws ec2 describe-security-groups \ --filters "Name=vpc-id,Values=$AWS_VPC_ID" \ --query 'SecurityGroups[?GroupName == `default`].GroupId' \ --output text) && AWS_CUSTOM_SECURITY_GROUP_ID=$(aws ec2 describe-security-groups \ --filters "Name=vpc-id,Values=$AWS_VPC_ID" \ --query 'SecurityGroups[?GroupName == `myvpc-security-group`].GroupId' \ --output text) ## Create security group ingress rules aws ec2 authorize-security-group-ingress \ --group-id $AWS_CUSTOM_SECURITY_GROUP_ID \ --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "Allow SSH"}]}]' && aws ec2 authorize-security-group-ingress \ --group-id $AWS_CUSTOM_SECURITY_GROUP_ID \ --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 80, "ToPort": 80, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "Allow HTTP"}]}]' ## Get Amazon Linux 2 latest AMI ID AWS_AMI_ID=$(aws ec2 describe-images \ --owners 'amazon' \ --filters 'Name=name,Values=amzn2-ami-hvm-2.0.????????-x86_64-gp2' 'Name=state,Values=available' \ --query 'sort_by(Images, &CreationDate)[-1].[ImageId]' \ --output 'text') ## Create a key-pair aws ec2 create-key-pair \ --key-name myvpc-keypair \ --query 'KeyMaterial' \ --output text > myvpc-keypair.pem ## Create user data for a LAMP stack cat <<EOF > myuserdata.txt #!/bin/bash sudo yum update -y sudo amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2 sudo yum install -y httpd mariadb-server sudo systemctl start httpd sudo systemctl is-enabled httpd sudo systemctl enable amazon-ssm-agent sudo systemctl start amazon-ssm-agent sudo systemctl status amazon-ssm-agent EOF ## Create an EC2 instance AWS_EC2_INSTANCE_ID=$(aws ec2 run-instances \ --image-id $AWS_AMI_ID \ --instance-type t2.micro \ --key-name myvpc-keypair \ --monitoring "Enabled=false" \ --security-group-ids $AWS_CUSTOM_SECURITY_GROUP_ID \ --subnet-id $AWS_SUBNET_PUBLIC_ID \ --user-data file://myuserdata.txt \ --private-ip-address 10.0.1.10 \ --query 'Instances[0].InstanceId' \ --output text) ## Create an Instance profile for SSM aws iam create-instance-profile \ --instance-profile-name "AmazonSSMInstanceProfileForInstances" ## Create a trust relation json file cat <<EOF > trust_policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } EOF ## Create an IAM role for SSM aws iam create-role \ --role-name "AmazonSSMRoleForInstances" \ --assume-role-policy-document file://trust_policy.json ## Attach the required policy for SSM aws iam attach-role-policy \ --role-name "AmazonSSMRoleForInstances" \ --policy-arn "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore" ## Add the role to the instance profile aws iam add-role-to-instance-profile \ --instance-profile-name "AmazonSSMInstanceProfileForInstances" \ --role-name "AmazonSSMRoleForInstances" ## Attach the Instance Profile to the EC2 instance aws ec2 associate-iam-instance-profile \ --instance-id "$AWS_EC2_INSTANCE_ID" \ --iam-instance-profile "Name=AmazonSSMInstanceProfileForInstances" |
Step 2: Add tags to your AWS resources. We will create a tag-based resource group on tag “Key = Name, Value = MYAPP”.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
## ------------------------------ ## Create tags for your resources ## ------------------------------ ## Add a tag to the VPC aws ec2 create-tags \ --resources $AWS_VPC_ID \ --tags "Key=Name,Value=MYAPP" "Key=env,Value=dev" ## Add a tag to public subnet aws ec2 create-tags \ --resources $AWS_SUBNET_PUBLIC_ID \ --tags "Key=Name,Value=MYAPP" "Key=env,Value=dev" ## Add a tag to the Internet-Gateway aws ec2 create-tags \ --resources $AWS_INTERNET_GATEWAY_ID \ --tags "Key=Name,Value=MYAPP" "Key=env,Value=dev" ## Add a tag to the default route table AWS_DEFAULT_ROUTE_TABLE_ID=$(aws ec2 describe-route-tables \ --filters "Name=vpc-id,Values=$AWS_VPC_ID" \ --query 'RouteTables[?Associations[0].Main != `flase`].RouteTableId' \ --output text) && aws ec2 create-tags \ --resources $AWS_DEFAULT_ROUTE_TABLE_ID \ --tags "Key=Name,Value=MYAPP" "Key=env,Value=dev" ## Add a tag to the public route table aws ec2 create-tags \ --resources $AWS_CUSTOM_ROUTE_TABLE_ID \ --tags "Key=Name,Value=MYAPP" "Key=env,Value=dev" ## Add a tags to security groups aws ec2 create-tags \ --resources $AWS_CUSTOM_SECURITY_GROUP_ID \ --tags "Key=Name,Value=MYAPP" "Key=env,Value=dev" && aws ec2 create-tags \ --resources $AWS_DEFAULT_SECURITY_GROUP_ID \ --tags "Key=Name,Value=MYAPP" "Key=env,Value=dev" ## Add a tag to the ec2 instance aws ec2 create-tags \ --resources $AWS_EC2_INSTANCE_ID \ --tags "Key=Name,Value=MYAPP" "Key=env,Value=dev" ## Add a tag to the ebs volume AWS_EBS_VOLUME_ID=$(aws ec2 describe-volumes \ --filters Name=attachment.instance-id,Values=$AWS_EC2_INSTANCE_ID \ --query 'Volumes[0].{VolumeId:VolumeId}' \ --output text) && aws ec2 create-tags \ --resources $AWS_EBS_VOLUME_ID \ --tags "Key=Name,Value=MYAPP" "Key=env,Value=dev" |
Step 3: Create and manage AWS Resource Group using AWS CLI.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
## ------------------ ## AWS Resource Group ## ------------------ ## Create a resource group resource query file cat <<EOF > rquery.json { "Type": "TAG_FILTERS_1_0", "Query": "{\"ResourceTypeFilters\":[\"AWS::AllSupported\"],\"TagFilters\":[{\"Key\":\"Name\", \"Values\":[\"MYAPP\"]}]}" } EOF ## Create a new resource group aws resource-groups create-group \ --name "myappResourceGroup" \ --resource-query file://rquery.json ## List all resource group name in a region aws resource-groups list-groups ## Get resource group details aws resource-groups get-group \ --group "myappResourceGroup" ## Get resource group query details aws resource-groups get-group-query \ --group "myappResourceGroup" ## Get all the resources in a resource group aws resource-groups list-group-resources \ --group "myappResourceGroup" ## Create a new resource (s3 bucket) aws s3api create-bucket \ --bucket "myapprmcloudaffaire" \ --create-bucket-configuration "LocationConstraint=ap-south-1" ## Add a tag to the s3 bucket aws s3api put-bucket-tagging \ --bucket "myapprmcloudaffaire" \ --tagging 'TagSet=[{Key=Name,Value=MYAPP}]' ## Add the s3 bucket to the resource group aws resource-groups group-resources \ --group "myappResourceGroup" \ --resource-arns "arn:aws:s3:::myapprmcloudaffaire" |
You can also view the resource group in the AWS Console under AWS System Manager.
Step 4: Cleanup.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
## ------- ## Cleanup ## ------- ## Delete the resource group aws resource-groups delete-group \ --group "myappResourceGroup" ## Terminate the ec2 instance aws ec2 terminate-instances \ --instance-ids $AWS_EC2_INSTANCE_ID && rm -f myuserdata.txt ## wait for some time sleep 30 ## Delete key pair aws ec2 delete-key-pair \ --key-name myvpc-keypair && rm -f myvpc-keypair.pem ## Delete custom security group aws ec2 delete-security-group \ --group-id $AWS_CUSTOM_SECURITY_GROUP_ID ## Delete internet gateway aws ec2 detach-internet-gateway \ --internet-gateway-id $AWS_INTERNET_GATEWAY_ID \ --vpc-id $AWS_VPC_ID && aws ec2 delete-internet-gateway \ --internet-gateway-id $AWS_INTERNET_GATEWAY_ID ## Delete the custom route table aws ec2 disassociate-route-table \ --association-id $AWS_ROUTE_TABLE_ASSOID && aws ec2 delete-route-table \ --route-table-id $AWS_CUSTOM_ROUTE_TABLE_ID ## Delete the public subnet aws ec2 delete-subnet \ --subnet-id $AWS_SUBNET_PUBLIC_ID ## Delete the vpc aws ec2 delete-vpc \ --vpc-id $AWS_VPC_ID ## Remove the IAM role from Instance Profile aws iam remove-role-from-instance-profile \ --instance-profile-name "AmazonSSMInstanceProfileForInstances" \ --role-name "AmazonSSMRoleForInstances" ## Dettach policy from the role aws iam detach-role-policy \ --role-name "AmazonSSMRoleForInstances" \ --policy-arn "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore" ## Delete the IAM role aws iam delete-role \ --role-name "AmazonSSMRoleForInstances" ## Delete the IAM Instance Profile aws iam delete-instance-profile \ --instance-profile-name "AmazonSSMInstanceProfileForInstances" ## Delete the s3 bucket aws s3api delete-bucket \ --bucket "myapprmcloudaffaire" ## Delete the directory for this demo cd .. && rm -rf myapp |
Hope you have enjoyed this blog post, to get more details on AWS SSM, please refer below AWS documentation
https://docs.aws.amazon.com/systems-manager/index.html