How To Get All Resources Deployed In AWS
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last blog post, we discussed how to create conformance pack in AWS config.
https://cloudaffaire.com/how-to-create-conformance-pack-in-aws-config/
Sometimes you may want to get the list of all resources deployed in your AWS landscape. You may be using a free tire account and getting billed for some resources that you cannot identify where its deployed. Or you may have a request from your manager to give a complete inventory of your AWS landscape. Or you have a security breach in a particular VPC and want to get all the resources deployed in your VPC. You can use AWS config service to create an inventory of all your AWS resources for the supported AWS services. An inventory serves as a CMDB for your AWS landscape recording all the configuration changes.
How To Get All Resources Deployed In AWS:
Prerequisites:
- AWS CLI installed and configured with proper access. You can use below link to install and configure AWS CLI.
https://cloudaffaire.com/how-to-install-aws-cli/
https://cloudaffaire.com/how-to-configure-aws-cli/
Step 1: Create an S3 bucket to store your config recordings.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
## Create the S3 bucket aws s3api create-bucket \ --bucket s3-for-config-recording \ --region ap-south-1 \ --create-bucket-configuration LocationConstraint=ap-south-1 ## Create a bucket policy definition file for config cat <<EOF > config_bucket_policy.json { "Version": "2012-10-17", "Statement": [ { "Sid": "AWSConfigBucketPermissionsCheck", "Effect": "Allow", "Principal": { "Service": [ "config.amazonaws.com" ] }, "Action": "s3:GetBucketAcl", "Resource": "arn:aws:s3:::s3-for-config-recording" }, { "Sid": "AWSConfigBucketExistenceCheck", "Effect": "Allow", "Principal": { "Service": [ "config.amazonaws.com" ] }, "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::s3-for-config-recording" }, { "Sid": "AWSConfigBucketDelivery", "Effect": "Allow", "Principal": { "Service": [ "config.amazonaws.com" ] }, "Action": "s3:PutObject", "Resource": "arn:aws:s3:::s3-for-config-recording/*", "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } } } ] } EOF ## Create a S3 bucket policy for Config aws s3api put-bucket-policy \ --bucket s3-for-config-recording \ --policy file://config_bucket_policy.json |
Step 2: Create an IAM role for AWS Config service.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
## Create assume role policy definition cat <<'EOF'> config_assume_role_policy.json { "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Service": "config.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } EOF ## Create IAM role aws iam create-role \ --role-name config_iam_role \ --assume-role-policy-document file://config_assume_role_policy.json ## Create IAM policy definition for config to access S3 cat <<EOF > config_iam_policy.json { "Version":"2012-10-17", "Statement":[ { "Effect":"Allow", "Action":[ "s3:PutObject", "s3:PutObjectAcl" ], "Resource":[ "arn:aws:s3:::s3-for-config-recording/*" ], "Condition":{ "StringLike":{ "s3:x-amz-acl":"bucket-owner-full-control" } } }, { "Effect":"Allow", "Action":[ "s3:GetBucketAcl" ], "Resource":"arn:aws:s3:::s3-for-config-recording" } ] } EOF ## Update the IAM role with the above IAM policy aws iam put-role-policy \ --role-name config_iam_role \ --policy-name config_iam_policy \ --policy-document file://config_iam_policy.json ## Add AWS managed AWS_ConfigRole policy to the role aws iam attach-role-policy \ --role-name config_iam_role \ --policy-arn arn:aws:iam::aws:policy/service-role/AWS_ConfigRole |
Step 3: Enable AWS Config service.
|
1 2 3 4 5 6 |
## Enable AWS Config using AWS CLI ACCOUNT_ID=$(aws sts get-caller-identity | jq -r .Account) && IAM_ROLE_ARN=arn:aws:iam::$ACCOUNT_ID:role/config_iam_role && aws configservice put-configuration-recorder \ --configuration-recorder name=myconfig,roleARN=$IAM_ROLE_ARN \ --recording-group allSupported=true,includeGlobalResourceTypes=true |
Warning: AWS Config has some additional cost associated with it, please refer the latest pricing doc for additional details.
Step 4: Create a configuration delivery channel for AWS Config.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
## Create config delivery channel object configuration file cat << EOF > config_delivery_channel.json { "name": "myconfig", "s3BucketName": "s3-for-config-recording", "configSnapshotDeliveryProperties": { "deliveryFrequency": "One_Hour" } } EOF ## Create config delivery channel object as S3 aws configservice put-delivery-channel \ --delivery-channel file://config_delivery_channel.json |
Step 5: Start AWS config recordings.
|
1 2 3 |
## Starts recording configurations of recording group aws configservice start-configuration-recorder \ --configuration-recorder-name myconfig |
Note: It will take some time for the AWS Config service to discover all the resources deployed in your AWS account and create an inventory out of that. Give it some time, take a coffee break 😊
Once you are back, login to your AWS console and navigate to AWS Config dashboard.
If you want to get all the resources details, click on the “Resources” in the left panel.
Step 6: Get the list of all AWS services running in your AWS account.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
## Get config recorder status aws configservice describe-configuration-recorder-status ## Get all the resources discovered aws configservice get-discovered-resource-counts ## List all the resources discovered aws configservice list-discovered-resources \ --resource-type "AWS::EC2::VPC" ## Get all the resources deployed in your AWS landscape aws configservice select-resource-config \ --expression "SELECT *" aws configservice select-resource-config \ --expression "SELECT resourceType,resourceName,awsRegion,resourceId" |
If you want to get the list directly from the AWS console, click on the “Advanced queries” in the left panel and then click on “New query”.
Write “SELECT *” in the query editor to get details of all the resources deployed in the account and click on “Run”
Finally check on the “Export as” to export the report once generated.
Next, we will delete all the resources created in this demo to avoid any additional cost.
Step 7: Clean up.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
## Stop config recorder aws configservice stop-configuration-recorder \ --configuration-recorder-name myconfig ## Delete config delivery channel aws configservice delete-delivery-channel \ --delivery-channel-name myconfig ## Disable config service aws configservice delete-configuration-recorder \ --configuration-recorder-name myconfig ## Delete the S3 bucket with objects (configuration items) aws s3 rb \ s3://s3-for-config-recording --force ## Delete IAM Role & Policy aws iam detach-role-policy \ --role-name config_iam_role \ --policy-arn arn:aws:iam::aws:policy/service-role/AWS_ConfigRole && aws iam delete-role-policy \ --role-name config_iam_role \ --policy-name config_iam_policy && aws iam delete-role \ --role-name config_iam_role |
Note: AWS config is a regional service and will give you the resource inventory for current region only. If you have deployed resources in multiple regions, then repete the above steps for each region, or use an config aggregator to gather data from all regions (covered in next blog post)
Hope you have enjoyed this article. To know more about AWS Config, please refer below official documentation