How To Create An ECS Container Instance With ECS-optimized AMI Using AWS CLI
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last AWS blog post, we have discussed ECS Task Definition and its parameters.
https://cloudaffaire.com/how-to-create-ecs-task-definition-using-aws-cli/
In this blog post, we will discuss Container Instances in ECS. We will also create an ECS Container Instance with ECS-optimized AMI using AWS CLI.
What is an ECS Container Instance?
An Amazon ECS container instance is an Amazon EC2 instance that is running the Amazon ECS container agent and has been registered into a cluster. When you run tasks with Amazon ECS using the EC2 launch type, your tasks are placed on your active container instances.
ECS Container Instance Requirement:
- ECS Container instance must be running the Amazon ECS container agent to register into one of your clusters. If you are using an Amazon ECS-optimized AMI, the agent is already installed. To use a different operating system, install the agent.
- Because the Amazon ECS container agent makes calls to Amazon ECS on your behalf, you must launch container instances with an IAM role that authenticates to your account and provides the required resource permissions.
- If any of the containers associated with your tasks require external connectivity, you can map their network ports to ports on the host Amazon ECS container instance so they are reachable from the internet.
- AWS strongly recommend launching your container instances inside a VPC, because Amazon VPC delivers more control over your network and offers more extensive configuration capabilities.
- Container instances need access to communicate with the Amazon ECS service endpoint. This can be through an interface VPC endpoint or through your container instances having public IP addresses.
- The type of Amazon EC2 instance that you choose for your container instances determines the resources available in your cluster.
- You should not deregister an instance from one cluster and re-register it into another. To relocate container instance resources, we recommend that you terminate container instances from one cluster and launch new container instances with the latest Amazon ECS-optimized Amazon Linux 2 AMI in the new cluster.
- You cannot stop a container instance and change its instance type. Instead, we recommend that you terminate the container instance and launch a new container instance with the desired instance size and the latest Amazon ECS-optimized Amazon Linux 2 AMI in your desired cluster.
ECS Container Instance Lifecycle:
When the Amazon ECS container agent registers an instance into your cluster, the container instance reports its status as ACTIVE and its agent connection status as TRUE. This container instance can accept run task requests.
If you stop (not terminate) an Amazon ECS container instance, the status remains ACTIVE, but the agent connection status transitions to FALSE within a few minutes. Any tasks that were running on the container instance stop. If you start the container instance again, the container agent reconnects with the Amazon ECS service, and you are able to run tasks on the instance again.
Important: If you stop and start a container instance, or reboot that instance, some older versions of the Amazon ECS container agent register the instance again without deregistering the original container instance ID. In this case, Amazon ECS lists more container instances in your cluster than you actually have.
ECS-optimized AMIs:
An ECS Container instance require a modern Linux distribution running at least version 3.10 of the Linux kernel, The Amazon ECS container agent (preferably the latest version) and A Docker daemon running at least version 1.9.0, and any Docker runtime dependencies. The Amazon ECS-optimized AMIs are preconfigured with these requirements and recommendations.
The AMI ID, image name, operating system, container agent version, and runtime version for the different Amazon ECS-optimized AMIs can be programmatically retrieved by querying the Systems Manager Parameter Store API.
ECS Container Instance Draining:
There are times when you might need to remove a container instance from a cluster; for example, to perform system updates, update the Docker daemon, or scale down the cluster size. Container instance draining enables you to remove a container instance from a cluster without impacting tasks in your cluster.
When you set a container instance to DRAINING, Amazon ECS prevents new tasks from being scheduled for placement on the container instance. Service tasks on the draining container instance that are in the PENDING state are stopped immediately. If there are container instances in the cluster that are available, replacement service tasks are started on them.
ECS Container Agent:
The Amazon ECS container agent allows container instances to connect to your cluster. The Amazon ECS container agent is included in the Amazon ECS-optimized AMIs, but you can also install it on any Amazon EC2 instance that supports the Amazon ECS specification. The Amazon ECS container agent is only supported on Amazon EC2 instances.
Note: Tasks using the Fargate launch type are deployed onto infrastructure managed by AWS, so this topic does not apply.
How To Create An ECS Container Instance With ECS-optimized AMI Using AWS CLI:
Step 1: Create a custom VPC for your ECS Container Instance.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
#################################################### ## Create An ECS Container Instance Using AWS CLI ## #################################################### ## Prerequisite: AWS CLI installed and configured with proper access ## https://cloudaffaire.com/category/aws/aws-cli/ ##------------------------------------------ ## Create custom vpc for your ecs cluster ## ##------------------------------------------ ## Create a directory for this demo mkdir ecs_container_instance_demo && cd ecs_container_instance_demo ## Create a VPC with DNS hostname enabled AWS_VPC_ID=$(aws ec2 create-vpc \ --cidr-block 10.0.0.0/16 \ --query 'Vpc.{VpcId:VpcId}' \ --output text) && aws ec2 modify-vpc-attribute \ --vpc-id $AWS_VPC_ID \ --enable-dns-hostnames "{\"Value\":true}" ## Create a public subnet AWS_SUBNET_PUBLIC_ID=$(aws ec2 create-subnet \ --vpc-id $AWS_VPC_ID --cidr-block 10.0.1.0/24 \ --availability-zone ap-south-1a --query 'Subnet.{SubnetId:SubnetId}' \ --output text) && aws ec2 modify-subnet-attribute \ --subnet-id $AWS_SUBNET_PUBLIC_ID \ --map-public-ip-on-launch ## Create an Internet Gateway and attach to the VPC AWS_INTERNET_GATEWAY_ID=$(aws ec2 create-internet-gateway \ --query 'InternetGateway.{InternetGatewayId:InternetGatewayId}' \ --output text) && aws ec2 attach-internet-gateway \ --vpc-id $AWS_VPC_ID \ --internet-gateway-id $AWS_INTERNET_GATEWAY_ID ## Create a custom route table with route to internet gateway and explicite assosiation to public subnet AWS_CUSTOM_ROUTE_TABLE_ID=$(aws ec2 create-route-table \ --vpc-id $AWS_VPC_ID \ --query 'RouteTable.{RouteTableId:RouteTableId}' \ --output text ) && aws ec2 create-route \ --route-table-id $AWS_CUSTOM_ROUTE_TABLE_ID \ --destination-cidr-block 0.0.0.0/0 \ --gateway-id $AWS_INTERNET_GATEWAY_ID && AWS_ROUTE_TABLE_ASSOID_ONE=$(aws ec2 associate-route-table \ --subnet-id $AWS_SUBNET_PUBLIC_ID \ --route-table-id $AWS_CUSTOM_ROUTE_TABLE_ID \ --query 'AssociationId' \ --output text) ## Create a custom security group with ingress rules AWS_CUSTOM_SECURITY_GROUP_ID=$(aws ec2 create-security-group \ --vpc-id $AWS_VPC_ID \ --group-name myvpc-security-group \ --description 'My VPC non default security group' \ --query 'GroupId' \ --output text) && aws ec2 authorize-security-group-ingress \ --group-id $AWS_CUSTOM_SECURITY_GROUP_ID \ --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "Allow SSH"}]}]' && aws ec2 authorize-security-group-ingress \ --group-id $AWS_CUSTOM_SECURITY_GROUP_ID \ --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 80, "ToPort": 80, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "Allow HTTP"}]}]' && aws ec2 authorize-security-group-ingress \ --group-id $AWS_CUSTOM_SECURITY_GROUP_ID \ --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 443, "ToPort": 443, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "Allow HTTPS"}]}]' |
Step 2: Create a custom IAM role for your ECS Container Instance.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
##--------------------------------------------------------- ## Create a custom IAM Role for your ECS Container Instance ##--------------------------------------------------------- ## Create a policy for ecsInstanceRole IAM Role vi ecsInstanceRolePolicy.json ----------------- { "Version": "2008-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } ------------------ :wq ## Create an IAM EC2 service role and attach AmazonEC2ContainerServiceforEC2Role policy AWS_IAM_ROLE_ARN=$(aws iam create-role --role-name ecsInstanceRole \ --assume-role-policy-document file://ecsInstanceRolePolicy.json \ --query 'Role.Arn' \ --output text) && aws iam attach-role-policy \ --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role \ --role-name ecsInstanceRole ## Create an IAM Instance Profile for Your ECS EC2 Instances using ecsInstanceRole role aws iam create-instance-profile \ --instance-profile-name ecsInstanceProfileRole && aws iam add-role-to-instance-profile \ --instance-profile-name ecsInstanceProfileRole \ --role-name ecsInstanceRole && AWS_IAM_INSTANCE_PROFILE_ARN=$(aws iam list-instance-profiles-for-role \ --role-name ecsInstanceRole \ --query 'InstanceProfiles[0].Arn' \ --output text) |
Step 3: Create your ECS Cluster and Task Definition for ECS Container Instance.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
##--------------------------- ## Create your ECS cluster ## ##--------------------------- ## Create your ECS cluster aws ecs create-cluster \ --cluster-name myecscluster ## Create a task definition file vi mytaskdefinition.json ----------------------- { "family": "mytaskdefinition", "containerDefinitions": [{ "name": "myapp", "image": "httpd:2.4", "cpu": 68, "portMappings": [{ "containerPort": 80, "hostPort": 80, "protocol": "tcp" }], "memory": 50, "essential": true, "entryPoint": [ "sh", "-c" ], "command": [ "/bin/sh -c \"echo 'hello from ecs container instance' > /usr/local/apache2/htdocs/index.html && httpd-foreground\"" ] }] } ----------------------- :wq ## Register the task definition aws ecs register-task-definition \ --cli-input-json file://mytaskdefinition.json |
Step 4: Create an ECS Container Instance using ECS-optimized AMIs.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
##--------------------------------- ## Create ecs container instance ## ##--------------------------------- ## Get Amazon Linux 2 ECS-optimized latest AMI ID AWS_AMI_ID=$(aws ssm get-parameters \ --names "/aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id" \ --query 'Parameters[0].Value' \ --output text) ## Create a key-pair aws ec2 create-key-pair \ --key-name myvpc-keypair \ --query 'KeyMaterial' \ --output text > myvpc-keypair.pem && chmod 400 myvpc-keypair.pem ## Create user data to register your ECS container instance with ECS Cluster vi userdata.txt ----------------------- #!/bin/bash echo ECS_CLUSTER=myecscluster >> /etc/ecs/ecs.config ----------------------- :wq ## Create one EC2 instance in the public subnet AWS_EC2_INSTANCE_ID=$(aws ec2 run-instances \ --image-id $AWS_AMI_ID \ --instance-type t2.micro \ --key-name myvpc-keypair \ --monitoring "Enabled=false" \ --security-group-ids $AWS_CUSTOM_SECURITY_GROUP_ID \ --subnet-id $AWS_SUBNET_PUBLIC_ID \ --iam-instance-profile Arn=$AWS_IAM_INSTANCE_PROFILE_ARN \ --user-data file://userdata.txt \ --private-ip-address 10.0.1.10 \ --query 'Instances[0].InstanceId' \ --output text) ## Check if the instance one is running ## It will take some time for the instance to get ready aws ec2 describe-instance-status \ --instance-ids $AWS_EC2_INSTANCE_ID --output text ## Create a task ## Check the task definition family revision number if you get any error aws ecs run-task \ --cluster myecscluster \ --task-definition mytaskdefinition:1 \ --count 1 |
Step 5: Get details of your ECS Container Instance and check your docker application.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
##----------------------------------------------- ## Get details for your ecs container instance ## ##----------------------------------------------- ## List all available ecs cluster aws ecs list-clusters ## Get the details of ecs cluster aws ecs describe-clusters \ --cluster myecscluster ## List all available task definitions aws ecs list-task-definitions ## Get the details of ecs cluster task definition aws ecs describe-task-definition \ --task-definition mytaskdefinition ## List your ECS container instance aws ecs list-container-instances \ --cluster myecscluster ## Get the details of ecs container instance ## If you have multiple ecs container instance, provide ARN of ## the ecs container instance for which you want to get details AWS_ECS_CONTAINER_INSTANCE_ARN=$(aws ecs list-container-instances \ --cluster myecscluster \ --query 'containerInstanceArns' \ --output text) && aws ecs describe-container-instances \ --cluster myecscluster \ --container-instances $AWS_ECS_CONTAINER_INSTANCE_ARN ## List all the task in your ECS cluster aws ecs list-tasks \ --cluster myecscluster ## Get the details of ecs task ## If you have multiple task, provide ARN for the task you want to get details AWS_ECS_TASK_ARN=$(aws ecs list-tasks \ --cluster myecscluster \ --query 'taskArns' \ --output text) && aws ecs describe-tasks \ --cluster myecscluster \ --tasks $AWS_ECS_TASK_ARN ## Get the public ip address of your ECS container instance AWS_ECS_CONTAINER_INSTANCE_PUBLIC_IP=$(aws ec2 describe-instances \ --instance-ids $AWS_EC2_INSTANCE_ID \ --query 'Reservations[*].Instances[?PrivateIpAddress == `10.0.1.10`].PublicIpAddress' \ --output text) && echo $AWS_ECS_CONTAINER_INSTANCE_PUBLIC_IP:80 ## Open the public ip address with port (above output) in your browser ## Or curl public ip address with port curl $AWS_ECS_CONTAINER_INSTANCE_PUBLIC_IP:80 ## hello from ecs container instance |
Step 6: Cleanup.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
##---------- ## Clenup ## ##---------- ## Stop the task aws ecs stop-task \ --cluster myecscluster \ --task $AWS_ECS_TASK_ARN ## Deregister container instance aws ecs deregister-container-instance \ --cluster myecscluster \ --container-instance $AWS_ECS_CONTAINER_INSTANCE_ARN \ --force ## Deregister the task definition aws ecs deregister-task-definition \ --task-definition mytaskdefinition:1 ## Delete your ECS cluster aws ecs delete-cluster \ --cluster myecscluster ## Delete the ecs instance aws ec2 terminate-instances \ --instance-ids $AWS_EC2_INSTANCE_ID ## Delete key pair aws ec2 delete-key-pair \ --key-name myvpc-keypair ## Delete custom security group (once instances are terminated) aws ec2 delete-security-group \ --group-id $AWS_CUSTOM_SECURITY_GROUP_ID ## Delete internet gateway aws ec2 detach-internet-gateway \ --internet-gateway-id $AWS_INTERNET_GATEWAY_ID \ --vpc-id $AWS_VPC_ID && aws ec2 delete-internet-gateway \ --internet-gateway-id $AWS_INTERNET_GATEWAY_ID ## Disassociate the subnets from custom route table aws ec2 disassociate-route-table \ --association-id $AWS_ROUTE_TABLE_ASSOID_ONE ## Delete custom route table aws ec2 delete-route-table \ --route-table-id $AWS_CUSTOM_ROUTE_TABLE_ID ## Delete the public subnets aws ec2 delete-subnet \ --subnet-id $AWS_SUBNET_PUBLIC_ID ## Delete the vpc aws ec2 delete-vpc \ --vpc-id $AWS_VPC_ID ## Delete the directory used in this demo cd .. && rm -rf ecs_container_instance_demo |
Hope you have enjoyed this article, In the next blog post, we will discuss ECS Container Agent.
All the public cloud providers are changing the console user interface rapidly and due to this some of the screenshots used in our previous AWS blogs are no longer relevant. Hence, we have decided that from now onwards most of the demo will be done programmatically. Let us know your feedback on this in the comment section.
To get more details on AWS ECS, please refer below AWS documentation
https://docs.aws.amazon.com/ecs/index.html
Thank you for very useful and interesting post. I’ve created script based on your example with very minor modifications. When I run that script I am getting the following error when calling the RunInstances. Any idea why I am getting this error?
The “aws iam list-instance-profiles” shows that instance profile exists and valid, see below.
An error occurred (InvalidParameterValue) when calling the RunInstances operation: Value (arn:aws:iam::619592225065:instance-profile/test9876_RoleInstanceProfile) for parameter iamInstanceProfile.arn is invalid. Invalid IAM Instance Profile ARN
$ aws iam list-instance-profiles
{
“InstanceProfiles”: [
{
“Path”: “/”,
“InstanceProfileName”: “test9876_RoleInstanceProfile”,
“InstanceProfileId”: “AIPAZAQUTCEUZ7YOZV47P”,
“Arn”: “arn:aws:iam::619592225065:instance-profile/test9876_RoleInstanceProfile”,
“CreateDate”: “2021-01-22T04:33:08Z”,
“Roles”: [
{
“Path”: “/”,
“RoleName”: “test9876_Role”,
“RoleId”: “AROAZAQUTCEU626TTALXL”,
“Arn”: “arn:aws:iam::619592225065:role/test9876_Role”,
“CreateDate”: “2021-01-22T04:33:05Z”,
“AssumeRolePolicyDocument”: {
“Version”: “2008-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Principal”: {
“Service”: [
“ec2.amazonaws.com”,
“ecs-tasks.amazonaws.com”
]
},
“Action”: “sts:AssumeRole”
}
]
}
}
]
}
]
}