How To Create An ECS Cluster Using AWS CLI
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last AWS blog post, we have discussed Elastic Container Service (ECS) in AWS.
https://cloudaffaire.com/elastic-container-service-ecs/
In this blog post, we will discuss the Elastic Container Service Cluster. We will also create an ECS Cluster using AWS CLI.
What is an ECS Cluster?
AWS ECS cluster is a logical grouping of tasks or services. If you are running tasks or services that use the EC2 launch type, a cluster is also a grouping of container instances. If you are using capacity providers, a cluster is also a logical grouping of capacity providers. When you first use Amazon ECS, a default cluster is created for you, but you can create multiple clusters in an account to keep your resources separate.
ECS Cluster Capacity Providers:
ECS cluster capacity providers determine the infrastructure to use for your tasks. Each cluster has one or more capacity providers and an optional default capacity provider strategy. The capacity provider strategy determines how the tasks are spread across the capacity providers. When you run a task or create a service, you may either use the cluster’s default capacity provider strategy or specify a capacity provider strategy that overrides the cluster’s default strategy.
Cluster Capacity Provider Concepts:
Capacity provider:
A capacity provider is used in association with a cluster to determine the infrastructure that a task runs on. One or more capacity providers are specified in a capacity provider strategy, which is then associated with a cluster.
For Amazon ECS on AWS Fargate users, the FARGATE and FARGATE_SPOT capacity providers are provided automatically.
For Amazon ECS on Amazon EC2 users, a capacity provider consists of a name, an Auto Scaling group, and the settings for managed scaling and managed termination protection.
Capacity provider strategy:
A capacity provider strategy gives you control over how your tasks use one or more capacity providers. When you run a task or create a service, you specify a capacity provider strategy. A capacity provider strategy consists of one or more capacity providers with an optional base and weight specified for each provider.
The base value designates how many tasks, at a minimum, to run on the specified capacity provider. Only one capacity provider in a capacity provider strategy can have a base defined.
The weight value designates the relative percentage of the total number of launched tasks that should use the specified capacity provider. For example, if you have a strategy that contains two capacity providers, and both have a weight of 1, then when the base is satisfied, the tasks will be split evenly across the two capacity providers.
Default capacity provider strategy:
A default capacity provider strategy is associated with each Amazon ECS cluster. This determines the capacity provider strategy the cluster will use if no other capacity provider strategy or launch type is specified when running a task or creating a service.
ECS Cluster Auto Scaling:
ECS cluster auto-scaling enables you to have more control over how you scale tasks within a cluster. Amazon ECS capacity providers use Auto Scaling groups to manage the Amazon EC2 instances registered to their clusters. When creating a capacity provider, you can optionally enable managed scaling. When managed scaling is enabled, Amazon ECS manages the scale-in and scale-out actions of the Auto Scaling group. On your behalf, Amazon ECS creates an AWS Auto Scaling scaling plan with a target tracking scaling policy based on the target capacity value you specify. Amazon ECS then associates this scaling plan with your Auto Scaling group. If managed termination protection is enabled when you create a capacity provider, the Auto Scaling group and each Amazon EC2 instance in the Auto Scaling group must have instance protection from scale in enabled as well.
Note: When using managed termination protection, managed scaling must also be used otherwise managed termination protection will not work.
AWS Fargate Capacity Providers:
Amazon ECS cluster capacity providers enable you to use both Fargate and Fargate Spot capacity with your Amazon ECS tasks. With Fargate Spot you can run interruption tolerant Amazon ECS tasks at a discounted rate compared to the Fargate price. Fargate Spot runs tasks on spare compute capacity. When AWS needs the capacity back, your tasks will be interrupted with a two-minute warning.
Note: The Fargate and Fargate Spot capacity providers do not need to be created. They are available to all accounts and only need to be associated with a cluster to be available for use.
ECS Custer State:
- ACTIVE: The cluster is ready to accept tasks and, if applicable, you can register container instances with the cluster.
- PROVISIONING: The cluster has capacity providers associated with it and the resources needed for the capacity provider are being created.
- DEPROVISIONING: The cluster has capacity providers associated with it and the resources needed for the capacity provider are being deleted.
- FAILED: The cluster has capacity providers associated with it and the resources needed for the capacity provider have failed to create.
- INACTIVE: The cluster has been deleted. Clusters with an INACTIVE status may remain discoverable in your account for a period of time.
ECS Cluster Features:
- Clusters are Region-specific
- A cluster may contain a mix of tasks using either the Fargate or EC2 launch types.
- A cluster may contain a mix of both Auto Scaling group capacity providers and Fargate capacity providers, however when specifying a capacity provider strategy they may only contain one or the other but not both.
- For tasks using the EC2 launch type, clusters can contain multiple different container instance types, but each container instance may only be registered to one cluster at a time.
- Custom IAM policies may be created to allow or restrict user access to specific clusters.
- When you specify a capacity provider strategy, the number of capacity providers that can be specified is limited to six.
- A cluster may contain a mix of both Auto Scaling group capacity providers and Fargate capacity providers, however when specifying a capacity provider strategy they may only contain one or the other but not both.
- A cluster may contain a mix of tasks and services using both capacity providers and launch types. A service may also be updated to use a capacity provider strategy rather than a launch type, however you must force a new deployment when doing so.
- When you specify a capacity provider strategy, the base value is only supported when running tasks. When creating a service, the capacity provider strategy base parameter is not supported.
- When using managed termination protection, managed scaling must also be used otherwise managed termination protection will not work.
- It is recommended that you create a new Auto Scaling group to use with a capacity provider rather than using an existing one.
- An Auto Scaling group must have a MaxSize greater than zero to scale out.
- Managed scaling is only supported in Regions that AWS Auto Scaling is available in.
- When using managed termination protection, managed scaling must also be used otherwise managed termination protection will not work.
- If managed termination protection is enabled when you create a capacity provider, the Auto Scaling group and each Amazon EC2 instance in the Auto Scaling group must have instance protection from scale in enabled as well.
- If managed scaling is enabled when you create a capacity provider, the Auto Scaling group desired count can be set to 0.
- The Fargate and Fargate Spot capacity providers do not need to be created. They are available to all accounts and only need to be associated with a cluster to be available for use.
- When a new cluster is created using the Amazon ECS console along with the Networking only cluster template, the FARGATE and FARGATE_SPOT capacity providers are associated with the new cluster automatically.
- To add the FARGATE and FARGATE_SPOT capacity providers to an existing cluster, you must use the AWS CLI or API.
- Using Fargate Spot requires that your task use platform version 1.3.0 or later.
- When tasks using the Fargate and Fargate Spot capacity providers are stopped, a task state change event is sent to Amazon EventBridge. The stopped reason describes the cause.
Next, we are going to create an ECS Cluster using AWS CLI.
How To Create An ECS Cluster Using AWS CLI:
Step 1: Create an ECS Cluster with Fargate launch type.
|
1 2 3 4 5 6 7 8 9 10 |
######################################### ## Create An ECS Cluster Using AWS CLI ## ######################################### ## Prerequisite: AWS CLI installed and configured with proper access ## https://cloudaffaire.com/category/aws/aws-cli/ ## Create a cluster with fargate capacity provider aws ecs create-cluster \ --cluster-name myecscluster \ --capacity-providers FARGATE FARGATE_SPOT |
Step 2: Create a task definition to run tasks and services in your ECS Cluster.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
## Create a task definition to run a task on your ecs cluster vi myecsclustertaskdef.json ----------------------- { "family": "mytaskdef", "networkMode": "awsvpc", "containerDefinitions": [ { "name": "myapp", "image": "httpd:2.4", "portMappings": [ { "containerPort": 80, "hostPort": 80, "protocol": "tcp" } ], "essential": true, "entryPoint": [ "sh", "-c" ], "command": [ "/bin/sh -c \"echo 'hello from ecs fargate cluster' > /usr/local/apache2/htdocs/index.html && httpd-foreground\"" ] } ], "requiresCompatibilities": [ "FARGATE" ], "cpu": "256", "memory": "512" } ----------------------- :wq ## Register the task definition aws ecs register-task-definition \ --cli-input-json file://myecsclustertaskdef.json |
Step 3: Create a service using the task definition created in step 2.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
## Get your default vpc subnet and sg info AWS_VPC_ID=$(aws ec2 describe-vpcs \ --filters "Name=isDefault, Values=true" \ --query 'Vpcs[0].VpcId' \ --output text) && AWS_DEFAULT_SECURITY_GROUP_ID=$(aws ec2 describe-security-groups \ --filters "Name=vpc-id,Values=$AWS_VPC_ID" \ --query 'SecurityGroups[?GroupName == `default`].GroupId' \ --output text) && AWS_SUBNET_ONE_ID=$(aws ec2 describe-subnets \ --filters "Name=vpc-id,Values=$AWS_VPC_ID" \ --query 'Subnets[?AvailabilityZone == `ap-south-1a`].SubnetId' \ --output text) ## Create security group ingress rules aws ec2 authorize-security-group-ingress \ --group-id $AWS_DEFAULT_SECURITY_GROUP_ID \ --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "Allow SSH"}]}]' && aws ec2 authorize-security-group-ingress \ --group-id $AWS_DEFAULT_SECURITY_GROUP_ID \ --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 80, "ToPort": 80, "IpRanges": [{"CidrIp": "0.0.0.0/0", "Description": "Allow HTTP"}]}]' ## Create a service in the ecs cluster using task definition aws ecs create-service \ --cluster myecscluster \ --service-name myservice \ --task-definition mytaskdef:1 \ --desired-count 1 \ --launch-type "FARGATE" \ --network-configuration "awsvpcConfiguration={subnets=[$AWS_SUBNET_ONE_ID],securityGroups=[$AWS_DEFAULT_SECURITY_GROUP_ID],assignPublicIp=ENABLED}" |
Step 4: Get details of your ECS Cluster using AWS CLI.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
## List all available ecs cluster aws ecs list-clusters ## Get the details of ecs cluster aws ecs describe-clusters \ --cluster myecscluster ## Get the details of ecs cluster capacity provider aws ecs describe-capacity-providers ## List all available task definitions aws ecs list-task-definitions ## Get the details of ecs cluster task definition aws ecs describe-task-definition \ --task-definition mytaskdef:1 ## List all the available cluster services aws ecs list-services \ --cluster myecscluster ## Get the details of ecs cluster service aws ecs describe-services \ --cluster myecscluster \ --services myservice ## List all the task in your cluster aws ecs list-tasks \ --cluster myecscluster ## Get details of task in your cluster AWS_ECS_TASK_ARN=$(aws ecs list-tasks \ --cluster myecscluster \ --query 'taskArns' \ --output text) && aws ecs describe-tasks \ --cluster myecscluster \ --tasks $AWS_ECS_TASK_ARN |
Step 5: Check your application running in the ECS Cluster.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
## Get public ip of your ecs deployed application AWS_ECS_FARGATE_ENI=$(aws ecs describe-tasks \ --cluster myecscluster \ --tasks $AWS_ECS_TASK_ARN \ --query 'tasks[0].attachments[0].details[?name == `networkInterfaceId`].value' \ --output text) && AWS_ECS_APP_PUBLIC_IP=$(aws ec2 describe-network-interfaces \ --network-interface-ids $AWS_ECS_FARGATE_ENI \ --query 'NetworkInterfaces[0].Association.PublicIp' \ --output text) && echo $AWS_ECS_APP_PUBLIC_IP ## Open the public ip address (above output) in your browser ## Or curl the public ip address curl $AWS_ECS_APP_PUBLIC_IP |
Step 6: Cleanup.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
## Update service desired count to 0 aws ecs update-service \ --cluster myecscluster \ --service myservice \ --desired-count 0 ## Delete the service aws ecs delete-service \ --cluster myecscluster \ --service myservice ## Delete the cluster aws ecs delete-cluster \ --cluster myecscluster ## Deregister the task definition aws ecs deregister-task-definition \ --task-definition mytaskdef:1 |
Hope you have enjoyed this article, In the next blog post, we will discuss ECS Task Definitions.
All the public cloud providers are changing the console user interface rapidly and due to this some of the screenshots used in our previous AWS blogs are no longer relevant. Hence, we have decided that from now onwards most of the demo will be done programmatically. Let us know your feedback on this in the comment section.
To get more details on AWS ECS, please refer below AWS documentation
https://docs.aws.amazon.com/ecs/index.html