How To Install And Update ECS Container Agent Using AWS CLI

How To Install And Update ECS Container Agent Using AWS CLI

How To Install And Update ECS Container Agent Using AWS CLI

Hello Everyone

Welcome to CloudAffaire and this is Debjeet.

In the last AWS blog post, we have discussed how to create an ECS Container Instance with ECS-optimized AMI using AWS CLI..

In this blog post, we will discuss ECS Container Agent. We will also learn how to install and update ECS container agent.

What is an ECS Container Agent?

The AWS ECS container agent allows container instances to connect to your cluster. The AWS ECS container agent is included in the AWS ECS-optimized AMIs, but you can also install it on any AWS EC2 instance that supports the AWS ECS specification.

Note: The AWS ECS container agent is only supported on AWS EC2 instances.

ECS Container Agent Configuration:

The AWS ECS container agent supports a number of configuration options, most of which should be set through environment variables. The following environment variables are available, and all of them are optional.

If your container instance was launched with a Linux variant of the AWS ECS-optimized AMI, you can set these environment variables in the /etc/ecs/ecs.config file and then restart the agent. You can also write these configuration variables to your container instances with AWS EC2 user data at launch time.

If you are manually starting the AWS ECS container agent (for non AWS ECS-optimized AMIs), you can use these environment variables in the docker run command that you use to start the agent. Use these variables with the syntax –env=VARIABLE_NAME=VARIABLE_VALUE. For sensitive information, such as authentication credentials for private repositories, you should store your agent environment variables in a file and pass them all at one time with the –env-file path_to_env_file option.

Available Container Agent Parameters:

  • ECS_CLUSTER: The cluster this agent should check into.
  • ECS_RESERVED_PORTS: An array of ports that should be marked as unavailable for scheduling on this container instance.
  • ECS_RESERVED_PORTS_UDP: An array of UDP ports that should be marked as unavailable for scheduling on this container instance.
  • ECS_ENGINE_AUTH_TYPE: The type of auth data that is stored in the ECS_ENGINE_AUTH_DATA key.
  • ECS_ENGINE_AUTH_DATA: Docker auth data formatted as defined by ECS_ENGINE_AUTH_TYPE.
  • AWS_DEFAULT_REGION: The region to be used in API requests as well as to infer the correct backend host.
  • AWS_ACCESS_KEY_ID: The access key used by the agent for all calls.
  • AWS_SECRET_ACCESS_KEY: The secret key used by the agent for all calls.
  • AWS_SESSION_TOKEN: The session token used for temporary credentials.
  • DOCKER_HOST: Used to create a connection to the Docker daemon; behaves similarly to this environment variable as used by the Docker client.
  • ECS_LOGLEVEL: The level of detail that should be logged.
  • ECS_LOGFILE: The location where logs should be written. Log level is controlled by ECS_LOGLEVEL.
  • ECS_CHECKPOINT: Whether to checkpoint state to the DATADIR specified below.
  • ECS_DATADIR: The container path where state is checkpointed for use across agent restarts.
  • ECS_UPDATES_ENABLED: Whether to exit for an updater to apply updates when requested.
  • ECS_UPDATE_DOWNLOAD_DIR: Where to place update tarballs within the container.
  • ECS_DISABLE_METRICS: Whether to disable metrics gathering for tasks.
  • ECS_POLL_METRICS: Whether to poll or stream when gathering metrics for tasks.
  • ECS_POLLING_METRICS_WAIT_DURATION: Time to wait to poll for new metrics for a task. Only used when ECS_POLL_METRICS is true
  • ECS_RESERVED_MEMORY: Memory, in MiB, to reserve for use by things other than containers managed by AWS ECS.
  • ECS_AVAILABLE_LOGGING_DRIVERS: Which logging drivers are available on the container instance.
  • ECS_DISABLE_PRIVILEGED: Whether launching privileged containers is disabled on the container instance.
  • ECS_SELINUX_CAPABLE: Whether SELinux is available on the container instance.
  • ECS_APPARMOR_CAPABLE: Whether AppArmor is available on the container instance.
  • ECS_ENGINE_TASK_CLEANUP_WAIT_DURATION: Time to wait to delete containers for a stopped task. If set to less than 1 minute, the value is ignored.
  • ECS_CONTAINER_STOP_TIMEOUT: Instance scoped configuration for time to wait for the container to exit normally before being forcibly killed.
  • ECS_CONTAINER_START_TIMEOUT: Timeout before giving up on starting a container.
  • ECS_ENABLE_TASK_IAM_ROLE: Whether to enable IAM Roles for Tasks on the Container Instance
  • ECS_ENABLE_TASK_IAM_ROLE_NETWORK_HOST: Whether to enable IAM Roles for Tasks when launched with host network mode on the Container Instance
  • ECS_DISABLE_IMAGE_CLEANUP: Whether to disable automated image cleanup for the ECS Agent.
  • ECS_IMAGE_CLEANUP_INTERVAL: The time interval between automated image cleanup cycles. If set to less than 10 minutes, the value is ignored.
  • ECS_IMAGE_MINIMUM_CLEANUP_AGE: The minimum time interval between when an image is pulled and when it can be considered for automated image cleanup.
  • NON_ECS_IMAGE_MINIMUM_CLEANUP_AGE: The minimum time interval between when a non ECS image is created and when it can be considered for automated image cleanup.
  • ECS_NUM_IMAGES_DELETE_PER_CYCLE: The maximum number of images to delete in a single automated image cleanup cycle. If set to less than 1, the value is ignored.
  • ECS_IMAGE_PULL_BEHAVIOR: The behavior used to customize the pull image process.
  • ECS_IMAGE_PULL_INACTIVITY_TIMEOUT: The time to wait after docker pulls complete waiting for extraction of a container. Useful for tuning large Windows containers.
  • ECS_INSTANCE_ATTRIBUTES: These attributes take effect only during initial registration.
  • ECS_ENABLE_TASK_ENI: Whether to enable task networking for task to be launched with its own network interface
  • ECS_ENABLE_HIGH_DENSITY_ENI: Whether to enable high density eni feature when using task networking
  • ECS_CNI_PLUGINS_PATH: The path where the cni binary file is located
  • ECS_AWSVPC_BLOCK_IMDS: Whether to block access to Instance Metadata for Tasks started with awsvpc network mode
  • ECS_AWSVPC_ADDITIONAL_LOCAL_ROUTES: In awsvpc network mode, traffic to these prefixes will be routed via the host bridge instead of the task ENI
  • ECS_ENABLE_CONTAINER_METADATA: When true, the agent will create a file describing the container’s metadata.
  • ECS_HOST_DATA_DIR: The source directory on the host from which ECS_DATADIR is mounted.
  • ECS_ENABLE_TASK_CPU_MEM_LIMIT: Whether to enable task-level cpu and memory limits
  • ECS_CGROUP_PATH: The root cgroup path that is expected by the ECS agent. This is the path that accessible from the agent mount.
  • ECS_CGROUP_CPU_PERIOD: CGroups CPU period for task level limits. This value should be between 8ms to 100ms
  • ECS_ENABLE_CPU_UNBOUNDED_WINDOWS_WORKAROUND: When true, ECS will allow CPU unbounded(CPU=0) tasks to run along with CPU bounded tasks in Windows.
  • ECS_ENABLE_MEMORY_UNBOUNDED_WINDOWS_WORKAROUND: When true, ECS will ignore the memory reservation parameter (soft limit) to run along with memory bounded tasks in Windows.
  • ECS_TASK_METADATA_RPS_LIMIT: Comma separated integer values for steady state and burst throttle limits for task metadata endpoint
  • ECS_SHARED_VOLUME_MATCH_FULL_CONFIG: When true, ECS Agent will compare name, driver options, and labels to make sure volumes are identical.
  • ECS_CONTAINER_INSTANCE_PROPAGATE_TAGS_FROM: If ec2_instance is specified, existing tags defined on the container instance will be registered to AWS ECS and will be discoverable using the ListTagsForResource API.
  • ECS_CONTAINER_INSTANCE_TAGS: The metadata that you apply to the container instance to help you categorize and organize them.
  • ECS_ENABLE_UNTRACKED_IMAGE_CLEANUP: Whether to allow the ECS agent to delete containers and images that are not part of ECS tasks.
  • ECS_EXCLUDE_UNTRACKED_IMAGE: Comma seperated list of imageName:tag of images that should not be deleted by the ECS agent if ECS_ENABLE_UNTRACKED_IMAGE_CLEANUP is enabled.
  • ECS_DISABLE_DOCKER_HEALTH_CHECK: Whether to disable the Docker Container health check for the ECS Agent.
  • ECS_NVIDIA_RUNTIME: The Nvidia Runtime to be used to pass Nvidia GPU devices to containers.
  • ECS_ENABLE_SPOT_INSTANCE_DRAINING: Whether to enable Spot Instance draining for the container instance.
  • ECS_LOG_ROLLOVER_TYPE: Determines whether the container agent logfile will be rotated based on size or hourly.
  • ECS_LOG_OUTPUT_FORMAT: Determines the log output format. When the json format is used, each line in the log would be a structured JSON map.
  • ECS_LOG_MAX_FILE_SIZE_MB: When the ECS_LOG_ROLLOVER_TYPE variable is set to size, this variable determines the maximum size (in MB) the log file before it is rotated.
  • ECS_LOG_MAX_ROLL_COUNT: Determines the number of rotated log files to keep. Older log files are deleted once this limit is reached.

Note: There are undocumented variables that the agent uses internally that may be visible but that are not intended for customer use.

Storing Container Instance Configuration in AWS S3:

AWS ECS container agent configuration is controlled with the environment variables described in the previous section. Linux variants of the AWS ECS-optimized AMI look for these variables in /etc/ecs/ecs.config when the container agent starts and configure the agent accordingly. Certain innocuous environment variables, such as ECS_CLUSTER, can be passed to the container instance at launch through AWS EC2 user data and written to this file without consequence. However, other sensitive information, such as your AWS credentials or the ECS_ENGINE_AUTH_DATA variable, should never be passed to an instance in user data or written to /etc/ecs/ecs.config in a way that would allow them to show up in a .bash_history file.

Storing configuration information in a private bucket in AWS S3 and granting read-only access to your container instance IAM role is a secure and convenient way to allow container instance configuration at launch. You can store a copy of your ecs.config file in a private bucket. You can then use AWS EC2 user data to install the AWS CLI and copy your configuration information to /etc/ecs/ecs.config when the instance launches.

Private Registry Authentication for Container Instances:

The AWS ECS container agent can authenticate with private registries, including Docker Hub, using basic authentication. When you enable private registry authentication, you can use private Docker images in your task definitions. This feature is only supported by tasks using the EC2 launch type.

Another method of enabling private registry authentication uses AWS Secrets Manager to store your private registry credentials securely and then reference them in your container definition. This allows your tasks to use images from private repositories. This method supports tasks using either the EC2 or Fargate launch types.

The AWS ECS container agent below two environment variables for authentication:

  • ECS_ENGINE_AUTH_TYPE, which specifies the type of authentication data that is being sent.
  • ECS_ENGINE_AUTH_DATA, which contains the actual authentication credentials.

Automated Task and Image Cleanup:

Each time a task is placed on a container instance, the AWS ECS container agent checks to see if the images referenced in the task are the most recent of the specified tag in the repository. If not, the default behavior allows the agent to pull the images from their respective repositories. If you frequently update the images in your tasks and services, your container instance storage can quickly fill up with Docker images that you are no longer using and may never use again. For example, you may use a continuous integration and continuous deployment (CI/CD) pipeline.

Likewise, containers that belong to stopped tasks can also consume container instance storage with log information, data volumes, and other artifacts. These artifacts are useful for debugging containers that have stopped unexpectedly, but most of this storage can be safely freed up after a period of time.

Note: By default, the AWS ECS container agent automatically cleans up stopped tasks and Docker images that are not being used by any tasks on your container instances.

AWS ECS Container Metadata File:

Beginning with version 1.15.0 of the AWS ECS container agent, various container metadata is available within your containers or the host container instance. By enabling this feature, you can query the information about a task, container, and container instance from within the container or the host container instance. The metadata file is created on the host instance and mounted in the container as a Docker volume.

This feature is disabled by default. You can enable container metadata at the container instance level by setting the ECS_ENABLE_CONTAINER_METADATA container agent variable to true. You can set this variable in the /etc/ecs/ecs.config configuration file and restart the agent. You can also set it as a Docker environment variable at runtime when the agent container is started.

By default, the container metadata file is written to the following host and container paths.

  • For Linux instances:
    • Host path: /var/lib/ecs/data/metadata/cluster_name/task_id/container_name/ecs-container-metadata.json
    • Container path: /opt/ecs/metadata/random_ID/ecs-container-metadata.json
  • For Windows instances:
    • Host path: C:\ProgramData\AWS\ECS\data\metadata\task_id\container_name\ecs-container-metadata.json
    • Container path: C:\ProgramData\AWS\ECS\metadata\random_ID\ecs-container-metadata.json

How To Install And Update ECS Container Agent Using AWS CLI:

Step 1: Create a custom VPC for your ECS container instance.

Step 2: Create an IAM role for your ECS container instance.

Step 3: Create your ECS container instance.

Step 4: Install and configure ECS Container Agent.

Step 5: Get details of your ECS container instance.

Note: If you manually installing the ECS Container Agent, the container agent may not be of the latest version. Next, we are going to manually update our ECS Container Agent.

How To Install And Update ECS Container Agent Using AWS CLI

Step 6: Update your ECS Container Agent.

Our ECS Container Agent successfully updated to the latest version.

How To Install And Update ECS Container Agent Using AWS CLI

Step 7: Clenup.

Hope you have enjoyed this article, In the next blog post, we will discuss ECS Task.

All the public cloud providers are changing the console user interface rapidly and due to this some of the screenshots used in our previous AWS blogs are no longer relevant. Hence, we have decided that from now onwards most of the demo will be done programmatically. Let us know your feedback on this in the comment section.

To get more details on AWS ECS, please refer below AWS documentation


Leave a Reply

Close Menu