CloudWatch Logs

CloudWatch Logs

CloudWatch Logs

Hello Everyone

Welcome to CloudAffaire and this is Debjeet

In the last blog post, we have discussed CloudWatch Events and also create CloudWatch events on EC2 instance state change.

https://cloudaffaire.com/cloudwatch-events/

In this blog post, we are going to discuss CloudWatch Logs. We are also going to configure CloudWatch Log Agent in an EC2 instance hosting Amazon Linux 2 to collect and store system logs in CloudWatch console.

CloudWatch Logs:

You can use Amazon CloudWatch Logs to monitor, store, and access your log files from Amazon Elastic Compute Cloud (Amazon EC2) instances, AWS CloudTrail, Route 53, and other sources. You can then retrieve the associated log data from CloudWatch Logs.

CloudWatch Logs component:

Log Events:

A log event is a record of some activity recorded by the application or resource being monitored. The log event record that CloudWatch Logs understands contains two properties: the timestamp of when the event occurred, and the raw event message. Event messages must be UTF-8 encoded.

Log Streams:

A log stream is a sequence of log events that share the same source. More specifically, a log stream is generally intended to represent the sequence of events coming from the application instance or resource being monitored. For example, a log stream may be associated with an Apache access log on a specific host. When you no longer need a log stream, you can delete it using the aws logs delete-log-stream command. In addition, AWS may delete empty log streams that are over 2 months old.

Log Groups:

Log groups define groups of log streams that share the same retention, monitoring, and access control settings. Each log stream has to belong to one log group. For example, if you have a separate log stream for the Apache access logs from each host, you could group those log streams into a single log group called MyWebsite.com/Apache/access_log. There is no limit on the number of log streams that can belong to one log group.

Metric Filters:

You can use metric filters to extract metric observations from ingested events and transform them to data points in a CloudWatch metric. Metric filters are assigned to log groups, and all of the filters assigned to a log group are applied to their log streams.

Retention Settings:

Retention settings can be used to specify how long log events are kept in CloudWatch Logs. Expired log events get deleted automatically. Just like metric filters, retention settings are also assigned to log groups, and the retention assigned to a log group is applied to their log streams.

Prerequisite for CloudWatch Log Agent:

  • Running EC2 instance.
  • Instance configured with AWS CLI and IAM role for CloudWatch Log Agent

We have already created the EC2 instance with proper role and also configured AWS CLI

CloudWatch Logs

CloudWatch Logs

CloudWatch Logs

Next, we are going to install and configure CloudWatch Log Agent.

Step 1: Connect to your EC2 instance and install CloudWatch Log Agent.

sudo yum update -y
sudo yum install -y awslogs

CloudWatch Logs

Step 2: View CloudWatch Log Agent configuration files

sudo ls -l /etc/awslogs/

CloudWatch Logs

Step 3: Configure awscli.conf and awslogs.conf with region and log details

sudo vi /etc/awslogs/awscli.conf
sudo vi /etc/awslogs/awslogs.conf

CloudWatch Logs

CloudWatch Logs

You can referrer below AWS documentation for Log Agent configuration

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html

Step 4: Start the CloudWatch Log Agent

sudo systemctl start awslogsd

CloudWatch Logs

Once the Log Agent is started, it will publish logs as per configuration details in the configuration file. You can check the logs in CloudWatch console under Logs.

CloudWatch Logs

Hope you have enjoyed this article. With this, we are concluding our introductory series on CloudWatch and will start with a new AWS service from the next blog post.

To get more details on CloudWatch, please refer below AWS documentation

https://docs.aws.amazon.com/cloudwatch/index.html

 

 

This Post Has One Comment

  1. Hi sir,

    please start process IAM Documents .

Leave a Reply

Close Menu