April 27, 2020 / Nirav Shah
ABSTRACT : AWS CloudWatch is a cloudwatch agent service that can be used to view Custom Instance metrics like disk-space utilization, etc. and can be used to set alarms and notifications for the same to stay informed on the resources your instance is using as well as; the CloudWatch agent service can be extended to view custom logs in the dashboard itself without logging-in into your instance.
This blog is a continuation of the first part of the blog on AWS CloudWatch which focused on being able to see custom resource usage metrics and in this part we will focus on enabling viewing the custom log file.
We will go back to our guinea-pig in North Virginia and configure the CloudWatch agent running inside in it to add “/var/log/apache2/access.log” to be able to view the contents of this logfile from the dashboard itself.
The log file right now has the contents as shown in the screenshot above.
Now, we will reconfigure the config.json file being used for viewing custom resource metrics of the guinea-pig.
$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
In the 4th screenshot above we have replied yes to “Do you want to monitor any log files?” and then in the next prompt have provided the exact log file path of the custom log-file that we want to see under “Log file path:” with the path file “/var/log/apache2/access.log”.
Once this is done, now we can restart the CloudWatch agent service for it to start showing the apache access.log files.
$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s
Once we have restarted the service we have to wait at least 60 seconds for the CloudWatch dashboard to refresh with the custom log files.
Now, first lets check the log-file inside the instance.
In the file output the logs before “=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=” are older ones and the logs after it are newer ones.
Now let’s go back to the AWS CloudWatch console; over there click on “Log groups” and type your “Log Group Name Prefix” into the search bar which in our case over here is “access.log”.
Once you click on it you are shown a Log Stream with a name which in our case is the Instance-ID of our guinea-pig and can be customized by you in the config.json file used for CloudWatch agent service in your instance and when we click on that Instance-ID we can view the logs in the dashboard.
As you can see the screenshots the dashboard will load with real-time logs the same way that you can view them into your instance but the upper hand that you get in cloudwatch dashboard via the CloudWatch agent service is that you don’t need to access the Instance to access the logs. In the CloudWatch agent service configuration setup you can also add more than 1 log file to be viewed, so CloudWatch agent service does not allow to view any single log-file; you can view multiple log-files.
Again these steps are not inclusive and have been taken to show in the most minimal steps possible as to how to view custom log files in your CloudWatch dashboard, as previously specified you can customize the CloudWatch agent config.json file to view your custom logs according to your unique requirements.
Also Read – How to setup AWS CloudWatch Agent for Custom Metrics?
FAQs:
1. Where does cloudwatch store logs
2. What is difference between CloudWatch and CloudWatch logs?
3. How long do CloudWatch logs stay?
As a Director of Eternal Web Private Ltd an AWS consulting partner company, Nirav is responsible for its operations. AWS, cloud-computing and digital transformation are some of his favorite topics to talk about. His key focus is to help enterprises adopt technology, to solve their business problem with the right cloud solutions.
Have queries about your AWS project ideas and concepts? Please drop in your project details to discuss with our AWS experts, professionals and consultants.
