Monitoring an AWS Infrastructure

July 16, 2018 / Nirav Shah

Clear strategy saves a lot of time and future pain.

Keeping above line in mind we advise to follow below set of rules for monitoring an AWS infrastructure.
<ol>
<li>Elastic Load Balaner</li>
<li>WebServer EC2 instance</li>
<li>PostgreSQL isntance</li>
<li>S3 Buckets</li>
</ol>
We can make the infrastructure smooth and fine by applying various cloud watch monitoring and alerts.
<h2></h2>
<h2>At VPC Level</h2>
<b>1.1 AWS Identity and Access Management (IAM)</b>

IAM enables you to securely control how users create, configure, change, and delete AWS network resources. For example, an IAM Network Administrator group or role can be granted permission to administer VPCs, subnets, route tables, gateways, and peering connections while a Developer group can be granted more restrictive, view-only access to these resources, or permission to launch Amazon Elastic Compute Cloud (Amazon EC2) instances into only specific VPCs or subnets. Granular permissions can be used to restrict other network-related tasks, such as API calls that modify a security group or VPC route table changes.

<b>1.2 AWS Cloud Trail and Cloud Watch</b>

AWS Cloud Trail provides a history of AWS API calls for an account, including API calls made via the AWS Management Console, AWS SDKs, command line tools, and higher-level AWS services (such as AWS Cloud Formation). This AWS API call history enables security analysis, resource change tracking, and compliance auditing. You can also deliver CloudTrail data to Cloud Watch Logs to store, monitor, and process API calls for network-specific changes and to send appropriate notifications.

<b>1.3 VPC Flow Logs</b>

VPC Flow Logs capture network flow information for a VPC, subnet, or network interface in Amazon Cloud Watch Logs. Flow logs can help you with a number of tasks, such as troubleshooting why specific traffic is not reaching an instance, which in turn can help you diagnose overly restrictive security group rules. You can also use flow logs as a security tool to monitor the traffic that is reaching your instance, to profile your network traffic, and to look for abnormal traffic behaviours.
<h2></h2>
<h2>At EC2 Level</h2>
After you launch an instance, you can open the Amazon EC2 console and view the monitoring graphs for an instance on the Monitoring tab. Each graph is based on one of the available Amazon EC2 metrics.
We should consider the following implementations:
<table id=”main-table”>
<thead class=”col-xs-hidden text-c text-u”>
<tr>
<th class=”col-sm-6 committees-title gb-color”>
<h2 class=”w-color text-u”>Metric Name</h2>
</th>
<th class=”col-sm-3 committees-title gb-color”>
<h2 class=”w-color text-u”>Unit</h2>
</th>
<th class=”col-sm-3 committees-title gb-color”>
<h2 class=”w-color text-u”>Suggestion</h2>
</th>
</tr>
</thead>
<tbody>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Average CPU Utilization</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Percent</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Memory usage</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Custom Matrix</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>DiscSpace Usage</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Custom Matrix</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Maximum Network In</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Bytes</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Maximum Network Out</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Bytes</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Summary Disk Read Operations</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Count</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Summary Disk Write Operations</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Count</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Summary Status Instance</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Count</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Summary Status System</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Count</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
</tbody>
</table>
<h2></h2>
<h2>At Load Balancer Level</h2>
You can view the Cloud Watch metrics for your load balancers using the Amazon EC2 console. These metrics are displayed as monitoring graphs. The monitoring graphs show data points if the load balancer is active and receiving requests.
<table id=”main-table”>
<thead class=”col-xs-hidden text-c text-u”>
<tr>
<th class=”col-sm-4 committees-title gb-color”>
<h2 class=”w-color text-u”>Metric Name</h2>
</th>
<th class=”col-sm-5 committees-title gb-color”>
<h2 class=”w-color text-u”>Unit</h2>
</th>
<th class=”col-sm-3 committees-title gb-color”>
<h2 class=”w-color text-u”>Suggestion</h2>
</th>
</tr>
</thead>
<tbody>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Healthy Hosts</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>UnHealthy Host Count</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Unhealthy Hosts</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>UnHealthy Host Count</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Average Latency</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Latency</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Sum Requests</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Request Count</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Backend Connection Errors</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Backend Connection Errors</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Surge Queue Length</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Surge Queue Length</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Spillover Count</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>Spillover Count</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Sum HTTP 2XXs</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>HTTPCode_Backend_2XX</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Sum HTTP 4XXs</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>HTTPCode_Backend_4XX</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Sum ELB HTTP 4XXs</td>
<td class=”membership-type-list text-c” data-th=”Unit :”>HTTPCode_ELB_4XX</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
</tbody>
</table>
<h2></h2>
<h2>At RDS Level</h2>
You can monitor DB instances using Amazon Cloud Watch, which collects and processes raw data from Amazon RDS into readable, near real-time metrics. These statistics are recorded for a period of two weeks, so that you can access historical information and gain a better perspective on how your web application or service is performing
<table id=”main-table”>
<thead class=”col-xs-hidden text-c text-u”>
<tr>
<th class=”col-sm-6 committees-title gb-color”>
<h2 class=”w-color text-u”>Metric Name</h2>
</th>
<th class=”col-sm-6 committees-title gb-color”>
<h2 class=”w-color text-u”>Suggestion</h2>
</th>
</tr>
</thead>
<tbody>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>CPU Utilization</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Burst Balance</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>CPU Credit Balance</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Database Connections</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Disk Queue Depth</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Freeable Memory</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Free Storage Space</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Network Receive Throughput</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Network Transmit Throughput</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>ReplicaLag – if multi az deployment</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>WriteIOPS</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
</tbody>
</table>
<h2></h2>
<h2>At Cloud Front Level</h2>
Amazon Cloud Front integrates with Amazon Cloud Watch metrics so that you can monitor your website or application. Cloud Front currently provides six free metrics.
<table id=”main-table”>
<thead class=”col-xs-hidden text-c text-u”>
<tr>
<th class=”col-sm-6 committees-title gb-color”>
<h2 class=”w-color text-u”>Metric Name</h2>
</th>
<th class=”col-sm-6 committees-title gb-color”>
<h2 class=”w-color text-u”>Suggestion</h2>
</th>
</tr>
</thead>
<tbody>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Requests</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Bytes Downloaded</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Bytes Uploaded</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>-</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>Total Error Rate</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>4xx Error Rate</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
<tr>
<td class=”membership-type-list” data-th=”Metric Name :”>5xx Error Rate</td>
<td class=”membership-type-list text-c” data-th=”Suggestion :”>With Alarm</td>
</tr>
</tbody>
</table>