AWS has a lot of services and each service has its own pricing model which can become very confusing for the consumer to keep up with. There are a lot of instances on the internet where people have been shocked by their AWS bill.
In this article, we will look at the different ways you can be more proactive about your AWS cost monitoring. We have broken down the steps into the different methods into three categories:
AWS Budgets lets you set custom budgets that alert you when exceed your budgeted thresholds. AWS Budgets can help you track costs across different AWS services, accounts and tags. You can set up alerts to an email and/or SNS topic.
AWS Budgets should be the first step for anybody wanting to keep a closer eye on their AWS costs. However, one of the downsides of AWS Budgets is the lack of flexibility while setting up alerts. Budgets can only be set up at monthly, quarterly or annual frequencies. If there is a spike in your bill but you don’t cross the threshold, it won’t alert you.
How to set up AWS Budgets
Navigate to Budgets on the AWS console under the Billing & Cost Management Dashboard.
Click on Create Budgets. Then, select your budget type. For this example, we are setting up a Cost budget. You can also choose to set up a Usage or Reservation budget.
Set your budget. Choose what kind of budget you want to set up for your account. In this example, we are setting up a monthly budget of $1.
Next, you can configure alerts based on your budget. You can choose to alert based on the actual cost or the forecasted cost. You can notify via email or an SNS topic.
Lastly, you should be able to see your AWS Budget on the console.
Tagging AWS Resources
Once you have set up your budgets and alerts, it can be useful to tag all of your AWS resources. Amazon allows customers to assign metadata to AWS resources in the form of tags. Each tag consists of a user-defined key and optional value. These tags can be used to manage, and search for resources. These tags can also help with cost tracking. You can use tags to filter the AWS Budget you set up previously.
Amazon’s tagging strategies is a good read for understanding the best practices for tagging resources.
AWS Cost Explorer
AWS Cost Explorer is a tool that lets you view, understand and analyze your costs and usage. Cost Explorer lets you create custom reports based on your needs. You can use Cost Explorer to dive deeper into your cost and usage.
Cost Explorer can also give you suggestions about your Reserved Instances.
Cost Explorer is a useful tool as you start monitoring your costs more closely. However, there are some use-cases for which Cost Explorer can be hard to use. Some of these could be:
- Need to generate reports or views that aren’t supported by Cost Explorer.
- Drill deeper into certain costs. Cost Explorer provides higher level metrics but if you need to investigate further, it can sometimes be hard to do that.
AWS Cost and Usage Report
The AWS Cost and Usage report contains the most comprehensive set of AWS cost and usage data available. It lists AWS usage for each service category in hourly or daily line items, as well as any tags that are used for cost tracking.
AWS delivers the Cost & Usage Report (CUR) to an S3 bucket and updates the report throughout the day. This report is in a CSV format and can be downloaded, queried or analyzed to understand your costs better.
How to use the Cost & Usage Report
The Cost & Usage report can be integrated with Amazon Athena. Once integrated with Athena, you can run SQL queries against the CUR data to understanding, visualize and monitor your costs.
These are the steps to set up the Cost & Usage report and integrating it with Athena:
Create a Cost & Usage Report
Configure S3 bucket to store the report
Set up delivery options for the report
We enabled report data integration for Athena since we plan to use Athena to run queries against this data. You can also integrate the data with Redshift and Quicksight.
Set up AWS Glue crawler
Next, we will set up an AWS Glue Crawler so that Athena has access to the report data. To set up the Glue Crawler, we need to specify the S3 bucket where the report data is stored. We can also choose the crawling frequency for the data.
Querying the data on Athena
It can take a few hours for the data to be available after the initial set up. Once the data is available, you can run a simple query to see if the data is available.
Connecting to a visualization tool
Athena can be integrated with multiple visualization tools such as Looker, Superset or Tableau. You can query the data from any of these tools and set up custom dashboards.
In this example, we set up an integration with Superset which is an open-source Business Intelligence tool.
isitfit is a tool that analyzes EC2 utilization and that can provide you with recommendations around your usage.
There are multiple ways to monitor and be proactive about your AWS costs. It depends on how closely you want to monitor your AWS costs.
If you would rather not worry about having to set up cost monitoring on your own, ZenMetrics is an AWS Billing analytics tool that can help you with your cloud cost monitoring so that you can focus on growing your startup.