AWS Cloud Monitoring Dashboards
The following dashboard is an example of the type of "landing page" we often create for team leaders at Tableau. It quickly gives you the important metrics relating to your AWS deployment and provides interactivity that allows you to drill in and answer additional questions.
This dashboard can be used to investigate peaks in usage, identify stray instances, and track spending trends. We have implemented a version of this dashboard internally at Tableau to monitor AWS usage across our consulting team. By hosting the dashboard on Tableau Server we take advantage of several additional features. Setting up data-driven alerts gives us real-time notification if our AWS spend crosses a given threshold. Implementing user-based filters ensures that individuals don’t have access to sensitive information while also allowing managers to oversee their teams’ activities.
Many organizations also want to monitor more specific elements of their AWS deployments. Perhaps your company relies heavily on AWS databases, like Amazon Simple Storage Service (S3), or Amazon Elastic Cloud Computing (EC2). A dashboard like the one below gives you a more detailed view into a specific resource. Its interactivity enables a wide range of users to ask and answer questions about their AWS usage.
Looking forward, cloud deployments will only grow in size and complexity. We need to match that growth with increased transparency, governance and understanding. These sample dashboards are intended to give a brief glimpse into the value of using Tableau to analyze your AWS spend. However, every organization has different needs, structures and questions—the true value of this solution comes when you connect to your own AWS data and begin asking and answering questions about your business.
Getting Started: How to set up AWS cost reports in Tableau
Part one: Enable cost and usage reports in AWS
- Create a new S3 bucket to store usage reports with default permissions. Follow the steps here.
- Create a new AWS Cost & Usage Report. Follow the steps here. A few things to note:
- Reports are only available in a master account—not linked accounts.
- Select “Amazon Athena” for Enable report data integration for—the default file format for this is Parquet.
- Reporting begins within 24 hours of creation. You might not see files until tomorrow.
Part two: Connect to cost and usage reports with Tableau via Amazon Athena
- Download and customize the create-usage-table.sql file to reflect the S3 location of the reports enabled in part one.
- Update LOCATION to the S3 bucket where the report Parquet files live. (The format is “s3://
/ur/ / ”)
- Execute the Athena query to create the table.
- Open up the Query window in the AWS Athena console.
- Copy and Paste the updated SQL into the query editor.
- Click Run Query. This should take about two seconds.
- Add partitions. Run the following query to add partitions to the table: “MSCK REPAIR TABLE cost_and_usage;” Please note: You have to rerun this command anytime a new partition is added (each month).
- Connect from Tableau Desktop.
Part three: Load your data into the provided sample Tableau workbook
- Download and open the sample Tableau workbook AWS Cost & Usage sample.
- Add the Athena connection that you created in the previous section as a new data source.
- Replace the “Cost & Usage Sample” data set with the Athena connection. Read more here.
- Click through the sample dashboards to repopulate with your AWS data.