Fast, Accurate Cloud Reporting: Making It a Reality
When we read through all the FinOps blog posts and books out there, one subject appears time and again: the need for readily accessible information. References to quality reports, near real-time reporting and concise report data come up with frequency when discussing what is required for companies to achieve FinOps maturity. Yet, none of these conversations delve into the big question: How do you get a cloud reporting solution that is fast, accurate and self-service?
There is no one easy answer to this question. Like business Intelligence (BI), a high-performant FinOps reporting solution requires data from multiple sources to be surfaced, aggregated and made accessible. Many considerations come into play.
Getting to “how” is a Russian doll of dozens of nested questions that include
- Who should build this reporting solution?
- What data do we use and where is this data coming from?
- How frequently should the system update the data?
- Who are the different report consumers and what metrics do they require?
- What reporting tool(s) should we use?
The who factor
Looking to your existing BI and analytics staff is a good start. You’ll want a full data team. Data architects, modelers and report authors have the combined required expertise to work with, marry and showcase data coming in from multiple sources. But their experience is limited to producing standard BI data points like GL reports, HR reports and Sales reports.
Analyzing cloud cost, performance and usage is a different beast. It relies on data found in the cloud provider billing statements and cost and usage reports. These reports are often a firehose of raw data and complex tables.
To build a cloud reporting solution, you’ll need to include someone with cloud expertise who can decipher that data, understand what billing, usage and performance metrics are required from the cloud, and know where to find them
Where is the data coming from?
The billing, usage and performance data from your cloud provider are absolutely crucial to cloud optimization. If you rely on multiple providers for cloud service, data from each provider and instance will need to be brought together and normalized.
If you are only interested in driving down cloud costs and ensuring right-sizing of resources, then cost, usage and performance data will suffice. However, focusing only on dollars spent misses out on the big cloud picture.
The ultimate goal of cloud optimization is to understand how cloud expenditures relate to business value. Which cloud costs are helping drive business and which are not? Understanding unit economics requires pulling in data from internal business systems and combining that with the cloud metrics.
For example, perhaps your cloud costs are increasing each month. With that data in isolation, that is a disappointing stat. However, if you match that statistic to internal business metrics, you may discover the number of customer orders is increasing at a faster rate than cloud costs. Efficiency is actually improving.
Data frequency
Cloud billing statements arrive well into the next usage cycle. Relying on them for cloud optimization is like using tomorrow's newsfeed to figure out the best way around the multi-car pile up blocking your route home right now.
When we are talking about the real-time agility of the cloud, is a report with after-the-fact information enough? The answer is No.
At a minimum, FinOps requires the following:
- Access to usage AND performance data
- A granular view into cost and usage data
- Frequent, timely visibility into that data
For years BI teams have adamantly (and rightly) fought against the concept of real-time reporting (RTR). For standard analytics, it's safer — and smarter— to be looking at the data coming from the close of the prior business day. Data is taken from all the operational environments, processed through ETL, loaded into a data warehouse, and then brought into the reporting system, e.g. Power BI, Tableau or other. This approach has long been the norm for most BI groups and the delay in getting data is not a problem.
For cloud cost optimization, however, data latency poses a significant issue. A gap of hours, or worse, days is not acceptable. Because cloud reporting needs to move as fast as the cloud does, it requires visibility into live operational data with regular, frequent data refreshes.
What reporting tools should be used?
FinOps reports need to be easy to access and also flexible so they can meet the requirements of different stakeholders. They also need to support numerous data sources—from AWS, Azure or other cloud providers as well as ERP, CRM and other business data. Independent data visualization tools fit the bill.
Visualization tools like Power BI can produce single-page dashboards where KPIs are presented all in one place. Reports are easy to share and different stakeholders can quickly view their respective top-line numbers and drill down to the reason behind the figures. A well designed dashboard will help to facilitate cross-departmental collaboration and rapid alignment on decisions.
Building the reporting system: accelerating the process
Building this type of cloud reporting is an investment in time and money. Data from multiple sources need to be cleansed, modeled and joined upstream. It requires data architects, data modelers, report authors, and patience. If you have the talent in-house, a dashboard can likely be built and ready to roll in the 3-4 months. But most existing analytics or IT teams are already stretched thin with their day jobs and will lack cloud expertise.
Envisor Cloud Analytics is a robust, expedient alternative to building a DIY cloud reporting solution. Out of the box, it delivers cloud usage and performance reports, presented in a Power BI-based dashboard and with granular, near real-time metrics. It is a powerful tool for jumpstarting FinOps reporting and realizing timely decisions. It can also be customized to integrate additional data sources to address unit economics.
Cloud cost optimization starts with real-time reporting to accurately gauge and make decisions about spend and performance. By incorporating traditional data-driven processes built on traditional data architectures, you can focus on the best ways to leverage the cloud, matching business decisions to cloud resource decisions.
Contact us to learn more about our cloud optimization products and services and how our team can help you achieve the quality reporting needed for FinOps.