The last few years have brought a wave of changes for business intelligence (BI) solutions. A set of redefining technological trends is reshaping the landscape from a slow and cumbersome process practiced mainly by large enterprises to a much more flexible, agile process that mid-market companies as well as individuals can utilize.
This GigaOm report explores the key features that influence the evolution of agile business intelligence and takes a look at the BI landscape under this light.
We've also pulled out the first several pages of the whitepaper for you to read. Download the PDF on the right to read the rest.
Executive summary
The last few years have brought a wave of changes for business intelligence (BI) solutions. A set of redefining technological trends is reshaping the landscape from a slow and cumbersome process practiced mainly by large enterprises to a much more flexible, agile process that mid-market companies as well as individuals can utilize.
This report explores the key features that influence the evolution of agile BI and takes a look at the BI landscape under this light. At first glance, polarization seems to exist between traditional BI vendors, who are focused on extract, transform, and load (ETL) and reporting, and the newcomers, who are focused on data exploration and visualization, but a closer look reveals that, in fact, they converge as adoption of useful features is taking place across the spectrum.
This report will illustrate for both the traditional BI vendors and the newcomers that:
- As the market is expanding, features such as cloud support and embedded domain-specific knowledge in BI solutions are key. Initially, the benefits will be more obvious to those smaller players who do not have the resources for in-house infrastructure and extended internal projects and who are driven more by needing immediate results. Over the long run, however, these features can benefit all types of organizations.
- Ubiquity and mobility are key features of data today; therefore, the ability to support a multitude of data sources with as little effort as possible – integrating them and accessing analysis results via a multitude of channels – is important in order to keep up.
- We are shifting from static reports to interactive visualization. The focus is also shifting from having an overview of metrics to being able to discover what are the causes and effects of the phenomena the metrics express.
Motivation and market definition
Over the last few years, cloud, Software-as-a-Service (SaaS), and Platform-as-a-Service (PaaS) adoption have been increasing for several reasons, including:
- Smaller startup costs
- Faster implementation time
- Reduced administrative overhead
- Increased scalability and reliability
- Flexible pricing models
Concurrently, we are witnessing an explosion in the amount of data generated and accumulated over time, a phenomenon that has been labeled “big data.” The need to deal with massive amounts of data is nothing new; in some organizations and contexts, data generation and accumulation has always been massive, and the need has existed not only to store it but also to derive value from it. What has changed is the extent of this phenomenon, both in breadth and depth.
More organizations produce and rely on data, increasing the breadth of the big-data phenomenon. While IT infrastructure can be applied to support the operation of organizations in every domain, in the past, only a handful of them could afford the financial and know-how requirements. Nowadays, we see not just small-to-medium enterprises but in many cases also individuals leveraging software tools and services to support a wide range of their activities, from communication and networking to human resources management and from accounting to logistics.
Organizations produce more data and rely more on data, increasing the depth of the big-data phenomenon. This doesn’t apply just to legacy data being accumulated over time but rather applies mostly to an increase in the tempo of new data generation. As former Google CEO Eric Schmidt noted, every two days now we create as much information as we did from the dawn of civilization until 2003. This impressive increase can be attributed to new data sources for organizations, both internal (more activities being automated by software, thus more related data) and external (from sources such as social media or curated data sets).
The rise in capabilities and the democratization of IT pose new opportunities and challenges. These factors are the backdrop against which BI solutions have emerged from a niche product addressed to high-level executives in large organizations to a widely used tool for decision-makers and teams at every operational level and in any type of organization.
Widespread adoption of business software such as productivity suites, customer relationship management (CRM) suites, and enterprise resource planning (ERP) suites has not only commoditized the software itself but also the associated skills. Now people are increasingly familiar with the data-generation process and increasingly able to make correlations and derive insights based on data.
Traditionally, BI solutions were addressed at organizations that needed them and could afford them — big enterprises with sufficient resources to acquire and sustain the in-house infrastructure and know-how required for their operation. To support this, those organizations needed an operational stack and associated architecture consisting of infrastructure, skills, and teams of skilled professionals.
In this kind of architecture, data must be integrated in enterprise data warehouses, leveraging heavy-duty hardware, specialized software, and skilled labor. Then it must be formed and interpreted according to the syntactic idiosyncrasies and limitations of the query-processing software that mediates it. This process, in turn, is mediated by a team of specialized professionals in charge of interpreting, formulating, executing, and presenting queries.
Traditional BI
The investment in terms of money, time, and skills is not trivial. Not every organization is willing or able to commit to it along with a long chain of actions before any actionable insight can be obtained. Thus, a gap is created between those that can benefit from BI solutions and those that can afford them. Even those able to implement a BI system are not getting the most out of their investment.
This is where disruption factors come into the picture. Today’s organizations need short implementation times, little or no upfront investment, and increased flexibility. These are precisely the reasons why cloud infrastructure and BI solutions are such a good match and complement each other’s strengths and capabilities.