TDWI Checklist Report: Data Discovery and Unified Information Access

This Checklist Report focuses on helping organizations understand how data discovery and UIA tools address new requirements for data access, reporting, and analytics, particularly for nontechnical users in lines of business and operations. The Checklist examines how these new tools can help organizations add breadth, depth, speed, and flexibility to users’ pursuit of information insight from both structured data and unstructured content. The report closes with guidance for developing best practices and ensuring that IT data governance and provisioning concerns are addressed while meeting expectations for greater user self-service and flexibility.

We've also pulled out the first several pages of the whitepaper for you to read. Download the PDF on the right to read the rest.

Demand for business intelligence (BI) and analytics continues to be strong, but expectations are changing. At one time, business users who knew of BI understood and accepted that it involved a steep learning curve, batch queries on limited historical data, static reports, and long waits for IT to build new applications. Now that BI is topping research surveys as the most desired application, emerging expectations are for easy-to-use, self-service, and graphical environments that let users work interactively with and explore different types of data. Users want rapid application development and deployment capabilities for meeting changing requirements. From their dashboards and other kinds of portals, they want to access and analyze not only structured data, but also unstructured content.

Organizations struggle to meet such elevated expectations with most existing BI systems. This has created a prime opportunity for data discovery tools, which offer a fresh approach to data access, reporting, and analytics. Equally interesting are unified information access (UIA) tools, which share many of the qualities of the data discovery category but focus on integrating BI, search, and analytics for both structured and unstructured data. Leading tools in both categories give users better control over their environments, including the ability to test analytics in sandboxes and do rapid proof-of-concept systems before wider deployment. Many use new technologies for in-memory analytics and extended or hybrid SQL for different types of data such as spatial or geographical location data.

This Checklist Report focuses on helping organizations understand how data discovery and UIA tools address new requirements for data access, reporting, and analytics, particularly for nontechnical users in lines of business and operations. The Checklist examines how these new tools can help organizations add breadth, depth, speed, and flexibility to users’ pursuit of information insight from both structured data and unstructured content. The report closes withguidance for developing best practices and ensuring that IT data governance and provisioning concerns are addressed while meeting expectations for greater user self-service and flexibility.

Understand Requirements for Self-Directed Data Access and Analysis

Organizations have traditionally had a tough time extending business intelligence and data analysis capabilities to nontechnical users working in operations and lines of business (LOBs). The standard approach, in which IT drives BI development and deployment, can be fraught with difficulty. When IT seeks to gather user requirements, developers try to ascertain what users need in terms of data, reports, and analysis capabilities. However, this process can overlook two central issues: users’ need to explore information and their role in authoring reports and assembling dashboards.

Users are not uniform. If enterprise BI tools lack flexibility, they leave users who are at different levels of maturity unsatisfied. Even individual users can have varying degrees of experience or clarity about what kind of reports, analysis, and visualization they need for different data requirements. The old adage that you don’t know what your second question will be until you have the answer to the first one has never been more true. For some objectives, parameterized reports set up by IT are fine; for others, users may want to be in full control of the design, building, and sharing of analyses and best practices. The need for flexibility is a big reason why there’s strong interest in tools that enable self-service BI and analytics, which can allow users to shape BI to their roles in business processes rather than depend on IT.

Across the spectrum, users increasingly need—and expect—an integrated view of structured data and unstructured content. To complement the “facts” delivered by structured data that tell them what’s happening in their areas of concern, unstructured content from sources such as customer comments, social media/sentiment, news, forms, and more can help users see context and understand why things are happening the way they are. Yet, once again, it’s difficult for IT developers to customize unstructured content analysis for each user; it may be better to give users tools to do it for themselves.

With data discovery technologies maturing, IT and LOBs should change requirements-gathering procedures to promote opportunities for users to perform self-directed data access and analysis.

Put Users on a Fast Path to Information Insight

In most industries today, to the fastest belong the spoils. Poor information flow prevents organizations from being first to market with new products and services. If marketing functions have to wait several months for the development of BI applications that allow them to analyze data about campaign performance, opportunities can be lost. Supply chains are suboptimal when managers lack access to data; product flows are out of sync with demand; and customer feedback has little impact. Managers in operations cannot allocate precious labor resources to respond to immediate and projected needs.

Increasing speed to insight should therefore be a primary objective of deploying data discovery tools. In LOBs and operations, users need information to make daily decisions. If BI is too tightly controlled by IT and tools are geared to power users and experienced analysts, the rest of the user population has to function with incomplete and less timely information. Unstructured content comes to them haphazardly, preventing users from gaining a complete picture. Speed is not just about real-time data updates; it is also about giving users a faster path to all the information they need.

The development of BI dashboards has been a huge boon, bringing data access, display, and analysis to a wider population. BI and data discovery tool vendors are competing to provide users with the ability to personalize how they look at metrics, key performance indicators (KPIs), charts, and more. However, with expectations for dashboards rising, IT developers are under pressure to keep up with the pace of business for access to larger and more varied data volumes. Self-service capabilities that enable users to tailor dashboards can be crucial to reducing development backlogs.

Three key issues in evaluating whether dashboards are increasing speed to insight include relevance, completeness, and depth. Dashboards should give users information, both structured and unstructured, that is relevant to carrying out their responsibilities. Dashboards should bring together access and visualization of internal and external data sources, including unstructured content, spatial data, and documents, so that users gain a complete view. Finally, dashboards should enable users to access detailed data and take their discovery deeper.