Data Trends report

To address the data skills gap, businesses take inspiration from academia with training and certification

Digital transformation demands that businesses be data-driven, so organisations are making major investments – an average of $15.3 million (£11.3 million) this year – to ensure success. While some of these investments are directed towards technology and systems architecture, nearly half the 15.3 million (41%) is being allocated to skill development. This comes as no surprise, as more people have greater access to data than ever before, but are unable to "speak the language" of data. Workers with fundamental data literacy have the agency to understand and apply – to “translate” – the data they are increasingly exposed to, and are empowered to answer business questions and add greater value to their companies. Companies are solving this barrier to growth by cultivating analytics capabilities outside of data scientist roles, launching data literacy efforts to address the analytics skills gap. Gartner rates poor data literacy as one of the top barriers to creating a data-driven culture and experiencing its advantages.

While data literacy investments may be new to businesses, academic institutions have been developing critical-thinking and analytics proficiencies in students for years, as a requirement for decision-making and problem-solving in personal and professional environments. Institutions like the University of Pittsburgh, Carnegie Mellon University and the University of Edinburgh, among others, are producing the next generation of digital natives by offering data skills programmes and curricula for both undergraduate and graduate students across disciplines, producing a generation of data natives. The University of South Florida Muma College of Business has established a Citizen Data Scientist certificate programme, educating students who lack the technical expertise to collect and analyse data for business decision-making. With this data literacy, students are equipped with the analytics skills and literacy to navigate digital-first work environments.

Corporations are taking a leaf out of academia’s book by either establishing their own data skills programmes, centres of excellence and communities, or seeking out training with third-party data literacy programmes and external communities. We are seeing enterprises subsidise analytics certifications as well, as they begin to measure data literacy development. These investments also serve to level up the analytics skills of an existing workforce that lacks data literacy with those of incoming data natives.

Making data literacy a pillar of their digital transformation, Lockheed Martin launched formal data literacy workshops and courses to engage and educate employees across US campuses, with plans to expand in 2020. The global security and aerospace company hopes to roll out this training to people working in manufacturing and other non-traditional analysts roles. As a result, the Enterprise Analytics team has observed shifts in how employees treat data with respect to their roles and the added value they bring when they are data literate. Anthony Brown, Enterprise Analytics Leader at Lockheed Martin commented, “One thing we see [that] is important for basic data literacy is really understanding where your data is used and how it's used and why it's important. [...] But when they do understand that, they're going to care more about the data that they put in there, and that helps with data quality and data accuracy as well.” To manage and scale this investment in data literacy to over 100,000 employees, which is their goal, Anthony and his team are looking at their internal tool, Eureka. This Twitter-like platform has allowed people to establish a community, ask questions, post answers and provide each other support to improve the organization’s data visualisations and reporting.

Marina Brazhnikova, BI Manager of Data Visualization, has seen the explosive demand for analytics across her organisation, a large non-profit, academic healthcare system in the southern USA. To meet the current analytics needs of this data-driven environment – and to continue scaling as it grows – her team has focused on hiring and training staff to be data literate in order to deliver what clients, and the business, require. Marina’s team has grown from two developers to a team of 11. This increase in data literacy has created more engaged, empowered employees and improved operational efficiency overall.

This workforce of critical thinkers at Lockheed Martin, and other companies developing data literacy programmes, are empowered to solve business challenges with data, adding greater value to their organisations. Moving from an academic testing ground to the corporate space, data literacy is serving as “a core enabler of digital business – alongside people, processes and technologies”, according to Gartner. When organisations invest in formal training, communities to foster continued learning and certifications to measure data literacy, people can thrive with data and make a greater impact on the business – better prepared to be agile as digital transformation demands data literacy at all levels.

Domain and technology experts overcome the hype to create a shared vision for artificial intelligence and machine learning

Artificial intelligence (AI) is on everyone’s lips as the next stage of digital transformation, and organisations are diving head first into AI projects to stay competitive. Last year, IDC predicted “global spending on AI is projected to top $35 billion in 2019 and more than double to $79.2 billion by 2022.” Despite the increase in spending, most organisations are still failing to realise value from their AI investments because of an early focus on technology over practical use cases.

New technology and services have laid the foundation for a new stage of AI maturity. Instead of piloting AI projects in innovation labs, organisations are taking a more practical approach, starting with how they structure the teams that create, test and implement AI Projects. As Andrew Moore, Head of Google Cloud AI describes in his Harvard Business Review article, When AI Becomes an Everyday Technology, we are the age of “deployed AI”, where people are less focused on engineering and more focused on a “shared vision” for AI that outlines how AI and machine learning fit into existing processes and team structures. Instead of data scientists and engineers working on AI projects in a silo, these technology experts are bringing domain experts into strategic planning conversations to ensure that plans for AI and machine learning align with the wider business strategy.

Taking a collaborative approach can unveil the parts of a business decision that are best suited for AI and the parts that need human intervention. Let’s say you’re opening a new retail store, but you’ve never opened a store like this before. To forecast expected sales for the store, an AI-powered system might make a recommendation based on foot traffic or demographics in the area. But you would still need that human domain expertise to fill in the gaps around things like location visibility, competitor information or parking availability. The value of machine-learning recommendations only go as far as the people making the decisions. As Richard Tibbetts, Principal Product Manager for AI at Tableau explains, “The emergence of AI does not mean that an algorithm will tell you how to run a business. It will be the domain experts that help ensure that AI is adopted and trusted in an organisation.”

Immersing domain experts into AI and machine-learning conversations can have another strong benefit – creating educators and champions for AI throughout the business functions. Increasing collaboration between technology experts and domain experts encourages knowledge sharing on both sides. In an article from McKinsey Quarterly, Cameron Davies, Head of Corporate Decision Sciences at NBCUniversal shared a use case that involved annual forecasting. They decided to build and surface a set of machine-learning algorithms to help augment the process. In the early stages, they brought in a researcher to contribute to the project. He ended up becoming an evangelist in the business units and trained other people in how to interpret the recommendations.

Data experts gain useful knowledge into how the business uses data while domain experts elevate non-technical roles as advocates for data proficiency. Based on their relationships and expertise, domain experts will play a key role in putting the results of these AI projects into practice across departments and teams – ushering in a new wave of maturity for AI use cases.

Brands apply narratives to consumer data for more authentic and enlightening engagements

Which were your most liked photographs in 2019? What music did you listen to most? How much did you exercise? For consumers in the 21st century, these personal data stories are more accessible and inspiring than ever before. We’ve grown accustomed to companies collecting our data and we even anticipate their recaps of our behaviour in weekly or yearly summaries. And now, with these insights more pervasive than ever, brands are boosting consumer engagement by making our data stories more meaningful, convenient and interactive.

Data storytelling is an effective way for companies to illustrate our habits and engage us in ways that pure facts and figures could never achieve. Consider your personal banking transactions: you have ready access to detailed histories of all payments, deposits and charges associated with your account. But do you spend any time analysing these lists for insights? Then, at the end of the calendar year, the bank sends you an aggregated summary of your purchases, organised into categories and presented visually. Instantly, this different view of your data helps you spot trends and inspires you to ask meaningful questions: “Did I spend too much money on clothes when I could have been travelling or investing more in my hobbies?"

A quick glance at your spending is a simple example, but this type of annual recap demonstrates the power of contextualising data to help us extract insights. In a Gartner blog post, Use Data and Analytics to Tell a Story, author Christy Pettey writes, "It’s the context around the data that provides value and that’s what will make people listen and engage." As the stars of these stories, interacting with this data becomes a lot more exciting. And delivering the data right to your email or smartphone offers a level of convenience that makes exploring the data both approachable and worth the time.

As great as personalised insights that inspire new questions or actions are, some companies go a step further and prescribe recommendations in tandem with your data. Enterprise collaboration software Slack provides data about the channels and conversations that your organisation participates in, but also makes personalised suggestions – for example, leaving inactive or infrequently trafficked channels might help you stay focused on more productive conversations. (Just to be clear, no one is suggesting you stop sharing pet photos with your colleagues.)

These personal data interactions are becoming even more engaging with interactivity. In Spotify’s Year in Review recap, listeners were met with a quiz to guess which artist they clocked up the most hours streaming. Facebook also presents a quiz for you and a connection when you reach a significant milestone of online friendship. Questions may include guessing the total number of times you liked each other’s content, or selecting which of four photographs of the two of you was uploaded first. The quiz aspect not only makes interacting with your historical data more engaging, but offers an interesting opportunity to confront your preconceived notions of your own data story.

We expect to see more brands offering these kinds of data experiences and interactive insights. Personalised customer experiences help companies boost loyalty, satisfaction and new opportunities like repeat purchases, upselling or cross-selling. But it also evolves our relationship with these brands and offers significant value to us as consumers, including more meaningful experiences for people who don’t typically engage in data analysis. The more valuable and informative our interactions with our own data stories become, the more companies can empower us to change our behaviours for personal and professional growth.

Data as a resource shows equity is a prerequisite to improved employee satisfaction and business performance

Data is becoming a critical resource for organisations working to increase equity. Better data is allowing organisations – non-profits, but also government and corporations – to identify individuals or groups who are under-represented or face structural barriers to realising full representation, resulting in diverse benefits for all involved. Beyond addressing the moral imperative, establishing equitable environments and systems has positive impacts on both individuals and the organisations that initiate them. For corporations, transparency about their workforce data creates the opportunity to improve employee retention and satisfaction, while also accurately reflecting the communities and customers they serve. Research shows that diverse and equitable workplaces experience increased profitability, optimal operational efficiency and competitive talent acquisition, among other benefits.

Using data, organisations can analyse their diversity metrics at a granular level and use the information to identify and be successful at dismantling systemic inequities. People who are currently excluded from policy discussions or who are eligible for services can be seen, understood and supported through data in both the public and private sectors. Data is being used by non-profits to establish equity with advocacy tools for local officials to promote the rights of women and girls, while governments use racial equity dashboards to view where particular race, gender, economic or cultural groups are being left behind.

One example where data is being used to successfully establish equity is in the US educational system. Serving over 540 schools, Equal Opportunity Schools (EOS) works with school districts to analyse their data to improve the accessibility of advanced placement (AP) and international baccalaureate courses. Over the last decade, EOS has supported schools to enrol over 43,000 students of colour and low income students in these advanced academic pathways – students who are often overlooked and under-identified. Currently, 98% of advanced placement or international baccalaureate programmes in the United States do not reflect the racial diversity of their schools.

By using data, EOS helps schools to quickly and efficiently identify students of colour who can succeed in these advanced classes, creating access to educational opportunities not previously available and establishing greater racial equity in these programmes.

Similar analysis of workplace data can help private companies devise stronger diversity and inclusion programmes and diagnostic tools to quantitatively measure their efforts. With insights about pay, gender and racial equity, leadership is able to see who is and isn’t being hired, whether programmes are accessible to all, and how effective the existing programmes are.

Transparency around workplace data facilitates employee trust and satisfaction. A happy employee is a more productive, better-performing one, with a Deloitte report showing that inclusive organisations can deal with individual performance problems 3.6 times better than those that lack intentionally inclusive talent strategies. Human capital is often the largest investment of any business and efforts to be more inclusive result in improved retention and talent acquisition, producing long-term dividends.

Having a diverse, equitable workplace affects profitability too. In the McKinsey research study, Diversity Matters, a connection was found between financial performance and more diverse gender and ethnic environments. “The companies in the top quartile of gender diversity were 15 per cent more likely to have financial returns that were above their national industry median. Companies in the top quartile of racial/ethnic diversity were 35 per cent more likely to have financial returns above their national industry median.” This relationship between performance and diversity reinforces the critical role of data as a resource – and the transparency around it – to establish equity in the workplace for employee satisfaction and organisational success.

Executives share responsibility for the future of data & analytics

Organisations are investing trillions of dollars to become more data-driven, but a 2018 McKinsey Analytics survey found that there is a growing gap between the “leaders and the laggards” – those successfully completing data initiatives and those who struggle to see a return from their efforts. The most successful data-driven companies invest in culture as much as they invest in technology, embedding data into core business functions. And organisational culture – including data culture – starts at the top.

The rise of the chief data officer (CDO) in recent years signalled the growing value of data in the enterprise and the need for executive sponsorship around data and analytics initiatives. CDOs were tasked with bringing all aspects of the business under one analytics strategy, bridging the gap between IT and the business. Today, CDOs are still driving digital transformation efforts, but now all executives are committing to treat data and analytics as a shared responsibility.

All functional leaders are expected to inform their organisation’s data and analytics strategy with critical inputs from their own teams. Leaders should understand how people use data, share data and present data within their area of the business. “Your data strategy must incorporate the needs of the entire organisation to ensure it supports the overarching business goals,” shared Mike Hetrick, Senior Product Marketing Manager at Tableau. “Data and analytics strategy must cover people, process and change management.”

Another element of this responsibility lies in executive sponsorship. Beyond strategy, leaders are also expected to empower employees with the tools and training they need to create a data-driven future. “Executive sponsors need to consider not only how to scale technology for the entire organisation, but to provide adequate support, training, change management and the ability to overcome any organisational roadblocks,” said Hetrick. “Part of that sponsorship role is modelling behaviour and being a vocal advocate for data.”

This expectation rolls all the way up to the CEO. For example, at leading South African aviation company, Comair, the business intelligence team wanted to open up self-service analytics to more people throughout the company. To help educate and onboard employees, they had an idea for a data stewards programme. The programme would embed data experts within business functions to assist new users and promote a self-service approach. With buy-in from the CEO, the programme made notable strides in creating a culture of analytics. “We are very lucky that our CEO has a keen interest in data,” shared Liezl Brouckaert, Business Intelligence Manager at Comair. “Without CEO buy-in, driving a business intelligence programme is close to impossible.”

This approach means the entire C-suite will have a more informed perspective about data and analytics, making data a key part of strategic conversations and board meetings. When this collaboration starts at the C-level, data and analytics initiatives have the support and advocacy they need to transform the way an organisation fundamentally operates – embedding data into the fabric of day-to-day conversations and behaviours for every department and role.

Organisations expand data management participation to support data-driven decision-making at scale

Have you thought about the status of your data? Do you know where it lives, who is using it and how often? Do people in your organisation know which data is appropriate to use for making decisions, and how to access it?

Data-driven leaders are differentiating their organisations with new solutions to integrate their distributed data pipelines. The roles and processes for how data is prepared, curated and shared across the business are shifting, alongside the evolution already happening within data technologies. IT should take a leaf out of Ghostbusters’ book: although it’s not advisable to cross streams, sometimes this can solve the biggest, scariest problems. In this case, blurring the lines between IT and business responsibilities around data management, organisations will no longer be limited by functional boundaries, enabling enterprise-wide data integration at scale and empowering people across the organisation with the right data at the right time.

Solving these data integration challenges is imperative for maintaining internal and external compliance, as well as enabling the organisation to obtain a complete picture of the business, understand customers and find new business opportunities. Many organisations are working to identify, prepare, govern and make widely available the data that most benefits the entire organisation. And where there’s success, data management is changing – beginning with technologies.

Solutions providers are increasingly incorporating data management capabilities with broader users than just IT in mind. And as functionality becomes more embedded in business users’ workflows – including in analytics platforms – employees will take a more active role in data management responsibilities that were traditionally owned by IT. This is the natural next step in the evolution of self-service in business intelligence. Organisations first broadened data access, then enabled deeper exploration and new types of users to author analytics content. Now, some business users are able to get involved with the data itself. At each of these stages, IT learned how to balance governance and self-service so business users could absorb some of the load. Crossing these streams will be critical to managing data and analytics as its adoption scales across the enterprise.

Self-service data prep demonstrates this evolution well. Various aspects of the traditional extract, transform, load processes can now be executed in a self-service manner using modern tools that integrate with the analytics workflow. This not only allows for greater ad hoc discovery, but can serve as a starting point for new use cases to be tested before being scaled to the entire organisation. And it’s a win-win: the business is empowered to take on greater ownership in data management, thereby reducing the (traditionally) lengthy development lifecycle, and IT is freed up to take on the highly specialised work that they are in the best position to do.

Another example of this evolution is the data catalogue – an inventory of data assets that helps define and qualify data while tracking relationships between data sources, content and users. In organisations with distributed accountability for integrating and managing data, a catalogue is important as a central view of what is going on with the company’s data assets. Catalogues can help to discover and promote data more easily, understand its relevance and freshness, and monitor who is using certain assets.

Modern catalogues are surfacing this valuable information and adding business context right in the flow of users’ analyses. As more data is integrated and becomes broadly available in the organisation, people learn to understand the quality of their data and how to use it, while staying within policy guidelines. This is where data literacy is critical – as a minimum, users will learn to interpret data indicators and identify trusted, relevant data. When data users can be their own data stewards, this lessens the burden on IT and helps ensure responsible use when making decisions. Then, more sophisticated users with the right skills might go on to participate in self-service data prep, certify new data sources for the organisation to use, or add business context as metadata in a curation process.

IT and the business can reach new collaboration and harmony with the lines of functionality and responsibility blurred. With a tailored approach that includes business users and objectives, broader data management initiatives will succeed because IT and the business can share in the efforts to increase visibility, discoverability and trust of their data environment. This also means the organisation is empowered to identify and prioritise the data assets that are most broadly valuable, and better support governed data and analytics at scale.