Data disaster as inefficient processes and lack of collaboration case data analytics complexity headache

IDC research uncovers 54 million data workers around the world are facing challenges associated with the complexity, diversity and scale of their data.

The Alteryx-commissioned IDC Infobrief, The State of Data Science and Analytics, uncovered inefficiencies, ineffectiveness and wasted time as many organisations attempt digital transformations.

  • Eighty percent of organisations now leverage data across multiple organisational processes, but workers waste 44 per cent of their time each week being unsuccessful in their activities. Workers spend over 40 per cent of their time searching for and preparing data instead of gleaning insights
  • On average, they use four to seven different tools to perform data activities, adding to the complexity of the data and analytics process. In fact, data workers leverage more than six data sources, 40 million rows of data and seven different outputs along their analytic journey.
  • Over 30 per cent of data workers say they spend too much time in data preparation, a task that can often be automated.

Participants reported the lack of creative and analytic thinking, analytic and statistical skills, and data preparation skills as the highest-ranked skills gaps responsible for productivity issues, indicative of the pervasive that exists between data scientists and data workers in the line of business. To overcome these issues and more, many organisations are hiring Chief Data and/or Analytics Officers to streamline analytic processes, build a culture of analytics and encourage data literacy across the enterprise as part of their broader digital transformation strategy.

“Collecting data alone won’t digitally transform a business and the answer is not as easy as hiring a leader, a few data scientists or over-investing in disparate technologies. The key is to empower all users, many of whom are currently stuck in spreadsheets, to analyse data effectively to drive real, business-changing results,” said Alan Jacobson, chief data and analytics officer (CDAO) of Alteryx.

The State of Data Science and Analytics is based on a comprehensive survey of more than 800 individuals performing data functions across geographies, industries, company sizes and departments. The full IDC Infobrief can be viewed and downloaded here.