One of the most hyped tech ‘buzz terms’ of the past few years, big data is finally starting to deliver the ROI that some thought might never truly emerge. A recent AIIM survey of organisations working on big data projects revealed that early adopters are now seeing results and are using them to inform decision-making.

However, many companies don’t yet have the skills or the content to make big data work and see ‘dark data’, security and lack of expertise seen as their biggest roadblocks. What areas do organisations most need to improve on to maximise the potential of their big data?

Delivering On The Hype?

When we last conducted research around big data in April 2012, the two main themes were of hype and confusion. Users were waking up to the fact that it wasn’t about being crushed under mountains of data, but about extracting meaningful business insight from high volume data – although very few could truly be said to be achieving this.

On returning to the subject recently we were expecting far greater understanding and for more organisations to be using and reaping the benefits of big data. On the whole we did see that.

The number of respondents who don’t know what big data is has thankfully fallen from 14 per cent to six per cent. Around one-third of respondents said that big data will be an ‘essential capability’ for their business, up from 27 per cent from our previous big data research 18 months ago. Sixty five per cent of organisations surveyed admitted that they have disorganised content, 81 per cent have limited search capability and 60 per cent have only basic BI reporting.

This is not the best place to start for big data projects, but those that have started, are showing ROI. Of course, ROI is of the greatest importance for many organisations, so taking into account benefits, issues and costs, we asked for early indicators of ROI.

More than a third of respondents said it was too early to tell, but 60 per cent of big data early adopters considered their ROI to be good. Balanced against this are the 30 per cent who feel that at this early stage of deployment, the high cost of both expertise and technology can cancel out the rewards.

Barriers To Big Data Deployment

There are a number of issues that are preventing organisations from making more of their big data, a skills shortage being one of the main ones. Although the past 18 months have seen more trained practitioners join the ranks, there remains an overall skills shortage, whilst the tools available are still considered hard to use and somewhat expensive.

Security is another major big data adoption challenge – a potential show-stopper for nearly one in five organisations. Protecting personal data is the primary concern, but commercial and financial information is also sensitive. Security is also an inhibitor for cloud or SaaS deployment of big data tools.

The term “dark data” has emerged recently to describe content which is not under any degree of control, but which could hold useful information. This was borne out in our research, with 65 per cent of respondents considering their unstructured content to be somewhat chaotic (36 per cent) or not well indexed or controlled (29 per cent).

Only six per cent said they have full ECM in place, so organisations would be well advised to implement content management as a priority before embarking on big data analytics.

Big Data, Big Recommendations

Whilst big data has matured considerably in the last 18 months, the overarching situation is that many organisations are too immature in their content management, search, and basic reporting to contemplate big content projects. However, they are making technology decisions today with a view to a big data future and there is enormous optimism in terms of ROI and organisations’ ability to deploy big data effectively.

That optimism can be channelled further by following these recommendations:

  • Focus on what piece of information, business intelligence, customer understanding or incident prediction would transform your business
  • Identify the data that you would need to analyse in order to find this key parameter – is it in-house and accessible? If it exists outside of the business, how readily available is it?
  • If your content data is dirty, duplicated or inconsistently tagged, consider using a data cleaning or migration package to apply better policy rules
  • Source the most suitable analytics tools – a productised analytics toolset may be the best answer, particularly if it is well integrated with existing content and search systems
  • Get the strength of toolset required for the volume of your data and the speed at which it needs to be analysed
  • Acquire the skills needed to carry out the project – external consultants will get you started but recruiting new staff or training existing staff may work better in the long term.