At any given moment we are producing millions of terra-bytes of information. This information comes from multiple sources, from audio files, video, visual communication, emails and mobile. Research shows that 90% of this vast amount of data has only been created over the last three years.

This ‘Big Data’ is one of the hottest topics around today. Big data is a loosely-defined term used to describe data sets that are so large and complex that they become awkward to work with using on-hand database management tools. Difficulties include capture, storage, search, sharing, analysis, and visualization.

Big Data is often mentioned in the same breath as Apache’s Hadoop that enables applications to work with thousands of computational independent computers and petabytes of data. However, the term Big Data is now used in other contexts within the IT industry, even if not strictly adhering to the definition of the term. Many IT professionals today associate Big Data with the more familiar term BI, assuming that Big Data has made BI as we have come to know it obsolete.

Of course, proponents of BI do not agree as can be seen in discussions in professional forums and the many fascinating discussions held in magazines in the IT world. The truth is that the hysteria among BI advocates is due to the fact BI vendors, as well as database vendors, do not have a real answer when it comes to handling Big Data.

Traditional suppliers of BI know that they do not have viable solutions based on existing database technologies for those organisations that are looking to take advantage of Big Data. A CIO of one of the largest organisations in the US was recently quoted in InformationWeek as saying “ETL, Data Storage and BI will provide a better view of the data but will not bring you real problem solving”

So where’s the value in Big Data? Let’s look at the top five benefits it can bring. Firstly, additional value is created by making the data transparent, and increasing the amount of access to that data. Secondly, by amassing transactional data, organisations have access to more accurate and detailed performance information on all relevant aspects of the business, whether stock related, sales related, hours worked etc. This is turn allows innovative companies to conduct controlled experiments to better tune the business and improve forecasting.

The third major benefit of Big Data is that it allows organisations to fine tune customer segmentation which enables them to provide precisely tailored products and services. The fourth key point is that with Big Data it is possible to introduce sophisticated analytics which obviously can greatly assist with decision making. The fifth major benefit is that it can be used to improve next generation products and services, for example by introducing feedback mechanisms such as sensors into products, pre-emptive maintenance and services can be introduced before the customer experiences any issues.

It can be seen then, that the science of predictive analytics can take advantage of historical data, recognising the patterns locked within Big Data that can be used to create informed future strategies. For example if I added graphics to an advertisement what effect would that have?

All well and good, you may say, but what’s that got to do with mobile? Well it is interesting to note that right now there are approximately 600 petabytes of data passing through mobile devices every month, an eightfold increase since 2000 and this is forecast to grow to around 6,000 petabytes a month by 2015. It’ll be no surprise that Big Data solution vendors see the big bucks in the mobile world. This may well be a natural outcome, but if this phenomenon results in petabytes of unasked for data arriving in the mobile user world then this clearly could have the reverse effect.

When we consider mobile marketing, I think there is no doubt that many more business will enter this market through applications, text and the expected boom in mobile payments. This has to be a given when you consider that there are more mobile users than people with bank accounts, access to the internet or even electricity. Their challenge, when introducing new mobile branding will be to create personalised and effective customer engagement in order to succeed in the mobile marketing arena.

It could be that the mobile marketers who use Big Data effectively are the most likely to succeed and be the most popular in customer satisfaction surveys.