It is becoming clearer than ever that everything we do is leaving a trail of data that can be analysed and used, whether this be the payments we make on a credit card, or the book we read on a Kindle, or our energy use through a smart meter.

In order to process the unprecedented volume and variety of data out there – what is commonly known as Big Data – a range of new tools and techniques are emerging, many from Internet giants such as Google, Amazon and Facebook, which depart from traditional approaches to database management.

Big Data can allow organisations to harness and make sense of vast quantities of heterogeneous data from disparate sources. These organisations may gain valuable insights into market trends and opportunities, and thus find themselves with a genuine competitive advantage.

There is a complex range of data types that Big Data solutions need to embrace and, as such, an ‘ecosystem’ of new management tools is emerging to cover the various layers of the data stack in the enterprise. While there was initially an incremental approach to traditional descriptive analytics, we now see that there has been a gradual emergence of more innovative models, and it is becoming clear that the benefits of Big Data multiply when aggregated with other emerging disruptive technologies, in order to develop effective overarching Data Strategy approach.

For example, Pattern-Based Strategy approaches bring a strong focus on predictive and real time capabilities, exploiting hidden gems in the underlying data. This strategy enables huge amounts of data to be analysed for previously invisible patterns, with insights being derived and actions initiated as close to Real Time as possible.

For less time-sensitive scenarios, traditional Data Warehousing can be enhanced with wider usage of advanced techniques like AI-based machine learning to reveal important hidden trends and correlations; and for the more entrepreneurial, there are greater business model disruptions, as demonstrated by the emergence of Data Analytics as a Service, from a combination of Big Data, Pattern Based Strategy and Cloud technologies.

All these are examples of how Big Data has to work together with its sister technology of optimised IT networks, rapid mobilisation communication tools and cloud computing, in order to fulfil its potential as a bit business enabler. Concepts like Data Transparency, Personal Data, ‘Data as a Service’, and Pattern-Based Strategy will alter the way business works and even encourage the creation of new businesses.

While specific characteristics of each vertical sector may differ and have to be considered when it comes to market adoption, these are certain concrete business trends that are already attracting attention; and the more transformative gains will always come from models that go beyond the simple “business as usual, but bigger” approach.

Big Data should not be misjudged as a latest “fad”. It has serious business substance, and those transformational business and organisational models centred on the emerging data-based mind set, enabled by new services such as Pattern Based Strategy, Data Transparency (OpenData) and Data Analytics as a Service, are the ones that will have a lasting impact. In the long term, after an initial confusion phase, traditional and Big Data approaches to analytics will converge in a unified Total Data platform.

This will require various factors to be taken into consideration – for instance, businesses will need to improve in areas such as High-Resolution Customer Knowledge, the analysis of contact with customer service centres. At the moment, the ability of businesses to process to process increasingly digitised and connected information is still unclear.

What these developments – and others such as 3D printers or cognitive computers able to replicate smell and touch – mean for society, laws and concepts such as individual privacy need to be debated and will prove a huge challenge to governments, businesses and individuals in the 21st Century.