The furore surrounding the UK Government’s plans to require ISPs and telcos to store all available information on emails, texts, phone calls and web history, demonstrates that the Government is clearly becoming aware of the incredible power of metadata.

The data the Government is hoping to tap – and allow access by law enforcement officials – is known as descriptive metadata. It is now technically realistic to capture and meaningfully analyse metadata despite the large volumes of raw data that will be generated by the 52.7 million or so UK citizens as they go about their digital world.

By generating and storing descriptive metadata on people’s digital activities, the government can conceivably profile an enormous number of users, and then take a closer look at specific individuals when necessary. What I find fascinating, however, is how we’re realising the power of metadata, how valuable it is, and how much people feel threatened by the Government’s use of it: who they are communicating with and how often, which sites they are accessing and who they are calling/texting.

The data governance and storage requirements that the Government is planning to impose on ISPs and CSPs is a reflection of a greater understanding amongst business professionals that metadata is the key to better management and protection of their digital data assets.

It is also the key to a more efficient and smarter organisation: one that knows where its data is, who is accessing it and what is happening to it, at all times. This goes way beyond IT security and into the realms of data governance and intelligent digital collaboration.

Against this backdrop, it’s no wonder that the Government is also becoming interested in metadata, although it is useful to understand that there may be a question mark about who actually has the rights to access data on specific members of the public.

Some experts would argue that metadata generated on behalf of an organisation by its employees is fair game for the organisation to capture and analyse, but it’s an open question about who owns – and should be able to make use of – the metadata generated by citizens in their normal day-to-day digital life.

The Government has made it clear that some information must be protected and private, as witnessed by the introduction and enhancement of privacy legislation and laws on medical records, so you have to ask why metadata is not subject to similar levels of protection.

A key example of this is the intelligent interpretation and extrapolation of metadata. Whilst a celebrity might request their medical records be sealed to protect their privacy, if their phone metadata shows they made repeated calls to a specialist in a particular type of disease, it’s relatively easy to work out what health problem the celebrity may be suffering from.

The power of metadata association is one of the reasons why IT professionals are looking at metadata framework technology to better understand how their organisation uses data, what data may be sensitive, and where it may need to be locked down—as well as to keep the metadata safe in a trusted system. When metadata is allied with content, the results can be devastatingly accurate.

The volumes of data that UK citizens generate are eye opening, as the August 2011 Internet access update from the UK’s Office of National Statistics clearly shows:

  • 45 per cent of Internet users use a mobile phone to connect to the Internet
  • 6 million people accessed the Internet over their mobile phone for the first time in the previous 12 months
  • The use of wireless hotspots almost doubled in the previous 12 months to 4.9 million users
  • 21 per cent of Internet users did not believe their skills were sufficient to protect their personal data
  • 77 per cent of households had Internet access

Those organisations that have not yet harnessed the power of metadata are blissfully unaware of what is happening to their data – both on a real-time and historical basis – so the introduction of new data retention and indexing requirements in the Queen’s Speech is going to be game-changer for many companies.