With all the talk about big data in so many different contexts, one area that’s been somewhat overlooked is the supply chain, yet it’s one of the biggest producers and consumers of data. Just think of all those purchase orders, invoices, confirmations and delivery notes swilling around customers’ and suppliers’ systems, and you can see how big the data challenge is. But the prize from making sense of it all is also huge.

Take the example of the food supply chain, where better tracking and tracing of ingredients within and across supply chains can help provide vital public health information, particularly during major health scares, such as the serious outbreak of salmonella in Germany in 2011.

In order to do this, data needs to be traced back to its source, right back to what seeds were used in the fields, what chemicals were used on the crop, when they were harvested, stored, processed, split and which cattle the crops were fed to. Initiatives such as animal passports and the Cattle Tracing System, which tags, records and tracks the movements of livestock across the EU, provide much greater visibility into the livestock side, but the don’t tell us what animals were fed on.

Tagging information as it flows through the supply chain and then applying rules so users can run queries at the front end is the key to this. The query might answer simple questions that the regulatory authorities and consumers are seeking answers to such as:

  • Where your food came from
  • What was applied to it
  • Where it was processed
  • How it was handled

But behind the scenes these would involve a huge amount of data points, and the cross-correlation of data across those. How we achieve that technically is the “secret sauce” of our approach to the problem – and we’ll be looking into this in more detail in a future post. But suffice to say traditional databases and heavy lifting integration brokers don’t work in these highly-distributed computing scenarios.

One example of the application of analysing big data in the supply chain is tracking chemicals through the supply chain. Chemicals manufacturers provide guaranteed yields if their products are applied correctly and they can be subject to claims from farmers if these yields are not realised. In order to protect themselves against what are potentially multi-million dollar claims, the manufacturers have to know who bought their chemicals, and where they were applied, effectively tracing a particular batch right through the supply chain.

Each batch of chemicals has a bar code that uniquely identifies it, and which market it was sold in. This information filters down through the supply chain from the manufacturer to the distributor at the wholesale level and right through to the retailer. As a final checkpoint, the farmer can even be encouraged to scan it in the fields as the chemicals are being applied. The process is driven by the manufacturer in whose interest it is to know what happens to its products – but it provides great visibility throughout the supply chain.

These principles can of course be applied to great effect across different supply chains – and protection against counterfeiting of premium-branded goods in the CPG supply chain would be a particularly good example. But the visibility it provides to the manufacturer about where its goods are sold and to what use they are put is the key prize. And it can only be attained through the gathering and analysis of supply chain’s very own big data.