Shared Data Services or Sending Messages?

COVID exposed the world's lack of supply chain resilience. To have resilience one needs visibility, and to have visibility one needs data. This presents a challenge in an industry dominated by data silos; that is information systems that struggle to communicate with other information systems.

These data silos exist not only at the organisational level, but also within an organisation, or even within a facility. In the latter case different departments, under the same roof, use humans to exchange data between systems - editing files, printing paper, and inputting data at the keyboard.

So what can be done?

Data Standards

I am a huge proponent of data standards (I'm even working on one!). Machines, like people, need a common speech to communicate. Data Standards provide a shared vocabulary usable by multiple parties. When sharing data with another organisation, much needless discussion can be eliminated by use of a preexisting standard - there's no debate or different interpretations of what something means, the standard itself has the final word.

But data standards are only the beginning. Agreeing on the meaning of the data does not mean it can be shared.

Public Digital Infrastructure

One way to share data is to persuade companies to jointly interact with a third data service. Indeed this is the approach taken by Freight Logistics Optimization Works ("FLOW"), and seems to be bearing fruit. The companies involved would struggle to even have the conversation of coordinating a third party platform between them. But with an actual, neutral third party they all communicate with - the US Federal Government - they can all be brought to the table.

Conway's Law

The success of FLOW is in many ways a realisation of Conway's Law, first coined by computer scientist Melvin Conway in the 1960s. To wit:

"Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure."

And hence FLOW. All companies involved communicate with the US Federal Government. And so all of them can be persuaded to communicate with a data service provided by the US Federal Government.

However, by Conway's law, in the absence of a suitable third party, third party data services are unlikely to succeed. And it's impractical to expect governments or NGOs to provide all the public digital infrastructure needed

Messages

So what other communication structures exist in supply chain?

Well, your node (in the supply chain) talks with upstream and downstream, by necessity. But it's not necessarily the case that your upstream talks with your downstream - so a shared data service is unlikely to succeed here.

But what can succeed is a decentralized approach. Each data service is still an island to itself, but unlike a Data Silo they are islands with a postal service (or indeed, a submarine internet cable!). They are set up to send and receive data, machine to machine, system to system. They are no longer Data Silos, but Data Services. To quote Pat Helland:

"Services are different than the classic application living in a silo and interacting only with humans in that they are interconnected with messages to other services. Services communicate with each other exclusively through messages. No knowledge of the partner service is shared other than the message formats and the sequences of the messages that are expected"

Messages are pieces of data send between machines, usually representing something that has happened, or an event that has occurred. Well designed messages have a globally unique ID, mention where/when/what/why and are self contained - in that they are a useful piece of data on their own within having to reference too much else. Supply chain is full of natural messages; Purchase Orders, Bills of Lading, Customs Declarations, Production Orders, Inventory Reports. When these are standardised, they become the Lingua Franca of messages sent across the supply chain.

(The concept of distinct services sending messages to each other is not a new one. Pat Helland wrote the above in 2005, but I would argue the pioneering research was Carl Hewitt & Henry Baker's paper from 1977.)

So how does this work in practice? Machines send each other messages, and each node in the supply chain becomes enriched with its neighbours' data, empowering local decision making. This can have a compounding effect - Node B is enriched by messages from Node A, which subsequently improves the messages it can send to Node C. This allows data sharing to develop in an organic, piecemeal way, in the absence of a central authority.

Shared Data Services and Sending Messages Compared

In many ways, shared data services like FLOW are the ideal solution. It's a simple, powerful model. But while I do see the success of FLOW as a welcome development, and I hope to see its model be more widely adopted, I don't think it will ever fully bring us the end-to-end supply chain visibility we all aim for.

Message sending, on the other hand, is organic. It can start small, be developed piecemeal, and expand through a supply chain to the benefit of everyone, without the need for a trusted, neutral and resourceful third party. The methodology and mechanics of message passing are well established and well studied. Let us not wait around and let perfect be the enemy of good. Let's work on data standards and start exchanging those messages - one supply chain link at a time.