Data is Not Oil or Butter, It's Footage

24 Feb 2024

Data is footage.

It’s not “the new oil”. Or “butter”.

It’s footage of specific scenes over specific periods of time. Of people, places and events in those scenes.

But here’s the thing: no one wants footage, they want films.

People want stories, and respond to them better than raw, unedited facts.

Pre-Production

Just like making a film from footage, data can only be made compelling when it is produced. Pre-production to distribution.

At pre-production, there should already be an interesting story to tell. Scenes must be chosen deliberately to include the relevant people, places and events.

There should be some end in mind at the beginning. Like a script and story boards in film. Or design and mockups in software.

Production

At production, a team needs to make sure quality footage is being recorded, just like a film crew. A device (often a piece of software) needs to be set up to do the recording. It has to have the right resolution and frame rate. Checks have to be put in place to prevent errors.

If the data is meant to convey truth–most commonly the case, and, in filmaking, a sign of great art–all relevant scenes must be captured. A common issue is missing footage from a critical scene. Or capturing the wrong scene entirely.

To establish truth, data has to generalize to other scenes (i.e. other people, places, events) at other times. Which is only possible when you sample broadly across scenes and times. The degree to which data is sampled broadly is the degree to which it is true.

Post-Production

At post-production, just like footage, data must also be edited (which may seem surprising). Blemishes need removing. Noise needs to be filtered to isolate the signal. People can’t focus on the story if there’s static.

Scenes also need to be spliced to weave together the story. They need to be sequenced logically in the form of a plot. Beginning, middle, ending. A hero’s journey. That’s what people respond to.

This is especially important when data is compiled from multiple disparate sources, and particularly if that data was collected passively as “exhaust” (which means it didn’t go through pre-production). Like making a documentary from old pieces of footage, it takes lots of research to understand the available footage and then even more work to synthesize it into a coherent whole.

Data should also be joined to other data, like video is to audio. In some cases, this is critical for the final output, like audio dialogue is for a film. In other cases, it enriches the experience, like setting scenes to music.

Distribution

Finally, and maybe most importantly, the final product must be broadcast and distributed to viewers. Films need to be shown in movie theaters and on streaming platforms. Data needs to be consumed similarly through a compelling visual interface.

So, no, data is not oil. Nor is it butter. It’s footage.

But no one wants footage, they want films.