Data Filters, Rational Bandwidth & Iraq

It was between US troops surges that I found myself sitting in a hospital in the sunny south of Iraq. I was attached to a medical team invited in by the Ministry of Health to create a regional medical center that, it was hoped, would generate the economic drivers to counter Iran’s sizable local investment in mayhem and chaos. We were welcomed with both open arms and, strangely, a dragging carcass of resistance.

The problem was that Saddam Hussein’s Arab socialism had reduced even the most educated into mere cogs. When that happens, people will do what they have to for self-preservation. In this case, that meant hoarding information. The local rule was “watch your back” and the best way to do that was to know something the other cogs didn’t. The result was system paralysis, despite that being in no one’s best interest.

What happened there wasn’t unique, nor was it entirely irrational. Without a secretary, the chief doctor pretty much ran the entire surgery operation in his head, so it wasn’t incompetence either. It was an example of what the economist Herbert Simon called “Bounded Rationality.” Basically, people are sensible enough and make more or less rational decisions based on their incentives and the available information – drawn from immediate real-world data and matched against known patterns the subconscious recognizes to be true.

This is not unique to developing economies and conflict zones. Modern rich-world organizations routinely assume a level of information and unbound rationality that humans simply do not possess, much less act on. The fabled homo economius - that rational thinker operating on complete information – is about as elusive as Sasquatch: We’ve all heard the rumors, but have never run into one in the wild.

Counter-intuitively, having a fire-hose of free-flowing data will often put you in the same place as information delayed or hoarded. Late information causes a decision to be off target. On the other hand, when it comes too fast, or is premature, it amplifies the nervous chaos creating instability in the system. Take the ham-handed government response to the Covid pandemic: Information moved too fast, creating a torrent of real-time data without vetting the raw numbers, or having enough knowledge (it was a novel virus) to develop real insight about how to logically move forward. The only palatable move, we saw, was to simply appear to be doing something (anything!) even if it didn’t make much practical sense.

Whether we are operating without crucial data or have been soaked in too much of it to process doesn’t truly matter. Either way, we misperceive risk by chronological or psychological distance, exaggerating the vivid present and discounting the past. From that perspective, we make a fairly rational and self-serving guess in the way we’ve evolved to do so - come up with a good enough answer to get you to the next decision and rationalize hell out of it later.

Granted, there isn’t a whole lot you or your team is going to do about evolution, but it helps to remember that:

• Data is not information;

• Information is not understanding;

• Understanding is not certainty; and

• Certainty proves nothing.

---

The Field Guide

I learned a lot about stressed systems in Iraq, and so did Donald Rumsfeld, for that matter. He famously scooped knowledge into three piles:

1. Known Knowns: What do we actually know?

Separate verified facts from assumptions, find out what you’re giving too much weight (recent events, emotions, that sort of thing) and what information is arriving too quickly to be useful.

2. Known Unknowns: What do you not know that matters?

Not every unknown is worth investigating, but it helps to identify the ones that might change a critical decision. This is also where you ask: What information is missing because someone has an incentive to withhold it. If Iraq’s guiding principle was “watch your back”, we might say that the G7 rule is “cover your ass.” It happens.

3. Unknown Unknowns: This one is tricky…

A few of good questions to ask are: What would we believe if the opposite of our current assumption were true? It helps unwind confirmation bias and presents one of the few reason why you might want to hire an aspiring novelist. The second question is to ask: What would make you change course? Here you define a trigger before it hard to make a reversal.

The lesson here isn’t that people are irrational, but that they are making sensible decisions that, collectively, produce terrible results. Everyone is acting rationally within the bounds of their own subsystem while the organization makes decision that are in no one’s best interest. Your job isn’t to collect more information, but to know when you’ve got enough to make a solid decision - and what will trigger a reassessment. Assuming you haven’t hired incompetent maniacs, all that is needed is better decision architecture. To do that you need to reduce information hoarding, filter information overload, question assumptions through some channel of dissent, and make reversibility part of the decision.

And if you run afoul of Sasquatch, give him my number. I’ve got questions…