What makes a better decision? The data you exclude matters most

By John Zealley and Paolo Valdemarin

AI without context is a deluge of data trying to pass itself off as information. The way out is not more data. It is knowing which decision you are trying to improve, what would make it better, and then being disciplined about everything you leave out. The surprising part is where the material that survives the cut tends to come from: not from the documents a company keeps, but from the informal exchanges it rarely records and the options it considered and rejected.

This piece grew out of an afternoon in London comparing notes on how an organisation can know what it knows. The conversation was recorded and analysed by the system we were discussing.

Start from the decision, not the data

Few executives start from a blank sheet. They usually know the three or four things they need to get right, and those are the decisions worth improving. The first question is therefore not what data the company has but what it would mean, for this business, to decide better: faster, at lower cost, with better timing, or on better information. The material considerations differ from one company to the next, and until they are named, no system can know what to look for or what to leave out.

An earlier generation of data analysis had this discipline built in. If a drug took five years to reach market and better processing of trial data reduced that to four, the value of the extra year under patent could be measured, and everyone knew which curve they were trying to move. The same question applies anywhere: what outcome are we trying to bring forward, or what cost or risk are we trying to reduce? Asked of a leadership team, “what would it mean for you to make better decisions?” is a far more productive opening than asking for their AI strategy.

The data you exclude matters most

You have to dig deep to understand a business, and you must not drown people in what you find. The discipline has three steps: understand the detail, ask “so what?”, and connect the answers.

The tools make it easy to produce long reports that nobody has time to read. The harder and more useful thing is to send each person only the two things they need to know today, and perhaps a question a colleague has asked that they could answer. The value is not in the two things you are told. It is in the hundred things you are not told.

The value is not in the two things you are told. It is in the hundred things you are not told.

One case made this concrete. A bank’s communications team wanted a shorter version of its daily press report to share with colleagues working on mergers and acquisitions. The first instinct was to condense the existing report. The better question was what those colleagues needed to know, and the answer was not the coverage, which they could read for themselves, but the main investors’ public positions and how they had changed. The new version drew on the same sources and answered a different question, at a third of the length.

A related rule is that agents should complete the work they identify. If a report concludes that answers to frequently asked questions would help, the agent should draft them and attach them, not add a line to someone’s to-do list. People are already stretched; a tool that tells them about more things to react to is a tool they will stop reading.

The inputs that usually go missing

Once you know which decision you are serving, the question is where the relevant knowledge lives. Much of it is micro knowledge: small facts, judgements and preferences embedded in informal discussion, in conversations, Slack threads and emails, rather than in anything a company would file. It has always been there. What has changed is that it no longer has to be written up to be used. Conversations can be recorded, the threads and emails already exist, and a language model can read all of it. In the Activate office, the question “are you recording this?” now arrives within five seconds of two people starting to talk, because it is faster than writing it up afterwards. You can ask what happened in the company in the last hour and get an answer, without anyone stopping to write a report.

Storing the documents is only the beginning. The useful unit is smaller: a claim, a decision, a stated preference. Call it an insight. Grouped by subject and connected to the people who expressed them and the clients and projects they concern, insights let you see where a team agrees and where questions remain open, and let you ask who knows about a topic and trace the answer back to what that person said. The result is a layer above the source material: a model of the ideas within it, which can be assembled in new ways. This is also how memory works. We rarely remember whole documents. We remember ideas, recall them when needed, and combine them with others to make something new.

The second missing input is the reasoning that was discarded. When you use an AI tool to write a proposal, the finished document is one path through a tree of choices. It shows what you included, but not the options you rejected or why. Think of the moment in a working session when someone says “no, that’s a leap of logic”. That exchange is usually lost, and it may be as useful to the company as what went in. Keeping it is still research, and some of it will not work.

Brought together, these two sources produce something a company cannot buy or download: insights that are specific to it.

Why this was not possible before

We had both tried a quarter of a century ago. In 2001 Paolo built KCollector, a knowledge management tool based on blogging. It organised posts by who, where, what and when, so people could find the same information in different ways. The BBC rang on the day of the announcement and became one of two customers. At about the same time, John’s employer was trying to build a system that would find the right person to talk to on any subject, anywhere in the world, drawing on people’s emails and their contributions to Lotus databases. That did not get far either.

KCollector depended on people writing about their work. Its creators were bloggers who enjoyed doing that; the people they hoped would use it generally did not. John’s project faced two further obstacles: there was not enough computing power to process everyone’s correspondence, and people were understandably concerned about privacy. The difference today is that nobody has to contribute. The micro knowledge is captured as a by-product of work.

Timing is part of better

Most organisations now receive far more data than they can process, and the gap is widening. One useful measure is reaction time: how long passes between something happening and the right person acting on it. Sometimes a shorter response time makes the difference between acting and missing the opportunity altogether. Today is good; tomorrow is yesterday.

Today is good, tomorrow is yesterday.

A fact is more useful when you understand what it means and what you could do next. A warning that you are approaching the vehicle ahead is useful; a car that slows down before the collision is more useful still. A business system can at least bring an important change to someone’s attention while there is time to act.

History is part of timing too. The first version of the daily press agent read only that day’s news, and every morning it woke up astonished that something dramatic had happened again. Giving it a memory that weighs recent statements more heavily while keeping older ones lets a reader see how a position has changed and judge its direction, much as a weather forecast reasons from how the pressure systems have been moving. Prediction markets may add a view of what is expected ahead; whether that helps a particular business is an open question, and coverage is far better for American questions than European ones.

Decisions happen everywhere

Decision-making is not confined to the boardroom. Someone pitching a client decides what to say and what to leave out, continuously. Someone writing a document does the same. A useful test: think of a colleague whose departure would leave a gap. What choices does that person make, and what could others learn from their judgement?

In 2001 or 2002 Paolo co-wrote a paper describing effort in an organisation as vectors. Each person’s work has a size and a direction. Add them tip to tail and the resulting line is what the organisation achieves. People can work hard in opposing directions and cancel out one another’s efforts; when their work supports a shared purpose, the organisation moves faster. For that, information has to travel both ways, from leadership to the people doing the work and back again, and the time between learning something and responding to it has to shrink, without waiting for the next meeting. Where information is missing, people fill the gap with judgement, and their judgements differ. That is where misalignment begins.

Much of how a company works is understood without being written down. Describing those practices for an AI agent turns out to help people understand them too: a company may share a language without sharing an understanding of what lies behind it. Making the knowledge explicit gives everyone something they can use, question and improve.

The practical starting point

Identify the three or four decisions that matter. Agree what better would mean for each. Then look for the evidence in the places it actually lives, including the conversations nobody wrote up and the options nobody chose, and give people only what they need to act. The hard part is not the technology. It is being disciplined about everything you choose not to say.

The hard part is not the technology. It is being disciplined about everything you choose not to say.