Topolog
Browse all articles

The planning fallacy is a systems problem, not a personal failing

Forecasting

In 1979 Kahneman and Tversky gave a name to something every project veteran already knew: the planning fallacy. People asked to predict how long their own work will take produce estimates clustered around the best case, even when they have personally watched dozens of similar efforts overrun. The bias survives experience, survives incentives, and, most uncomfortably, survives knowing about the bias. Students who had just been taught the planning fallacy underestimated their own assignments anyway.

Half a century later, most planning software is still built as if the fix were personal discipline: estimate harder, review more, add a buffer. The research suggests a different conclusion. If a bias is systematic, universal, and resistant to awareness, then it is not a user error. It is a property of the input device. And properties of the input device get handled in the system.

What "systematic" buys you#

Here is the optimistic half of the research: the planning fallacy is not noise, it is bias, and bias is the easy half of the problem.

A noisy estimator is wrong unpredictably. A biased estimator is wrong predictably, and predictable error can be measured and corrected. If your last twenty writing tasks each took about 1.4 times your estimate, the correction for the twenty-first is not a mystery. The reason humans do not apply it themselves is well documented (we take the "inside view" of each new task, treating it as unique, when the "outside view" of the reference class is the better predictor). But software has no inside view. It can hold the reference class with perfect patience.

So the design follows directly from the literature:

  1. Separate bias from variance. An estimate has a systematic component (you say 10 hours, it is usually 14) and a random component (sometimes 12, sometimes 17). These need different machinery. Conflating them, which is what "just pad everything by 50%" does, corrects neither: padding moves the centre while hiding the spread.

  2. Correct bias with evidence, not vibes. Topolog applies a Bayesian multiplier layer per area of work. As real completion times accumulate, the multiplier between your estimates and your actuals converges, and the forecast quietly applies it. The correction engages once there is enough evidence to be meaningful (eight or more observations in a category) rather than overreacting to one bad week. You keep estimating the way you naturally do; the system learns your personal exchange rate.

  3. Express variance as variance. The residual randomness, what is left after the bias correction, is carried by each estimate's coefficient of variation and propagated by Monte Carlo simulation (the mechanics are in the ranges article). The output is honest about the spread instead of hiding it inside a padded point.

The combination means the question "how long will this take?" is answered by three cooperating sources: your estimate (centre), the research-calibrated and self-correcting multiplier (bias), and the sampled distribution (variance). No single one of them has to be right alone.

The optimism your tool adds on top#

There is a second, less discussed layer of the problem: most planning tools are themselves optimistic, structurally, in ways that compound the human bias.

A tool that schedules two of your tasks in parallel because the dependency graph allows it has silently assumed you are two people. A tool that shows a single finish date has silently picked a quantile and will not tell you which (it is usually close to the median, which you will miss half the time). A tool that lets a plan end in a checkbox has assumed the risky review at the end always passes. Each assumption is small; a plan is made of hundreds of them; the errors all point the same direction.

This is why "honest tooling" is a design property, not a tone of voice. A forecast pipeline has to refuse the optimistic default at every layer: serialise one human's work because one human is a serial resource, branch the future at every gate, report quantiles rather than a date, and let the plan end in failure states with real probability attached. (What that output looks like is the completion spectrum.)

"Will it not just learn that I sandbag?"#

Yes, and that is the system working. The multiplier is symmetric: chronic underestimators get scaled up, chronic sandbaggers get scaled down, and the well calibrated converge to a multiplier of one. The goal is not to moralise about estimation style. It is to make the forecast invariant to it. Two people with opposite biases, both using the system honestly, end up with comparably accurate forecasts, which is something no amount of estimation training has ever achieved.

A reasonable worry in the other direction: does the correction excuse never improving your estimates? In practice the opposite happens. The system shows you plan-versus-actual explicitly, per area, which is exactly the outside view the research says humans fail to take. Most people's raw estimates improve once the exchange rate is visible. The multiplier just stops the project paying for the learning curve.

What to take from this even without the tool#

The pattern generalises, and you can run a manual version of it today:

The fallacy's mechanismThe systemic counter
Inside view: "this task is different"Keep a reference class: your last N similar tasks and their actual-to-estimate ratios
Best-case anchoringEstimate the typical case, then write the spread down separately
Bias hidden inside paddingTrack the multiplier explicitly; never bake it into the estimate
Single-date overconfidenceCommit at a stated quantile, not at the median

The planning fallacy was never going to be fixed by trying harder, and the research said so from the start. It is fixed the way engineering fixes any systematic error in a sensor: measure the offset, model it, and correct for it downstream, continuously, without blame. Your estimates are a perfectly good sensor. They just need a calibration layer, and that is a job for the system, not for your willpower. (If you are evaluating tools for this property, here is what to ask for.)

Ready to plan in graphs?

7-day free trial · 250 credits · No card required

Get Started →