Why software projects fail, and why the average is the wrong number

Ask why software projects fail and you will get a list: unclear requirements, scope creep, bad estimates, the wrong team. All of those are real and all of them are symptoms. The more useful question is why the failures are so much worse than anyone plans for, because that has an answer with actual evidence behind it.

The distribution is the finding

The most important piece of research here is not a survey of causes. It is a study of shape. In The Empirical Reality of IT Project Cost Overruns: Discovering A Power-Law Distribution, Bent Flyvbjerg, Alexander Budzier, Jong Seok Lee, Mark Keil, Daniel Lunn and Dirk W. Bester examined 5,392 IT projects and found that cost overruns do not follow a normal distribution. They follow a power law: a large number of projects with relatively small overruns, and a fat tail containing a smaller number with extreme ones.

The authors state the consequence plainly. If managers assume a normal or near-normal distribution, as is common, they may be "unwittingly exposing their organizations to extreme risk by severely underestimating the probability of large cost overruns".

That is worth sitting with, because it invalidates the way most contingency is calculated. Adding twenty percent to an estimate is a defence against a bell curve. It is not a defence against a distribution whose whole character is that the bad outcomes are much further out than the average suggests, and more frequent than intuition allows.

What that changes about planning

If the risk lives in the tail, the goal of planning is not to predict the average more accurately. It is to cap the tail. Those are different activities, and only the second one actually protects you.

• Shorten the commitment, not the estimate. A project that has to be right for three months has a much shorter tail than one that has to be right for two years.
• Make the expensive unknown the first thing you touch. Tail events come from the part nobody understood, so schedule it first rather than last.
• Keep a real kill point. A stage gate that has never stopped anything is decoration, and it is the main mechanism that converts a bad project into a catastrophic one.
• Prefer reversible decisions. The cost of being wrong matters more than the chance of being wrong when the downside is unbounded.

The failure modes underneath

The distribution explains the severity. It does not explain the mechanism, and in practice a small number of mechanisms account for most of what goes wrong.

Two of these deserve a note on how they hide. Integration risk hides because the other system is usually owned by someone outside the project, so the cost of finding out it does not work the way the documentation says is deferred until a point where the schedule has no slack left. Sponsor loss hides because nothing visibly breaks on the day it happens. The project keeps moving, decisions just start queueing, and by the time the delay is obvious it has been compounding for a month.

The last row is the one that quietly produces the tail. A project reported as eighty percent complete for months is not a project that is nearly finished. It is a project whose remaining work was never sized, and the honest treatment of that is in why software projects take so long, and what actually fixes it, which deals with duration where this note deals with failure.

Requirements are where most of it is decided

Two of the five modes above are requirements problems, and they are the cheapest to fix because they are fixed with writing rather than engineering. A short, specific requirements document is not bureaucracy at this scale. It is the artefact that makes disagreement visible before it is expensive. The short version of how to write one is in how to write a software requirements document, and the scoping step that precedes it is in how to scope a software project before anyone writes code.

If you take one thing from this note into the next project, make it the acceptance criteria. Not a specification, not a wireframe. A list of statements that will be either true or false on delivery day, agreed by the person paying.

Accountability, and the thing that is not a process problem

There is a management failure that no methodology catches, which is holding a team to a method and an outcome at the same time. If you prescribe how the work is done, you own the result. If you want the team to own the result, you have to let go of the method. Khaled Badr makes this argument well in hold people to results or to method, not both, and it is the single most common governance mistake on projects that fail slowly.

The related version inside the build is estimating by optimism and then managing by pressure. Pressure does not change how long the integration takes. It changes what gets reported, which removes the early warning that the tail is arriving.

What this does not tell you

The power law finding is about cost overruns in IT projects generally, drawn from a large sample of mostly substantial projects. It is strong evidence about the shape of the risk and weak evidence about your particular two month build. A small, well-scoped project with one integration is not the same animal, and treating it with enterprise governance is its own kind of waste.

What transfers regardless of size is the asymmetry. The upside of a software project is capped at what you asked for. The downside is not capped at all. Plan the parts that are cheap to be wrong about quickly, and spend your care on the ones that are not.

And if the project has not started, the highest-leverage decision available is still whether to build at all. That is the subject of custom software vs off the shelf, and a project never started has no tail.