“Delusion” and “Deception”

An Oxford University study of 245 major dams has recently been published. It reviews more than you will ever want to know about dam cost and schedule overruns … but here’s the full report if you’re a glutton for punishment.

The average cost of dams studied was US$700 million; the average dam height was 77 metres; 59 of the dams studied were for irrigation or flood control.

The bottom line:

• Three out of every four large dams suffered a cost overrun.
• Actual costs were on average 96% higher than estimated costs; the median was 27% higher.

The report finds that the “evidence is overwhelming that costs are systematically biased towards underestimation”.

The report notes that large dams built in North America had considerably lower cost overruns (11%) than large dams built elsewhere (104%). But meeting that standard, even an 11% overrun on the Ruataniwha dam would cost another $30 million.

And regarding schedules, the study says that eight out of every ten large dams suffered a schedule overrun. Actual implementation schedules were on average 44% (or 2.3 years) higher than the estimate, with a median of 27% (or 1.7 years). Again, the study says, the U.S. has the best track record, with a 27% average schedule overrun. Such an overrun for the Ruataniwha project would effectively add a year to the availability of water (and revenue therefrom).

What does a sensitivity analysis say regarding how construction delays might affect scheme revenue losses? I can’t say … I haven’t seen any such sensitivity analysis.

What causes dam overruns in actual experience? Here’s what the study concludes:

“… the evidence presented here is consistent with previous findings that point to twin problems that cause adverse outcomes in the planning and construction of large and complex facilities such as large hydropower dams: (1) biases inherent in human judgment (delusion) and (2) misaligned principal-agent relationships or political incentives (deception) that underlie systematic forecasting errors.”

Delusion and deception … isn’t that encouraging?!

The report mentions what students of decision-making call the ‘planning fallacy’. The guru of such matters is Nobel Prize winner Daniel Kahneman, who wrote the acclaimed Thinking, Fast and Slow, a New York Times book of the year and a masterful study of flawed human and organisational decision-making.

Kahneman coined the term ‘planning fallacy’ to describe plans and forecasts that “are unrealistically close to the best-case scenarios” and “could be improved by consulting the statistics of similar cases”.

He observes:

“The authors of unrealistic plans are often driven by the desire to get the plan approved – whether by their superiors or by a client – supported by the knowledge that projects are rarely abandoned unfinished merely because of overruns in costs or completion times. In such cases, the greatest responsibility for avoiding the planning fallacy lies with the decision makers who approve the plan. If they do not recognize the need for an outside view, they commit a planning fallacy.”

To avoid such forecasting errors on such large construction projects, the authors advise using what are called ‘outside view’ techniques. Most simply, this involves closely examining the actual outcomes of already concluded ‘reference class’ projects and testing the alignment of the new project’s assumptions against these.

Unfortunately, the planning process for the CHB dam has been anything but ‘outside view’ … it’s been insider-driven from the outset. But what we are promised is that a ‘fixed-cost, fixed-time’ contract with the project’s contractor will totally protect ratepayers (and other investors) from any cost overruns. Presumably, the contractor is figuring the scheme will actually cost less than $280 million, and has built in some sort of contingency cushion, plus of course its profit margin.

As a councillor, I can’t reassure myself – let alone you – that the risk of overruns is totally contained … I haven’t seen the contract.

Our outside reviewer, Deloitte, in their cameo appearance, told us verbally not to worry about cost overruns. But when asked if they had probed the underlying construction assumptions, their answer was no … they simply trusted that the estimation work was done by credible experts. We’ve yet to see a written report from Deloitte (it was promised for 30 April) … but in their defence, the dam’s finances in their totality are a constantly moving target!

If we do proceed with the dam, and manage to build it without overruns, the Ruataniwha project will join elite company … becoming one of the one-in-four dam projects that suffers no cost overrun and one of the two-in-ten that suffers no schedule overrun. Probably the first ‘public works’ project to achieve those distinctions in Hawke’s Bay history.

Who’s in a betting mood?

Tom Belford