A couple of weeks ago I spoke with Marion Terrill of the Grattan Institute regarding her new report on The Rise of Megaprojects (check out Megaprojects chat with Marion Terrill from Grattan). Marion made so many excellent points in the discussion that I thought it would be worthwhile publishing the transcript (generated using the amazing AI tool otter.ai, and which required minimal editing by humans).

If you’re based in Queensland, you may be especially interested in the part of the conversation in which Building Queensland is mentioned (from 29:10). BQ is the state government’s infrastructure advisory body which is at risk of losing any independence it possessed following its absorption into Treasury. The transcript of our full conversation is below the fold.

Gene – 00:00
Welcome to the Economics Explored podcast. I’m your host, Gene Tunny. This is Episode 62 on megaprojects. My guest today is Marion Terrill, who is the Transport and Cities program director at the Grattan Institute, one of Australia’s leading think tanks. Marion, good to have you back on the program.

Marion- 00:25
How are you, Gene?

Gene – 00:30
I’m going well, thanks. Marion. You’ve recently co-authored a report on these so called megaprojects, by which you mean projects which cost billions of dollars, and, in the overview for your report, you write: megaprojects are already breaking records for cost overruns. There’s an overrun so far of $24 billion on just six current projects. Inland Rail was costed at $4.4 billion in 2010. It’s now estimated to cost $9.9 billion. Melbourne’s Northeast Link was costed at $6 billion in 2008. It’s now expected to cost $15.8 billion even though the Victorian Government selected the cheapest route. The Sydney Metro City and Southwest was costed at $11 billion in 2015. This year, the New South Wales Government announced the latest estimate was $15.5 billion. So, these are some extraordinary cost overruns, Marion. Did the size of the cost overruns, surprise you? Or are these projects prone to this type of overrun?

Marion – 01:45
It did surprise me, Gene. So, what we did here is we looked at all of the projects, all the transport projects done by Australian Governments over 19 years from 2001 to the first quarter of 2020. And if you look at that suite of projects, we found that we’d spent $34 billion more than we were first told that we would spend. But once you fast forward to the projects that are still under construction today, then you realise that something has changed, because just for six megaprojects, we already know about an extra $24 billion, and they’re not finished yet.

Gene – 02:33
Okay. Now as a percentage of the costs, are we talking about 20% extra? 50% extra? 100% extra? Because I know that for a notorious megaproject in the US – I don’t know if you’ve heard of this one, the Big Dig in Boston, which was the central artery tunnel project – and it was initially costed, this is in today’s dollars, at $7.4 billion, and it ended up costing nearly $22 billion. So, it was almost three times as expensive. In Australia, so far, what’s the general magnitude of the cost overrun in proportional terms?

Marion – 03:21
The 34 billion is about 21% of the initially announced costs. What’s going on here? In recent years there’s been an absolute explosion in megaprojects. So normally, when we talk about megaprojects, we’re talking about a billion dollars or more. So, they’re big, and we’ve been doing them for 20, 30 years, something like that. And they’re important infrastructure. They often have quite a transformative effect on a part of the city. Something like an entirely new train line or new highway through urban area. But 20 years ago, these were relatively rare, and we had never had a $5 billion project. We got our first $5 billion project ten years ago. Now, we’ve got nine of them. And the reason that this is important is because one of the predictors of existing costs, or costs exceeding what’s initially announced, is size. So, the more you have big projects, the more you can expect for those projects to have cost overruns.

Gene – 04:39
Okay, so your report is titled, The rise of megaprojects: counting the costs. So, you’re saying that this is a relatively new phenomenon in Australia, whereas, in other countries, they’ve probably had more experience of megaprojects in the past. There was the Channel Tunnel famously between England and France. I think that went massively over budget and then didn’t earn the revenue it needed so the company that constructed it and was running it ended up collapsing. So, we’ve seen these types of things in other countries and there’s a large body of evidence, isn’t there, internationally, that suggests these megaprojects are problematic? Is that correct?

Marion – 05:20
Yes, that’s right. And we see it across many sectors as well. So, I’ve looked at transport infrastructure done for governments, but people say to me “Look, in the oil and gas industry, it’s not that different”. And I think industrial megaprojects will be another example. Megaprojects are very complex. So, the heart of this is that you wouldn’t expect to be able to perfectly estimate the cost for something very, very complex. And I think that’s a reasonable argument that people make. As the cost estimates get refined over time, as the scope gets nailed down tightly, and you know more clearly what you’re dealing with and what the underground conditions are like, you do get a more precise idea of what the cost is going to be. But on the other hand, I would say, well, costs underruns are much rarer than cost overruns. So, we still do have a phenomenon where costs invariably do tend to go higher. And the figures I’ve cited to you are including underruns, so they’re net overrun cost. So, you do get underruns, but they are much rarer and they’re not as big.

Gene – 06:57
Okay. Do we know what it is that’s driving up the costs? Are there any common reasons for the cost blowing out?

Marion – 07:11
Yes, so, apart from size, the other issue that we looked at was premature announcements. So only a third of projects do have a premature cost announcement. And what I mean by that is that a government or an opposition announces that it’s going to build a particular project for a particular cost. But the project doesn’t yet have the regulatory and financial approvals that are needed for it to actually proceed. So an example would be, in the lead up to an election, both major parties will say “oh, if you elect us, we’re going to build this and we’re going to build that, and this is what it’s going to cost”. So those sorts of things. So that only happens about a third of the time. But these sort of projects do account for four-fifths of the quantity of extra costs. What’s going on here is governments, or oppositions, say they’ll do this thing and that it’s going to be a billion dollars, or whatever. And then they go off and do the work. That would be fine if we then had a robust process for cancelling the projects that after closer scrutiny look like a bad idea. But we don’t have that process. Most projects ever announced do go on to built.

Gene – 08:32
Yes. So a government might promise to deliver, say, a new subway system for Brisbane, which we’re building at the moment, the Cross River Rail system, and they might say, well, it’s going to be $5 billion. But that was a preliminary estimate. They didn’t really do all the analysis that they should have, or that they could have. Isn’t one of the problems that because a lot of these big megaprojects involve digging, and you’ve got to get these big tunnel drilling or boring machines at work, and you may not know exactly all of the specific soil types or the rock types. I don’t know, is that a problem? Is it because once you actually get into constructing these projects, you’re not sure of all the contingencies, all the things that could happen? Is that the problem?

Marion – 10:00
Yes, that’s right. They are very complex and as soon as you start tunnelling, you bring in a level of complexity. And we’re doing way more tunnelling now than we used to. So that is part of what is going on. I think a lot of industry people have also said to me that what causes all the problems is conditions underground. So, whether that is utilities, or whether it’s soil contamination, or whether it’s simply as you say, what’s the soil like – those kind of risks. The underground risks are a very big part of the known unknowns. So, it is difficult to know what’s going to be there. So, once you are tunnelling, you do enter into a world where there’s a lot more of that type of uncertainty involved. What that’s driving, really, is more costs rather than necessarily cost overruns.

I think the cost overrun comes about more because the project’s complexity means if one part of it gets delayed that can ripple through the other parts. So, sort of aggregating these things up into great big projects, or not disaggregating them, is part of the problem. And some governments are actively working to try to unbundle some of these big packages of works to try to have more discreet bundles. Partly, they’re trying to help small firms in industry, too, but partly it’s dealing with this problem of size being associated cost overrun risk.

Gene – 11:30
Okay. When you mentioned the problems you run into when you’re tunnelling, I just remembered that story about Crossrail in London. Have you heard that story? I think they’ve found old burial pits for people who died during the plague in the 17th century.

Marion – 11:45
Yes, that’s right. Actually, the archaeological stuff is a phenomenon here, too. I think they found a burial ground or something like that at Central Station in Sydney. I think this is the thing: underground it’s harder to know what’s there.

Gene – 12:01
Okay. I’m going to ask you now about this idea of optimism bias. So, is the problem that we’re just too optimistic about our ability to keep things on time and on budget, our ability to minimise costs? And we’re also too optimistic about the benefits that we’ll see, the number of people who will use the infrastructure, who will drive on the toll road, or who will catch the train? Is that it? Is that a problem? Is there a bias that all humans have?

Marion – 12:45
Yes, there’s a couple of different stages at which optimism bias can kick in. So, when a politician or an official makes the first cost announcement or cost promise, they probably don’t know all the possible complexities, but the margin that they might build in for that is generally nothing like enough. We see that time and time again, that the initial cost announcement is sort of assuming everything’s perfectly simple and goes perfectly smoothly.

And the cost estimators are also very hamstrung, I think, in their ability to do good cost estimates, because we don’t collect data on finished projects in any useful format. And what that means is, for example, in the report, one thing we’ve looked at is business cases that are in the public domain; we looked at the P-50, or the median cost estimate as compared to the P-90, or the worst case cost estimate, if you like. And the typical difference is that a P-90 estimate is about 7% higher than a P-50 estimate. So that is not much more money between what you think is going to happen in the worst-case option. And when we looked at the data for 19 years, what we found is that [P-90] should be more like 49% higher. That’s massively higher. So, you can see the lack of data supports a process of cost estimation that makes insufficient provision for what history has told us is quite likely to happen.

And then there’s a final stage to this, which is when this goes to market. A lot of companies have been quite vocal this year and last year about them not making money. When I say to people, well, why is your company bidding for these jobs when you can’t make money? The feeling that comes out is essentially optimism bias. They want the job and they think, oh, we’ll find a way. They’re kind of inclined to see the advantages and to minimise all the things that could go wrong. And that’s perhaps what it takes to get the job. But it’s given rise to a lot of angst in industry.

Gene- 15:07
Yes. This point you make about the difference between the P-90 and the P-50 values, I think is incredibly important. Could I just confirm what P-90 and P-50 are? So, P stands for probability? So, we’re talking about a probability value, is that correct?

Marion – 15:30
That’s right. So, one of the ways that cost estimation occurs is to do probability pricing – the range of cost outcomes, you can see it as a distribution if you like, and the P-50 is the medium value. So [the P-50] is the value for which half the outcomes will be above and half below. And P-90 is when 90 times out of 100 you’ll be within the budget. So, what the P-50 and the P-90 are doing is essentially taking two points from a distribution of outcomes that are based on an idea of what that underlying distribution is like. Sometimes people are inclined to think it’s a normal distribution. In other words, underruns and overruns are equally likely. But that’s not true at all. We know cost overruns are much more likely and bigger. We think those worst-case-type cost outcomes can be very bad, as in very high cost, but that’s just not reflected in the current practice in developing business cases.

Gene – 17:15
Absolutely, I agree. I’ve just got two more questions. The first will be about whether it’s just optimism bias, and it sounds like, you know, that’s part of the story, but it can’t be the whole story because there is all of this complexity with projects. I’ve been reading Nassim Nicholas Taleb’s book, Antifragile. I know you’ve got a copy of it Marion, and he talks about optimism bias and the planning fallacy. He notes however:

“The puzzle was that such underestimation did not seem to exist in the past century or so, though we were dealing with the very same humans endowed with the same biases. Many large-scale projects a century and a half ago were completed on time. Many of the tall buildings and monuments we see today are not just more elegant than modernistic structures but were completed within and often ahead of schedule. These include not just the Empire State Building, but the London Crystal Palace erected for the Great Exhibition of 1851, the hallmark of the Victorian reign based on the inventive ideas of a gardener.”

And he goes on to argue that it can’t just be optimism bias. It has got to do with something about the modern economy and modern supply chains. He writes that “the obvious is usually missed here. The Crystal Palace project did not use computers and the parts were built not far from the source with a small number of businesses involved in the supply chain. Further, there were no business schools at the time to teach something called Project Management and increase overconfidence. There were no consulting firms. The agency problem, which we defined as the divergence between the interest of the agent and that of his client was not significant. In other words, it was a much more linear economy, less complex than today. And we have more nonlinearities, asymmetries, and convexities in today’s world.” What do you think of Taleb’s view, Marion? Do you have any thoughts on that?

Marion – 19:31
It’s very interesting. I found that very thought provoking, too. There is a lot of talk, more in the US than in Australia, about the contribution to costs that comes about from regulation. Opinions differ as to how much of this is good and necessary and important regulation, like safety on work sites and some environmental regulation, but some of it is also more contested: for example, the ability of local citizens to cause delays on a project and to effect changes to the project. I’m thinking of things like noise barriers, or a different route that avoids particular, sensitive things. So, all of these have costs and benefits, and I think it would be fair to say that one difference between today, and the times that Taleb is talking about is that we have a lot more of that regulation, for better or worse. And so there are a lot more touch points along the way, rather than just sort of steering a path through. So that’d be one thing I’d say.

I think the other thing that Taleb really contributes on is this idea of this fat tail distribution, which goes back to what we’re saying about T-50 and T-90 costs. When you think about cost overruns, it’s not really a question of a lot of little overruns adding up to a big number. What it really is, is a reasonably small fraction of projects overrunning dramatically. That’s kind of much more what’s going on. So I think the way he talks about these fat tail distributions is really instructive, because you’re not really trying to kind of tighten the screws on every little project. It’s really just to see the warning signs, and to be very careful about some projects, or particularly managing the risk associated with some of these projects.

Gene – 21:56
Okay, so I’ll get on to our final question, but I better ask first. So, when we’re talking about megaprojects, we’re typically talking about projects that are funded by governments, because they’re major bits of infrastructure, and they need some government contribution to ensure that they’re financially viable. Is that correct?

Marion- 21:56
That’s right. We’re looking at projects done for Australian Governments. And our interest really is what were the implications for the taxpayer. So, it is true that you do get private providers, for example, a private coal railway or something like that. But if that runs over there is no implication for the taxpayer and so it’s not in the scope of what we looked at.

Gene – 22:50
Yes. So, it’s up to them to have done the proper due diligence. And I know from working with companies that have been building major infrastructure projects, they do a lot of work on the number crunching and making sure that these things stack up.

You mentioned before with governments there’s often the tendency to make the announcement before doing the number crunching. So, there’s a problem with the process. You make some recommendations regarding how we improve the process around megaprojects. And there’s a view that if you do that, you’ll end up with better outcomes in the long run in terms of minimising costs and maximising benefits, and not fooling yourself into thinking that something is a good project. Would you be able to take us through what those recommendations are, please, Marion?

Marion- 22:40
Yes, sure. So, I thought about them in two different categories. Firstly, given that we’ve got a record amount of construction underfoot right now, such that in the March quarter the value of these transport projects under construction for Australian Government’s reached $125 billion worth, we think there’s an imperative to have things to do now, and then there are some longer term structural things.

So, on some of the things to do now, I think that the Auditor General in each state should do an immediate stocktake of the large projects currently underway and see what sort of condition they are in.

Secondly, sort of reflecting on why it has been so difficult to get information about a lot of these projects, there’s not a huge amount of information in the public domain, we’re recommending a continuous disclosure regime, like you get with the listed companies with the stock exchange. In other words, whenever there’s a material change to the expected cost, benefits or completion date for big projects, that should be reported promptly to Parliament.

Thirdly, I’d like to see that when governments, or oppositions for that matter, announce a cost that they are clear about how much or little work has gone into that project just so that it’s more transparent, I suppose, with the public, but also that they should then reconcile that, as the cost becomes more clear, with the previous one.

And, finally, that there should be an independent assessment of the quality of infrastructure business cases. So, we found that actually only a quarter of large projects, so half a billion dollar plus projects, actually do have a business case published or assessed by one of the I-bodies (infrastructure bodies). At the time of announcement, three quarters don’t. A lot of them never get one. So just to do business cases, obviously, but also to attend to the quality of those business cases with some degree of independent scrutiny. That’s what to do today.

In terms of how to not end up in this place in the future, I’ve got four recommendations.

Firstly, that megaprojects should be the last resort, not the first resort. The first resort should be to make the best out of the infrastructure we’ve already got. That means maintenance, and that means time of day pricing and other demand management tools, and a lot of the more moderate sized projects actually have a much better bang for the buck for mature cities that we have in Australia. That’s really where governments should be reaching, rather than sort of endlessly thinking that megaprojects are the answer whatever the question.

Secondly, as I’ve mentioned, [a recommendation] is to collect the data on completed projects. The lack of reliable data on past projects makes it very difficult, if not impossible, for cost estimators to do things like reference class forecasting, comparing this project, not just on a bottom up basis, but looking at it on a top down basis, without projects that were similar to see how they turned out.

Thirdly, to improve the consistency of cost estimation handbooks. There’s more than 50 of these handbooks in this country and they’re essentially recommending the same basic toolkit but in a very wide variety of ways, not particularly consistent, and [it’s] quite difficult to know the seniority of different documents and the requirements. We do sometimes see jurisdictions, distinct states, disagreeing with the Commonwealth and therefore issuing two versions of a benefit-cost ratio and that sort of thing. So, it is really a bit of a mess. We can just [have] model guidelines from the Commonwealth that the states can adopt and adapt if they need to. But it does seem like it’s unnecessarily complex to no great end.

And, finally, everybody’s favourite is post completion reviews. Everybody thinks we should do these and they almost never happen. So, I feel that we’re destined to keep making the mistakes, same mistakes, if we do not go back and scrutinise what happened and just learn the good and the bad. To get more and to share that across jurisdictions, because, again, the states are not that different to each other. It would be particularly beneficial for smaller states who just don’t have as many projects coming through. Evaluation is always instructive.

Gene – 29:10
Yes, yes. But what happens is governments just move on to the next megaproject, to the next announcement, unfortunately, rather than assessing, frankly and fearlessly, how they’ve performed in the past. Marion, on those I-bodies, you’re talking about Infrastructure Australia, Building Queensland, is that right?

Marion – 29:44
That’s right. I noticed in Queensland, since the election, [Building Queensland is] now going into the Treasury. Every state does this a little bit differently. But, yes, I think Building Queensland is probably at one end of the continuum and Infrastructure Victoria is the other way, because they don’t get involved in the business cases directly unless asked. And the other bodies are somewhere in between.

Gene- 30.17
Right, but the general idea is they are supposed to provide some type of independent oversight of infrastructure planning and prioritisation.

Marion- 30:28
That’s right, and sort of more globally to help governments make better infrastructure decisions. So, they are well placed to do this independent scrutiny, although it wouldn’t be a problem if some other bodies do. I think it’s just that it is done.

Gene – 30:45
Yes. But it sounds like from what you’ve been saying and from your recommendations, it sounds like the Grattan view is that, to date, these bodies haven’t performed as intended, or that they haven’t been able to provide the rigorous scrutiny that was suggested at one time that they would. Is that fair to say?

Marion- 31:11
So they do all operate in different ways and I would say that the bodies that have been around a little bit longer, not surprisingly, mature into that role. So, there is a bedding down for any kind of new body. But then they’re not particularly independent. They have boards that are generally appointed by the Minister, and often will have Secretaries of departments on them. So, I think it would be a mistake to imagine them as being particularly independent. There’s different ways you can do this. And I think you can see successful models operating differently. A lot of it is just how it’s done and how committed the government is that day to listen to those bodies.

Gene – 32:20
Yes, and none of them does post implementation reviews as far as I can tell, do they? And that’s one of the things you’re recommending, so we get those lessons.

Marion- 32.27
Yes.

Gene- 32:30
Okay. The other point I’d like to make based on your recommendations, which I think are very good, I just chuckled at one of them – when you mentioned they should show the work for their estimates if they’re going to make an announcement. I was thinking, oh, that could be embarrassing for some governments if they have done the estimates on the proverbial back of the envelope or back of the fag packet. I think there have been a lot of estimates that are just based on comparing this project with another project somewhere else in the state or the nation or internationally and working out, well, they built this many kilometres of tunnels or roads, and so therefore that’s a dollar [unit] cost for building this many kilometres. It’s just something simple like that. Do you think that’s what’s going on? Could they be embarrassed by some of these estimates?

Marion – 33:15
You may be closer to it than me, Gene. There is an early estimate for a project that is within a wide range, very uncertain. But there isn’t really a problem with that if a government is then prepared to cancel some of the duds. So, for as long as we don’t get the cancellation, there needs to be much more clarity with the announcements. And if you look at the outcomes, the relationship between that initial cost announcement, and what really happens in the world, you would be led to think that there wasn’t a lot of thinking that went on for the initial estimate.

Gene- 33:57
It sounds like what we should do is take whatever figure the government gives us, say its $5 billion, and we say, well, it’s bound to be at least $7.5 billion and possibly even 10 billion. So, we should be naturally inflating those estimates. If we’re in the public domain, if we’re commenting on it, we should be generally sceptical of any initial estimates from government of the cost of megaprojects. Fair or not fair?

Marion- 34:35
Yes, I would say that’s fair. When initial cost estimates are regularly and materially too low, it does distort investment decision making. So, it does make infrastructure more attractive than other spending options that governments have got in front of them – health care, for example, education and so on. Even within the bundle of infrastructure options: if the larger projects tend to have particularly underdone cost estimates, it weighs the portfolio in favour of the larger rather than the smaller. Not to mention that it misleads the public into thinking “Oh, well we can have that marvellous road and it’s $500 million” when it’s actually not. It’s a billion.

So, yes, I think adding a premium onto these not very good cost estimates, just to me, it sort of takes them further and further away from doing good quality estimates that’s appropriate to the stage of development that the project is at.

Gene – 35:46
Very true. We should require them to do more work at the beginning. Absolutely. Marion, any final points before we wrap this up?

Marion – 35:53
Well, the only thing I would say is that governments seem to be enthusiastic about an infrastructure-led recovery from the recession, and that does include these big projects. So, I’m hoping that we don’t see too many more in coming weeks where there are a few state budgets about to be to be released in New South Wales and in Victoria. I’m hoping we don’t see too much of these because I think that these projects do take too long to get going. And, before the pandemic, we were already facing capacity constraints in engineering construction. So, I’ll be watching with interest but hoping that the weighting here is not towards these megaprojects.

Gene – 36:45
Absolutely. Because, yes, they could be bad investments. And there’s also the possibility of white elephants, too, isn’t there? So you could have something that just is an ongoing burden to the budget because it never makes any money. You could have a toll road that’s massively underutilised or some other piece of infrastructure, public transport, that you keep having to subsidise. So, yes, I fully agree with you.

Marion, thanks so much for your time. I really enjoyed that and I hope our treasuries and finance ministries and our infrastructure advisory bodies pay close attention to your excellent report on the rise of megaprojects. Thanks again for appearing on the show.

Marion- 37:32
Thanks very much, Gene.