Tag Archive for: ANAO

The ANAO major projects report: new news on the F-35

We seem to have been both naughty and nice this year. The Australian National Audit Office isn’t going to put its much-anticipated report on the future submarine program in our stocking before Christmas and we’ll have to keep our breath bated until January. That’s a shame, since the ANAO’s website says, ‘The objective of this audit is to examine the effectiveness of Defence’s administration of the Future Submarine program to date.’ That’s an $80 billion issue that many Australians would be interested in.

On the other hand, the ANAO has come through with Department of Defence Major projects report. As usual, it’s a very detailed piece of work that provides an unparalleled look at 26 of Defence’s major projects in 2018–19. Also as usual, it contains a lot of old news and some intriguing new news.

One piece of old news is that it doesn’t report on either the future submarine or future frigate program. Each of these now has an approved budget of around $6 billion, which would rank them at third and fourth in Defence’s portfolio, even before the government approves construction of a single vessel. Fortunately, we have unofficial advice from the ANAO that both will be in the next edition. Let’s keep our fingers crossed. The third plank of the domestic shipbuilding program, the offshore patrol vessel project, is included this year and is tracking well against its aggressive schedule.

Another piece of old news is that the MPR is hamstrung by terminology that seems designed to trigger the media’s worst ‘gotcha’ impulses. For example, the report lists $24.4 billion in ‘budget variations’ (again, old news) at table 3. But once we take out $14.1 billion in scope increases (i.e. the government agreeing to buy more stuff, such as 58 joint strike fighters), $1.3 billion in adjustments to scope and transfers between projects, $2.8 billion in indexation (i.e. inflation) and $4.4 billion in exchange rate variations (it really hurts when the Aussie dollar goes down against the greenback and euro), that leaves only $1.8 billion in real cost increases.

I should note that Defence presents a slightly different view of real cost increases in table 2C on page 84, but its total of $1.7 billion is broadly similar to the ANAO’s. Since the portfolio of 26 projects in the MPR is worth $64.1 billion, that’s a cost increase of about 2.6%. Moreover, the projects that have been completed and exited the MPR were collectively under budget by about $2.2 billion. So overall Defence is doing well on cost—but there’s a caveat I’ll discuss below.

Poor schedule performance is another piece of old news, and it’s not particularly good. The combined total of 691 months is bad (see table 4 on page 15). But it’s important to remember that while projects may be late, they are generally still delivering capability. The worst offender, at 112 months’ behind, is ‘Collins reliability and sustainment’, a large constellation of subprojects that’s been delivering a broad range of improvements since 2000. Many of them have been completed and the rest will be as the boats go through their docking cycles. The submarines have still been able to do their job, despite the delays.

Similarly, the second worst offender, the MRH-90 helicopter project, at 89 months, has delivered all 47 aircraft and Defence has been flying them. But not all of the individual capabilities under the project have been delivered, which is why the delay clock continues to tick. Some of these have real implications; for example, the lack of a fast roping capability means that Defence has had to keep the Blackhawk helicopter in service longer to fill the special forces role.

While we want the army’s helicopters to be able to support things like counterterrorism, this does raise the question of whether in every case it makes sense to keep a project going for years just to get the final few percent of capability. Sometimes it may be better to draw a line under it and apply the people and resources that are freed up to new priorities (autonomous systems, anyone?).

So, what’s the new news? The key worrying piece is that the F-35 project is under funding pressure. The MPR states that, ‘In September and November 2018, due to cost pressures, the Joint Strike Fighter project received government approval to transfer project scope of $1.5 billion to other phases of the Joint Strike Fighter program (none of which have been approved by government). There was no corresponding transfer of funds out of the project budget’ (page 26). Since the approved budget for the project is $16.5 billion, this means that 9% of the project scope has been moved off into some undefined point in the future, but Defence is hanging onto 100% of the budget to acquire the remaining 91%.

It would seem fair to classify this as effectively a $1.5 billion real cost increase since the capability Defence is getting is costing more. This brings the total of real cost increases to $3.3 billion. What’s more, the report states that, ‘Some level of known cost risk remains with a possibility that further scope transfers may be required’ (page 135). So more content may get pushed down the track.

The capability question is, what’s in the 9% that has been moved? And how important is it? That’s a little harder to determine. One element is the ‘beyond line of sight’ communication system (page 148). Since one of the key selling points of the JSF was its networked capability, this would seem to be a problem. Overall, there’s enough evidence in the MPR to indicate that, while the F-35 is still on track to achieve initial operational capability by the end of 2020, Defence doesn’t have a plan to get all of the elements required for final operational capability, scheduled for late 2023.

And then there’s the issue of the JSF sustainment system. I wrote earlier this year on the immaturity of the system. The MPR confirms that. Moreover, it states outright, ‘The F-35 future sustainment affordability has been affected by an increase in through-life sustainment cost estimates.’ With an F-35 flying hour already costing twice as much (page 63) as the classic Hornet it’s replacing, that’s some more bad news.

The ANAO’s major projects report: missing sub-stance

It’s a fantastic idea: given the government is spending some 36.4 billion taxpayer dollars on defence, much of which is in the 10-year, $200 billion investment program buying major weapon systems, it makes sense to have an independent, public, assessment of how these megaprojects are going.

Imagine my excitement this week when I saw that the Australian National Audit Office had released its 2017–18 major projects report with just this purpose! A 430-page feast of data and analysis on all the high-risk and high-technology systems and platforms that will make up the ADF’s ‘future force’.

Here would be a treasure trove of analysis and insight into the iconic investments this government is making in our nation’s defence capabilities.

Opening it up, I looked eagerly for assessment of progress by the Department of Defence and France’s Naval Group on the $50 billion future submarine program, to shed light beyond media speculation about progress, difficulties and medium-term cost pressures.

Running my eyes over the contents page, I looked for the listing for the $35 billion future frigate project. (I was overeager here; the winning contractor, BAE, was only announced in June 2018, at the tail end of the financial year the report covers.)

Wanting to ensure balance between the services, I also looked for LAND 400—the $5.2 billion contract won by Rheinmetall to build the army’s combat reconnaissance vehicles.

Yup. You know what’s coming: they’re not there.

Maybe I could read over the ANAO’s thoughts on the $4 billion offshore patrol vessel project, won by Lürssen in November 2017? Nope. Not there either.

I’ve criticised Defence for not adjusting its plans for capability delivery in light of the deteriorating strategic environment. With the stately two-year drumbeat for the future submarines, we won’t get the last of the 12 until around 2054—if everything goes to plan.

But the ANAO seems even more successful in closing out the real world from its consideration. It appears insistent on rigidly applying its criteria for what gets put into this major piece of analysis of Defence’s capability program, even when those criteria have the perverse effect of sidelining the areas of major cost and risk in the massive modernisation of Australia’s defence systems.

As I understand it, this is because the eligibility criteria agreed by the parliament’s Joint Committee of Public Accounts and Audit weren’t met by any of the megaprojects I’ve mentioned above. Yet surely it’s within the ANAO’s power to advise the committee on how the criteria for this report should change to keep it relevant.

The future submarine project, for example, apparently isn’t in because it hasn’t attained second-pass  approval yet. But it’s been more than two and a half years since Naval Group won the project, and I note that this year’s defence budget statement says that already some $2.243 billion has been committed to design and construction.

That makes just this part of the submarine project bigger than 16 of the 26 projects the ANAO has put into its report. And, arguably, analysis of progress and problems at the project’s definitive design and mobilisation stage could tell us a lot about its prospects and provide valuable independent advice to those charged with making it a success. Why on earth is at least this part of the country’s biggest-ever project not covered?

Reading on, I was struck by the internally focused approach of this significant report. Many of the metrics make sense only when compared with previous versions and have little impact beyond it. For example, total schedule slippage moved from 708 months (26%) in the 2015–16 report to 793 months (29%) in 2016–17 and now it’s 801 months (32%) in 2017–18. Does that mean schedule performance by defence project managers is getting worse?

Not really. It just means that the changing basket of projects the ANAO has in the report causes the total delay to shift about. Looking a little deeper, over a quarter of this total project delay is from two Collins-class submarine projects that are pretty much done deals (replacement combat system, and reliability and sustainability). They are historical artefacts now, have been subject to relentless scrutiny, and are now part of a good-news story about Collins availability and capability—but without that context, the report paints a picture of poor project performance.

Then there’s the cost increase story. The headline that seems to jump out of the report is that the 26 projects it covers cost $59.4 billion all up. But $23.0 billion (38.7%, as the ANAO reports) of that is in what look like cost blowouts—budget variations since initial second-pass approval by government.

Before you hit the alarm, though, all is not as it seems. The ANAO’s internal metrics that define a budget variation make a non-problem seem to be a major scandal.

Reading into the detail, $14.1 billion of the cost increases are because the government bought more platforms across several projects—58 more F-35 joint strike fighters, 34 extra MRH-90 helicopters and 4 more P-8 maritime patrol aircraft. Another $6 billion was a result of contractually agreed price indexation or exchange-rate variations. Only $1.8 billion of the $23.0 billion is actually because of real cost increases in the projects. So, no headline, just a metrics problem.

No doubt there are hidden pearls, and I welcome 430 pages of publicly disclosed data—particularly because this is becoming rarer at a time of tightened public-information and media management.

Overall, though, I was underwhelmed by the ANAO’s analysis because of both its opaqueness and its superficiality.

For example, a major insight is that, ‘prima facie, … the more developmental in nature a project is, the more likely it will result in a greater degree of project slippage, as well as demonstrating one of the advantages of selecting MOTS [military off-the-shelf] acquisitions.’ And apparently ‘longitudinal analysis indicates’ that ‘slippage’ (delay, to most humans) ‘primarily reflects the underestimation of both the scope and complexity of work’.

So, let me end this disappointed set of thoughts on a Christmas note—channeling the Book of Common Prayer at this time of reflection and festivity. Looking through this voluminous collection of project data assembled at considerable public expense, let us all join together in saying to whatever deity we each call upon:

Lord, forgive the ANAO both
for what it has done
and for what it has failed to do.

May this season of reflection be one
in which the good people of the ANAO
recommit to the actual purpose
of this major projects report.

And may 2019 be the year that they
turn their minds to assess the programs and projects
that represent most of the money and risk
in Defence’s investment program.

In so doing, may they enhance
government decision-making,
defence management
and public understanding
about the live risks and challenges
in the $200 billion
forward investment program.

Naval shipbuilding in Australia: the case for a regulatory overhaul

clipboard 

The government’s decision to pursue multi-decade domestic ship and submarine building programs sits oddly with decades of economic reform. Successive governments have striven to maintain competition in the Australian economy, and for good reason. Competitive markets drive efficiency, promote innovation and contain costs.

But what’s left of the local car industry attests to the unintended consequences of such policies. And of course that helps explain how these local mega projects—well over $50 billion and extending to beyond the mid-century—came to be. Even if they won’t replace jobs lost in the manufacturing sector, they’ll be large, symbolic and make for great photo ops.

But they are also going to provide the firms involved with effective monopolies. In practice, it will be so hard to change horses in midstream that the incumbents will have an assured position for years to come. Monopoly supply arrangements arise frequently in the defence sector, usually because there’s only one supplier of a unique piece of equipment. The cost, time and expertise required to develop alternatives is a high barrier to effective competition. The risk is that these new shipbuilding programs will become as inefficient as the government-owned shipyards of the 1970s and 80s.

In other areas where it’s infeasible to maintain competition—for example, in the provision of some utilities—the government regulates the industry to protect the consumer. For example, the US Government tightly regulates its defence industry to encourage innovation and efficiency and, just as importantly, to ensure that contractors don’t extract excessive rents, notwithstanding Mr Trump’s recent concerns about aviation projects. When we buy from the US we piggyback on America’s massive investment in research and development, and on its self-interested oversight of its defence firms.

We think that the government needs to be thinking much more about regulation of the monolithic enterprises it’s creating. The Department of Defence cannot effectively do the job itself; its agenda is not the same as the taxpayer’s.  We need a new arrangement.

A case study is illustrative. Until recently, submarine maintenance was largely a monopoly arrangement—albeit one involving a government-owned supplier ASC. Up until a couple of years ago, the results were increasing costs and diminishing vessel availability. It wasn’t until the government (acting in its triple role of owner, regulator and customer) commissioned the 2012 Coles Review that performance and efficiency started to turn around. The result has been dramatic, and Australia’s submarine availability is now approaching industry best practice (though costs remain an unanswered question). The key point is that it took intervention to shake Defence and ASC into action.

We don’t want to have to wait for things to fail again. There were many institutional and technical reasons behind the poor performance of the Collins fleet, but the core problem was that successive governments failed to take action. Given the sorry record of Collins sustainment, there’s a strong case for facing up to the challenge of regulating new monopoly suppliers right from the start.

That involves much more than ensuring that profit margins are kept in check, because it’s entirely possible for a firm to be inefficient and still make modest profits. And in the absence of countervailing pressures, a monopoly will tend to be less efficient than a firm subject to market competition. We’ve been making that argument for years, and we were heartened yesterday to see Senator Xenophon telling the AFR that he’s similarly seized with the idea.

Like the good Senator, we’re unpersuaded by the government’s response that the attention of the Australian National Audit Office is sufficient. We love the ANAO’s work in the defence space but, by definition, their audits are retrospective. Instead, we need to get these things right up front, and to have the ability to continually monitor progress.

Nor are we convinced that our own organisation is the right fit, as flattering as it was to be mentioned (along with the Productivity Commission) as part of the putative oversight mechanism. We’re happy to help to define the job of the eventual regulator, and we have some ideas. But the sums and timescales involved suggest to us a need for an enduring stand-alone office, perhaps on a statutory basis, that can push against the tide of political or bureaucratic enthusiasm when required.

This piece also appears in today’s Australian Financial Review and is reprinted here with their kind permission.

The auditors get their teeth into the Tiger

An Australian Army Tiger armed reconnaissance helicopter from 1st Aviation Regiment sits in Battlegroup Griffin's position at Port Pirie, South Australia, on 2 July 2016 during Exercise Hamel. *** Local Caption *** The Australian Army’s 1st Brigade is being tested from 26 June to 14 July 2016 in South Australia as part of Exercise Hamel to ensure its soldiers are ready to meet the Australian Government’s needs as the Australian Defence Force’s ‘ready’ brigade. Battlegroup Griffin is a multirole Army aviation battlegroup made up aircrew and technicians and Tiger armed reconnaissance helicopters from 1st Aviation Regiment, and MRH-90 Taipan and CH-47F Chinook helicopters from 5th Aviation Regiment. Royal Australian Air Force personnel from the 3rd Security Forces Squadron are attached to Battlegroup Griffin for Exercise Hamel to provide intrinsic airfield security. Army uses the ‘Road to Hamel’ to build up its next ready brigade and Exercise Hamel is the final test. In 2016, Darwin's 1st Brigade is being put through its paces to ensure it is ready to support operational contingencies ranging from humanitarian assistance through to major combat operations. Exercise Hamel 2016 involves 8000 military personnel from the Australian Army, Royal Australian Navy, Royal Australian Air Force, United States Marine Corps, United States Army and the New Zealand Army. For the first time, Exercise Hamel is being conducted in South Australia and will take place around Port Augusta, Port Pirie, Whyalla and the Cultana training area.

There are plenty of ‘headline issues’ in the Australian national Audit Office’s new report on the Tiger Armed Reconnaissance helicopter. That’s not surprising, given that the government effectively gave up on the aircraft as a long term prospect in the Defence White Paper earlier this year, announcing that it would be replaced rather than upgraded in the 2020s.

On the face of it, the performance figures cited by the ANAO suggest that the Tiger was a poor choice. Even after a protracted development program that led to final operational capability being a full seven years late, there’s still a litany of issues:

  • As at April 2016, the Tiger also had 76 capability deficiencies relating to Army’s current and future operational requirements, 60 of which were deemed by Defence to be critical. Other key limitations relate to shipborne operations, pilot flying hours, interoperability and communications, airworthiness, and the roof-mounted sight.
  • To date, sustainment costs have exceeded the original contract value. The 15 year (2004–2019) sustainment contract provided for expenditure of $571 million. That sum was expended by June 2014, and expenditure as at June 2016 was $921 million.
  • As at June 2016, the cost per flying hour for the Tiger fleet was $30,335, compared to a target of $20,000. The long-term average was $39,472 per hour. Defence negotiated a cost cap to control sustainment cost growth in 2014.
  • On average only 3.5 aircraft of the operational fleet (16 aircraft) were serviceable at 10AM on any given day in 2015, against a target of 12 aircraft.

But I don’t think it would be right to conclude that the aircraft is a dud. After all, other Tiger users have deployed their aircraft on operations. French, German and Spanish aircraft have all operated in Afghanistan, collectively logging over 3,000 hours in combat operations. So it’s worth understanding what went wrong here.

Let’s start with the most fundamental issue of all. It’s abundantly clear that the initial advice to government from Defence regarding the Tiger as a low risk ‘off-the-shelf’ acquisition was wrong, when in fact there was a considerable amount of development risk remaining. That erroneous assumption would inevitably lead to cost and schedule problems. As is often the case, it was the schedule that proved more problematic, blowing out by over 100%, while acquisition cost overruns came in at just 16%.

That error was then compounded by the fact that the initial estimates for the aircraft’s sustainment cost were badly underestimated. The planned acquisition cost of the 22 aircraft was $1.6 billion, compared to the $571 million for 15 years of support. There’s an immediate red flag. When we did through-life costings for aircraft for our pre-2009 White Paper force structuring exercise, we found that supporting aircraft fleets over 20 years typically cost between 1.3 and 2.1 times the acquisition cost. Helicopters were at the higher end of the price range (2.2 for naval combat helicopters and 1.9 for the Army’s Chinooks).

There’s no escaping the relationship between acquisition and through-life support costs, as they’re both driven by the complexity of the aircraft. Based on our data, we’d expect fleet support over 15 years to at least match the acquisition cost—maybe a little less due to lower rates of effort in the early years. The contract figure of 36% of the expected acquisition price was manifestly inadequate. Not surprisingly, the actual figure as of June this year was 58% ($635 million in2001 prices), even given woeful availability leading to flying hours well short of expectations. This looks similar to the Collins class submarine story prior to the Coles review; woefully inadequate provision for support of a complex platform, which in turn leads to low rates of availability. In each case the net result was significant cost to the taxpayer for little ADF capability.

The ANAO says that Defence describe the support contract negotiations as ‘rushed’. That may be, but even a gross error check should identify an obviously over-optimistic assumption. If there’s any silver lining to be found in this aspect of the Tiger project, it’s that having to manage on inadequate resources has made for a very lean sustainment operation. Defence’s response to the auditor’s findings notes that the European users hold Australia’s cost controls in high regard. Hooray.

The other major issue with Australia’s acquisition of the Tiger is the indigenous configuration of the Army’s C4ISR network, which is not configured to work with the Tiger’s Eurogrid digital datalink. You can read more about that (and Army’s wider issues) in our recent ADF C4ISR review.

Finally, we’re not yet done with expenditure. I’ll let ANAO take us out:

‘The 2016 Defence White Paper allocated $500–$750 million to address the current capability requirements of the Tiger platform with a view to replacing the platform mid next decade, at a cost of some $5–$6 billion. In effect, an upgrade is scheduled for consideration less than 12 months after the Tiger achieved Final Operational Capability. Defence should conduct a thorough analysis of the value-for-money of investing further in the Tiger, pending the introduction of a replacement capability.’

Keep on truckin’

 An Australian Army Unimog truck, along with other Australian Army vehicles prepare to drive into Dili after a beach landing in the Comoro district of Timor-Leste.

Having to resist the urge to take a cheap shot is an occupational hazard in my business, and one that I don’t always successfully avoid. Once such was the title I gave to a review of Project LAND 121 ‘Overlander’ (replacement of Army’s 7,000 field vehicles and trailers). Published in the 2010 Budget Brief, I called it ‘How many DCPs does it take to buy a truck?’

One of my first jobs in Defence back in 1994 was to conduct analysis in support of Overlander (and the then-Army 21 program), and I wondered even then why this was being made so complicated. To see it still making its way through the system 16 years later really had me scratching my head. Five years later, Overlander is now in its third decade; an acerbic Tweet last week observed that it’s now been running several times longer than WW2.

Two things happened last week that made me think about it again. First, I heard a talk from DMO’s Land Systems Division on the project at ASPI’s land forces conference. Then, I got a copy of the Australian National Audit Office’s new report on LAND 121 Phase 3B (replacement of medium and heavy vehicles). Those two sources allow us to piece together an interesting, if doleful, story.

The conference talk very much made the point that it’s not just about trucks, because the modern Army requires its vehicles to be integrated into its wider communications and command and control system. As the land forces move into the digital age, each vehicle becomes a node in Army’s network. But we know from far too many previous experiences that systems integration is a fraught part of projects. In my talk at the same conference, I noted the time it has taken to get the Tiger helicopter to communicate effectively with the rest of Army.

Like the helicopters, the schedule for the acquisition of vehicles has slipped as well. When the first second-pass approval was made back in 2007 the Final Operating Capability (FOC) was initially expected to be in 2016. (Note: ‘first second pass’ isn’t a typo: after a project reset, there was a ‘second second pass’ in 2013, with an ‘interim pass’ in between for good measure.) After the interim pass, FOC was expected in 2019. Now the estimate is 2023, seven years late (like Tiger). Of course, that could slip too as we get to the build, systems integration and delivery. ANAO notes (as usual) that cost and schedule risks tend to rise at the systems integration phase, adding that ‘Defence will need to maintain a focus on managing the remaining integration issues’. The ANAO must have their fingers crossed when they write these words, given the sorry history of this and many other projects.

The plethora of passes follows the cancellation of contract negotiations with the preferred supplier identified through a tender process in 2005–2007. The process suffered from a litany of shortcomings, but at the heart of the problem was another ‘old faithful’ error: ‘Defence originally considered that the medium and heavy vehicle acquisition was a relatively low risk military off-the-shelf procurement’. Read the report for the story, but it was rushed, with insufficient testing and poor communication of assessed risk. Once reality set in, another tender process was launched, and more time ticked away.

As a result, we’re also seeing other time honoured effects at work—the cost to capability and the operating budget of keeping older platforms in service. ANAO make it clear that there’s been a significant financial price to pay:

The delays in the medium and heavy fleet acquisition have placed considerable pressure on the existing Unimog and Mack vehicle fleet, which has now well exceeded its life-of-type and is increasingly difficult and costly to maintain. …the average sustainment cost per vehicle for the Mack fleet has increased by some 80% between 2009–10 and 2013–14, reflecting the advanced age of the fleet and difficulty in acquiring spare parts.

And it’s not just a dollar cost; capability is getting shaky:

… one third of the South Queensland Mack [fleet] is in the process of being written off due to excessive rust following annual technical inspections. …based on vehicle availability and the number of platforms written off each year, the Mack fleet will have difficulty supporting force generation activities from 2016 and in meeting strategic guidance from 2019.

It’s hard to know what to say about all this, and it’s all drearily familiar. The Army acquisition folk talking last week seemed to have a good grasp of what’s required, but the system as a whole is seemingly incapable of producing a sensible and timely outcome. The First Principles Review is throwing the whole lot up in the air, and it remains to be seen what we’ll get next time around, but the precedents aren’t good. I can’t help but think it’s time to take a much bolder approach—perhaps finding a private sector complex acquisition specialist, telling them ‘off the shelf by strong default’, and outsourcing the lot.

A big boat doesn’t equal amphibious capability

The largest ship ever built for the Royal Australian Navy, Landing Helicopter Dock NUSHIP Canberra, passes through Sydney Heads for the first time. She will be commissioned into the RAN as HMAS Canberra. Today’s a great day for the Royal Australian Navy and the Australian Defence Force. It marks the commissioning of that $1.5 billion, 27,800-tonne behemoth soon to be known as HMAS Canberra. But as much as I hate to rain on this parade, Australia is still some time and many tough decisions away from true amphibious warfare capability. The ship is just a ‘host’ that enables the capability. Political and military leaders will need to take a  two-year appetite suppressant to consider organisational changes and the purchase of additional equipment. When the party on Garden Island ends tonight, the real work continues.

The ADF’s stated goal is to have an Amphibious Ready Group (ARG) capability by 2017. That might sound like plenty of time. But amphibious operations involve a complex and dangerous choreography and the seamless integration of joint military services. World-class amphibious players develop over decades. The US, UK, France and others have joint organisations consisting of service units dedicated solely to this kind of operation and have built an organisational culture around them. Amphibious warfare is a truly joint enterprise, requiring diligent and detailed integration of the three services.

The ADF is driven by the individual services and lacks the organisational mechanisms and culture for joint capability development. It will have to overcome those internal obstacles to get from naming a really big boat to conducting amphibious manoeuvres under non-permissive conditions. Here are a few issues to be addressed.

Read more