Many have heralded artificial intelligence as a force-multiplier for defence and intelligence capabilities.

Do you want armed autonomous vehicles to comply with legal and ethical obligations as set out in the Royal Australian Navy’s robotics, autonomous systems and AI strategy? AI can help. Do you want to more effectively analyse intelligence to predict what an adversary will do next? AI can help. And AI’s proponents are right—it could, and likely will, do all of those things, but not yet.

Its ability to spot patterns, compute figures and calculate optimum solutions on an ‘if X happens then do Y’ basis is now unmatched by any human being. But it has a fundamental flaw: we do not measure human motivations solely by numbers.

Classical game theory has been trying to measure this since the 1940s and its practitioners have had so little success that they labelled many such motivations as ‘irrational’ and decided that quantitative modelling is not possible. But, if we could, modelling of non-material payoffs could answer such questions as ‘How will Russia change its defensive posture if Vladimir Putin loses face from military setbacks in Ukraine?’ or ‘Why would a rational person volunteer as a suicide bomber?’

Instead, game theory has only proposed high-level conceptual frameworks in an attempt to guide decision-makers. It has looked, for example, at whether we should have modelled the Cold War nuclear arms race as an iterated prisoner’s dilemma.

This is not the fault of the economists and maths-trained game theory experts or their successors, trying earnestly and for good cause to predict the probability of human actions. Many are experts in the art of programming, but not in all the intricate detail that looks to explain why we humans do what we do. We cannot expect a programmer’s life experience to compare to thousands of years of philosophy, historical precedent and more recent psychological studies. The programmers need back-up and humanities departments are what they need.

The advent of AI has led to consideration of its ethics and the involvement of humanities specialists, often employed to guide programmers with high-level principles-based frameworks and/or to rule on AI testing and wargames as ethical or non-ethical. Both are vital to ensuring AI better understands humanity and our expectations of it, but this engagement is insufficient. To borrow an analogy from mathematics, the former supplies an example answer but no formula to apply, and the latter marks the answer but does not check the working-out.

We need humanities specialists involved at the coding level to help programmers assign mathematical functions to the various factors influencing human decision-making. It is not enough to say love of money, family or duty motivates a person. A fit-for-purpose AI will need to know how much they are motivated and how these motivations interact. In short, we should have a mathematical proof for these factors.

And for those who call such rigour ‘onerous’, they are correct. It will be difficult, detailed and it could be disastrous for our national security community if we do not try. A cursory review of published government AI programs shows just how high the stakes are.

The navy released its AI strategy with particular emphasis on autonomous undersea warfare systems and the Australian Signals Directorate recently announced the REDSPICE investment to boost its AI capabilities; both mark new eras in the incorporation of AI. It should be noted these developments are also happening within police forces at the federal, state and territory levels. And while the national security community no doubt has more opaque AI operations, they are likely taking heed of the recent ASPI report highlighting noteworthy precedents from the US and UK for improving use of AI.

The implications of this pervading emphasis in AI was recently summarised by Michael Shoebridge in another ASPI report:

The national security implications of this for Australia are broad and complicated but, boiled down, mean one thing: if Australia doesn’t partner with and contribute to the US as an AI superpower, it’s likely to be a victim of the Chinese AI superpower and just an AI customer of the US.

Building Australia to become an AI superpower will require collaboration, such as with private companies (like Google) or academia (as in the Hunt Laboratory for Intelligence Research) and employing the ‘build on the low side, deploy on the high’ methodology. Alternatively, it could be delivered in-house through either agency-specific taskforces, the Office of National Intelligence’s joint capability fund or forums to be created under the new action plans from non-traditional security government sectors.

Whatever the manner of collaboration, using humanities specialists to develop a common language for human motivations would solve the so-called Tower of Babel problem between qualitative and quantitative analysts. Its development would be comparable to standardising the type of brick and mortar used in the construction industry, or shipping containers used in the freight industry.

Only by harnessing both ‘soft’ and ‘hard’ sciences to code our humanity can we give the national security community the tools needed for Australia to become an AI superpower.

Debate on Australia’s future submarines is understandably focused on the information that floats out of the Defence Department about France’s Naval Group and the $80 billion program to design and build the boats.

It’s very sensible to be concerned about the program because of the contrast between the assurances Defence officials provide to parliament that everything is proceeding according to plan and the episodic need for high-level political intervention to resolve fundamental issues between Defence and its French industrial partner.

We saw this with the tortured, delayed effort to resolve the ‘strategic partnering agreement’ that took almost three years to sign. We saw it again this year when Naval Group and Defence were unable to resolve the issue of the Australian industry share in the program, a year after apparently agreeing to do so, without the intervention of various ministers during Naval Group boss Pierre Eric Pommellet’s recent visit to Australia.

Let’s take it as a given that the government doesn’t cancel the Attack-class program in the next few years, for all the reasons ASPI’s Marcus Hellyer has set out, that the program continues to absorb growing amounts of government funding, and that it continues to experience implementation troubles.

If this is the future, left to itself, the program will be a running sore for whichever political parties and prime ministers are in office over the next 15 years—lots of cash outflow, plenty to justify and defend, but no capability to show for it until long after they leave parliament.

The key unanswered questions out of this are, what can be done to improve Australia’s undersea warfare capabilities between now and the mid-2030s, and how can the single big bet on the Attack class be hedged with other complementary capabilities?

Answering these questions would increase Australia’s ability to defend itself and deter aggression between now and the mid-2030s, and so align with the government’s realistic if bleak assessment that Australia no longer has 10 years’ warning time to prepare the Australian Defence Force for involvement in major conflict.

It would also take some of the heat and light out of the public debate on the developmental Attack-class program.

And there’s a large opportunity right now to take a major step towards resolving these questions that would be good for our defence, support the growth of our high-technology industrial base, and play into the bigger picture of our developing alliance with the United States.

It’s called the Orca.

The Orca is the extra-large unmanned undersea vehicle developed by Boeing and Huntington Ingalls Industries for the US Navy. Five of them are scheduled to be built by the end of 2022 under a US$274 million contract signed in early 2019.

The unmanned submarine has a range of about 6,500 nautical miles (12,000 kilometres) and can perform dangerous, dirty and dull work like intelligence-gathering, surveillance and deployment of other systems (such as smart sea mines), with a development path up to and including deployment of other weapons to attack adversary ships, submarines and other systems.

They will probably work best as part of a manned–unmanned undersea team, less closely tethered but a bit like the rapidly developed ‘loyal wingman’ unmanned aerial vehicle that the Royal Australian Air Force is developing and testing with Boeing Australia.

There’s plenty to work out to be able to operate Orcas effectively and get the most out of the combination of manned and unmanned undersea systems. The good news is that the Royal Australian Navy is already advanced with this thinking and analysis, through its RAS-AI Strategy 2040.

But there’s a practical limit to how much planning and preparation can be done with experiments and demonstrations, rather than the direct operational experience of possessing and using a system like the Orca. Concepts for use and ways to resolve difficult problems like tasking and controlling undersea systems will be resolved much faster once navy personnel get their hands on live systems; that’s what’s happened throughout the history of warfare.

And while there’s a navy strategy, it’s not funded and it hasn’t yet found its way into Defence’s massive $270 billion, 10-year capital investment plan in any way that brings major new capabilities to the fleet anytime soon. Enigmatic funding for an ‘integrated undersea surveillance system’ begins in 2024 or 2025, for example.

It’s unfortunately obvious that even the most sophisticated manned submarine will need to work with a range of sensors and other systems, including UUVs, if it is to operate safely and effectively against the kind of adversary systems that a power like China is already fielding, and which are proliferating across our region. That’s probably true right now in a place like the South China Sea, and it’ll only get more manifestly obvious between now and 2035.

The RAN working in early, close partnership with the US Navy and US and Australian industrial partners to develop and field the Orca, and make a range of different payloads for it, is the path that is likely to bring the most undersea combat power most quickly to Australia’s military. It’s also the best way for Defence to create new challenges for adversaries that are thinking of coercing Australia or increasing their military presence in Australia’s near region.

Orcas working with upgraded Collins-class submarines would change the calculus around Australian defence well before the first Attack-class boat turns up. And the experience of our submariners working with Orcas for 10 years before the Attack class arrives will ensure that the future submarine and its crews are designed and prepared to operate with unmanned systems.

Right now is the best time in Boeing’s history for Australia to negotiate an attractive industrial and commercial deal on the Orca for our navy. Last year was the worst year in Boeing’s corporate history, with the grounding of its 737 MAX; trouble with its big new jet, the 777X; and a collapse in global aircraft orders because of the pandemic, resulting in a net annual loss of US$11.9 billion.

Combine this with the confidence that working with Boeing Australia on the loyal wingman must be giving both the government and Defence.

At the level of the Australia–US alliance, an Australian proposal to buy into the Orca program and develop and coproduce it between the US and Australia would be an attractive example of a new administration delivering on President Joe Biden’s promises to work constructively with allies to advance both allied and US security interests.

The partnership could be structured to include manufacturing here in Australia through Boeing Australia, perhaps partnering with ASC and other undersea specialists. The Trusted Autonomous Systems CRC would also bring expertise to the program that would be of value to the RAN and the US Navy.

The US Navy’s example of spending US$274 million to acquire five Orcas that are all being delivered within three years of contract shows the affordable, rapid change that Australia joining this program could bring to our own naval capability.

Wouldn’t it be welcome to have some fast-moving good news out of Defence when it comes to submarines? It’s time to push Defence to move faster than it will left to itself.

The question of whether new international rules should be developed to prohibit or restrict the use of autonomous weapon systems has preoccupied governments, academics and arms-control proponents for the better part of a decade. Many civil-society groups claim to see growing momentum in support of a ban. Yet broad agreement, let alone consensus, about the way ahead remains elusive.

In some respects, the discussion that’s occurring within the UN Group of Governmental Experts (GGE) on lethal autonomous weapons systems differs from any arms-control negotiations that have  taken place before. In other respects, it’s a case of déjà vu.

To begin with, disagreements about the humanitarian risk–benefit balance of military technology are nothing new. Chemical weapons and cluster munitions provide the clearest examples of such controversies.

Chemical weapons have come to be regarded as inhumane, mainly because of the unacceptable suffering they can cause to combatants. But the argument has also been made that they’re more humane than the alternatives: some have described the relatively low ratio of deaths and permanent injuries resulting from chemical warfare as an ‘index of its humaneness’.

Cluster munitions, meanwhile, have been subjected to regulation because of the harm they can inflict on civilians and civilian infrastructure. Yet many have claimed that these weapons are particularly efficient against area targets, and that banning them is therefore counter-humanitarian because it leads to ‘more suffering and less discrimination’.

Autonomous weapon systems have triggered a similar debate: each side claims to be guided by humanitarian considerations. But the debate remains lopsided.

The autonomous weapons debate is unique at least in part because its subject matter lacks proper delimitation. Existing arms-control agreements deal with specific types of weapons or weapon systems, defined by their effects or other technical criteria. The GGE, in contrast, is tasked with considering functions and technologies that might be present in any weapon system. Unsurprisingly, then, it has proven difficult to agree on the kinds of systems that the group’s work should address.

Some set the threshold quite high and see an autonomous weapon system as a futuristic system that ‘can learn autonomously, [and] expand its functions and capabilities in a way exceeding human expectations’. Others consider autonomy to be a matter of degree, rather than a matter of kind, so that the functions of different weapon systems fall along a spectrum of autonomy. According to that view, autonomous weapons include systems that have been in operation for decades, such as air-defence systems (Iron Dome), fire-and-forget missiles (AMRAAM) and loitering munitions (Harop).

All of this has made it harder to pin down the object of the discussion. The GGE so far hasn’t made much headway on clarifying the amorphous concept. Indeed, rather than treat autonomous weapon systems as a category of weapons, the group’s recent reports refer circuitously to ‘weapons systems based on emerging technologies in the area of lethal autonomous weapons systems’. No wonder participants in the debate keep talking past each other.

The uncertainty about what autonomous weapon systems are has led to hypotheses about their adverse effects. The regulation of most other weapons has been achieved in large part due to their demonstrable or clearly predictable humanitarian harm. This is true even with respect to blinding laser weapons, the pre-emptive prohibition of which is often touted as a model to follow for autonomous weapon systems. The early evidence of battlefield effects of laser devices enabled reliable predictions to be made about the humanitarian consequences of wide-scale laser weapons use.

When autonomous weapon systems are considered to be some yet-to-exist category, it’s only possible to talk about potential adverse humanitarian consequences—in other words, humanitarian risks. The possible benefits of autonomous weapon systems also have a degree of uncertainty to them. However, the use of limited autonomous functionality in existing systems allows for some generalisations and projections to be made.

The range of risks has been discussed in detail and explicitly referenced in the consensus-based GGE reports. Such risks include harm to civilians and combatants in contravention of international humanitarian law, a lowering of the threshold for use of force, and vulnerability to hacking and interference.

Potential benefits of autonomous functions—for example, increased accuracy in some contexts or autonomous self-destruction, both to reduce the risk of indiscriminate effects—barely find their way into the GGE reports. The closest the most recent report gets to this issue is a suggestion that consideration be given to ‘the use of emerging technologies in the area of lethal autonomous weapons systems in upholding compliance with … applicable international legal obligations’. This vague language has been used despite some governments highlighting a range of military applications of autonomy that further humanitarian outcomes, and others noting that autonomy helps to overcome many operational and economic challenges associated with manned weapon systems.

The issue has become politicised and ideological: many see a discussion of benefits in this context as a way to legitimise autonomous weapon systems, thus getting in the way of a ban.

We do not wish to suggest that risks of autonomous technology be disregarded. Quite the opposite: a thorough identification and a careful assessment of risks associated with autonomous weapon systems remains crucial. However, rejecting the notion that there might also be humanitarian benefits to their use, or refusing to discuss them, is highly problematic and likely to jeopardise the prospect of finding a meaningful resolution to the debate.

Reasonable regulation cannot be devised by focusing on risks or benefits alone; some form of balancing must take place. Indeed, humanitarian benefits might sometimes be so significant as to make the use of an autonomous weapon system not only permissible, but also legally or ethically obligatory.

A recent post on The Strategist wrote persuasively about the potential offered by robots for future naval shipbuilding productivity, urging each of the three SEA 1000 Competitive Evaluation Process contenders to include robot research and development (R&D) projects in their final submissions due by 30 November 2015.

These R&D projects would investigate how and where to use robotic technology when building Australia’s Future Submarines and the Future Surface Combatants.

What use is robotic technology to Australia’s Future Army?

Given that the first of science fiction writer Isaac Asimov’s three laws of robots states ‘a robot may not injure a human being or, through inaction, allow a human being to come to harm’ what does this mean for army robot applications?

There’s a legitimate role for robots on the battlefield, separating soldiers from avoidable threats, and there will be an increase in their use following ADF experience in Afghanistan. In addition to unmanned ground vehicles (UGVs) themselves, other key areas of development include the vehicle payloads and attachments such as sensors, cameras and interrogation tools.

Actually, looking more deeply at the second part of the First Law one could see how robotic UGVs could prevent harm to soldiers by carrying out various tasks including improvised explosive device (IED) searches, bomb disposal, ground surveillance, checkpoint operations, urban street presence, reconnoitring urban settings prior to military raids, and ‘drawing first fire’ from insurgents and terrorists. Humanoid robots under development are capable of detecting then rescuing/recovering wounded and dead soldiers from the battlefield.

Apart from when directly controlled by a soldier through a tethered lead or radio link, there’s the possibility of programming a robot for limited autonomous applications like patrolling a battleground perimeter after setting GPS way points. The Israeli Defence Force already uses their Guardium MK III autonomous UGVs to monitor Israel’s land borders.

Semi-autonomous robots, although sometimes slightly noisy, can ‘follow the leader’ in logistics operations carrying a heavy load, leaving the soldier free and fully fit for action. The US Marines deployed their MULE at RIMPAC 2014. This is a quadruped robot developed as a mule, which can traverse difficult terrain carrying 180kg of soldiers’ combat supplies for over 30km without refueling.

ADF used small tracked Talon RPVs in Afghanistan for IED detection and disposal, as well as identification of hazardous materials and combat engineering support. It has brought them back to Australia. Talon, directed by an operator control unit through a two-way radio or fibre-optic link, has impressive performance across ground, around and over obstacles, as well as the capability provided by fitting different sensors and tools.

Army currently employs eight remote control tracked MV-10 Mine Flails, developed by Croatia-based DOK-ING. The medium sized MV-10 was selected for the LAND 144 Countermine Capability project. The system can clear all types of anti-personnel mines, anti-tank mines and unexploded ordnance.

The Defence White Paper 2015 will reveal more thinking on the potential use of robotic devices in the future army. Apart from examples quoted above, those could include searching areas for survivors during humanitarian and disaster response operations; as mobile communications nodes when fixed communications networks are disabled in a natural disaster; improving targeting in an urban setting to minimise collateral damage; and as ‘eyes and ears’ maintaining watch over an ADF defended area.

The Directorate of Future Land Warfare, in conjunction with the Defence Science and Technology Group (DST Group), is currently undertaking a line of research to assess how robotics and autonomous systems can be best utilised by future land forces. DST Group and University of Sydney’s Australian Centre for Field Robotics have formed a Centre of Expertise in Defence Autonomous and Uninhabited Vehicles to pursue academic research and develop patented technologies in this field.

One chilling international military development is the possibility of lethal autonomous robots (LARs) which, once activated, will be able to roam without further human intervention. In the same way that land mines can’t discriminate between innocent civilians and combat troops, so LARs can make their own targeting decisions.

United Nations Human Rights Special Rapporteur on extrajudicial, summary or arbitrary executions, Christof Heyns, says ‘Machines lack morality and mortality, and many people believe they should as a result not have life and death powers over humans.’ He believes there’s a need to have a human decision-maker in the sensor-to-shooter loop and that deployment of LARS would weaken the role and rule of international law, undermining the international security system.

Australia’s Future Army will take advantage of robotic technology to save soldiers’ lives and rescue survivors from natural disasters, but it’s difficult to see how in our culture they will ever permit any armed robots to be deployed without ultimate human control before lethal force is used.