Nothing Found

Just over 20 years ago, Spanish naval personnel operating in the Arabian Sea intercepted a merchant ship sailing from North Korea to the port of Aden. Acting on intelligence from their US counterparts, the interdiction team discovered a consignment of Scud missiles hidden on board the vessel, the So San. At the time, Washington’s concern was that the Scud missiles were bound for Iraq and preparations were building in the region for the coalition’s ill-fated intervention against Saddam Hussein’s military.

Lacking any legal basis to seize the consignment, the Spanish team could do little more than release the So San and let it sail, after receiving an undertaking from the Yemeni government that the missiles wouldn’t be transferred to any third party. From that point on, a realisation took hold that the international community needed a new architecture for counterproliferation.

What followed was the establishment of a landmark international strategy aimed at controlling the spread of chemical, biological, radiological, nuclear and high-consequence explosive (CBRNE) technologies: the Proliferation Security Initiative, which this year marked its 20th anniversary. Over the past two decades, the initiative has consolidated an alliance of more than 106 countries committed to preventing the spread of CBRNE.

However, much has changed in the geopolitical and technological landscape since the heady days of the 2000s. While the war on terror concentrated on non-state actors and an isolated ‘axis of evil’, great-power competition has returned as a central focus of international security. And although missile components and dual-use centrifuges are still being hunted down, non-proliferation is today equally concerned with materials that are smuggled with much more ease. This is especially true with the tools of dual-use chemistry and synthetic biology.

In the same year as the So San incident, a group of scientists constructed the first entirely artificial virus, a chemically synthesised strain of polio. Three years later, reverse genetics was used to recreate H1N1 ‘Spanish’ influenza, which had killed more than 50 million people in the years following the First World War. Since then, pox viruses, coronaviruses, avian flu and several other pathogens have been revived, amplified or modified into forms that can evade herd immunity or render established medical countermeasures obsolete. The technologies used to produce these infectious agents are much smaller than anything recently interdicted on the high seas: whole genome sequencers that can be held in the palm of one’s hand, chemical reagents that can be ordered online, and a host of other products that don’t need to be transported on a slow-moving ship.

In the past year, the life sciences have been further turbocharged by the latest chapter of technological advancement: the newly emergent platforms of artificial intelligence. With the debut of novel large-language models in late 2022, the public has seen minor previews of how those with malicious intent could apply AI tools to CBRNE. An exercise in Massachusetts showed how a large-language model could help students without any scientific training construct synthetic versions of the causative agents of smallpox, influenza, Nipah virus and other diseases. Elsewhere, researchers using generative AI for drug discovery found that they could also design a range of nerve agents, including VX. Discussion has even circulated of integrating AI into nuclear launch systems, a kind of digital dead hand for the new age. To all of this was added a slew of articles on AI-enabled high-consequence munitions, including fire-and-forget hypersonic missiles, autonomous loitering munitions and unmanned attack vehicles.

While much debate has erupted over the existential risk posed by AI platforms, some have argued that these concerns are either non-specific or alarmist. We, the authors, are investigating precisely how AI could accelerate the proliferation of CBRNE, and how such applications might be realistically controlled. To this end, we will be seeking expert opinions on how generative AI could lower informational barriers to CBRNE proliferation, or add new capabilities to existing weapon systems. As both researchers and clinicians, our concern relates primarily to human security, and what can be done to safeguard international public health. We hope to understand how counterproliferation professionals are confronting this new era, and what new concepts will be needed to protect human life and prosperity.

Next year, Australia will host the Proliferation Security Initiative’s Asia–Pacific exercise rotation, Pacific Protector. Air force and naval assets will be deployed across a vast expanse of ocean at a time of increased tension over a place central to AI and its underlying technologies: the Taiwan Strait. While the exercise will have several enduring uses that will benefit the Asia–Pacific, it is undeniable that the task of countering CBRNE proliferation has fundamentally changed since the initiative’s establishment.

States, non-state actors and individuals now have access to technologies and informational aids that used to be the stuff of science fiction. How policymakers, researchers and practitioners confront weapons proliferation in the new age of AI will have long-term consequences for human security in this region, and across the globe.

Almost exactly 66 years ago, 22 pre-eminent scientists from 10 countries, including the United States and the Soviet Union, gathered in Pugwash, Nova Scotia, to identify the dangers that nuclear weapons posed and devise peaceful ways of resolving conflicts among countries. With that, the international organisation known as the Pugwash Conferences on Science and World Affairs, or the Pugwash Movement, was born. Though the world is hardly free of nuclear weapons, the movement’s efforts to advance disarmament were powerful enough to win it the Nobel Peace Prize in 1995.

Today, the world needs a new Pugwash movement, this time focused on artificial intelligence. Unlike nuclear weapons, AI holds as much promise as peril, and its destructive capacity is still more theoretical than real. Still, both technologies pose existential risks to humanity. Leading scientists, technologists, philosophers, ethicists and humanitarians from every continent must therefore come together to secure broad agreement on a framework for governing AI that can win support at the local, national and global levels.

Unlike the original Pugwash Movement, the AI version would not have to devise a framework from scratch. Scores of initiatives to govern and guide AI development and applications are already underway. Examples include the Blueprint for an AI Bill of Rights in the United States, the Ethics Guidelines for Trustworthy AI in the European Union, the OECD’s AI Principles and UNESCO’s Recommendation on the Ethics of Artificial Intelligence.

Instead, the new Pugwash movement would focus largely on connecting relevant actors, aligning on necessary measures and ensuring that they are implemented broadly. Institutions will be vital to this effort. But what kind of institutions are needed and can realistically be established or empowered to meet the AI challenge quickly?

United Nations Secretary-General Antonio Guterres has called for ‘networked multilateralism’, in which the UN, ‘international financial institutions, regional organisations, trading blocs and others’—including many nongovernmental entities—‘work together more closely and more effectively’. But to be effective, such multi-stakeholder networks would have to be designed to serve specific functions.

A paper released this month by a group of leading AI scholars and experts from universities and tech companies identifies four such functions with regard to AI: spreading beneficial technology, harmonising regulation, ensuring safe development and use, and managing geopolitical risks.

Understandably, many consider ‘ensuring safe development and use’ to be the top priority. So, efforts are underway to develop an institution that will identify and monitor actual and potential harms arising from AI applications, much as the Intergovernmental Panel on Climate Change monitors the existential risk of climate change. In fact, the Global AI Observatory, recently proposed by the Artificial Intelligence and Equality Initiative, would be explicitly modelled on the IPCC, which is essentially a network of networks that works very well for accumulating knowledge from many different sources.

Cumulation networks—from the US Cybersecurity and Infrastructure Security Agency’s incident reporting system to Wikipedia—have a central authoritative hub that can pull together information and analysis from many different types of institutions, some of which are already networked. But such a hub can’t take swift action based on the information it gathers. To govern AI, a hierarchical multilateral institution with the power to make and implement decisions—such as a functioning UN Security Council—is still needed.

As for the function of spreading beneficial technology (which is just as important for most people as preventing harm), a combined innovation–collaboration network is likely to work best. Innovation networks typically include many far-flung nodes—to ensure access to as many sources of new ideas and practices as possible—and a limited number of hubs focused on transforming ideas into action, collaborating on best practices and preventing exploitation. The hubs could be centred in specific regions or perhaps tied to specific UN Sustainable Development Goals.

Harmonising regulation—including experimenting with different types of regulation—will require a broader, looser and maximally inclusive structure. AI technologies are simply too broad and too fast-moving for one or even several centralised regulatory authorities to have any chance of channelling and guiding them alone. Instead, we propose a widely distributed multi-hub network that supports what we call ‘digital co-governance’.

Our model is based on the distributed architecture and co-governance system that are widely credited with maintaining the stability and resilience of the internet. Decades ago, technology researchers, supported by the US government and early internet businesses, created several institutions in a loosely coordinated constellation, each with its own functional responsibilities.

The Internet Society promotes an open, globally connected internet. The World Wide Web Consortium develops web standards. The Internet Governance Forum brings stakeholders together to discuss relevant policy issues. And the Internet Corporation for Assigned Names and Numbers (ICANN) coordinates and safeguards the internet’s unique identifiers.

The key to these institutions’ success is that they are operated through distributed peer-to-peer, self-governing networks that bring together a wide range of stakeholders to co-design norms, rules and implementation guidelines. ICANN, for example, has dozens of self-organised networks dealing with the Domain Name System—crucial to enable users to navigate the internet—and coordinates other self-governing networks, such as the five regional institutions that manage the allocation of IP addresses for the world.

These institutions are capable of handling a wide range of policy questions, from the technical to the political. When Russia invaded Ukraine in 2022, the Ukrainian authorities pressured ICANN to remove .ru from the Domain Name System’s master directory, known as the root zone, which is managed by 12 institutions across four countries, coordinated but not controlled by ICANN. Ultimately, the Internet Assigned Numbers Authority declined the request.

A Pugwash-like conference on AI would have no shortage of proposals to consider, or governmental, academic, corporate and civic partners to engage. The original Pugwash participants were responding to a call from intellectual giants like the philosopher Bertrand Russell and the physicist Albert Einstein. Who will step up today?

ASPI’s Sydney Dialogue is already on its way to becoming the world’s premier policy summit on critical, emerging, cyber and space technologies. When we wanted to build this dialogue four years ago, not all stakeholders saw the global gap we were trying to fill, and it almost felt a bit before its time.

Today, that gap is glaringly obvious and revealed daily in the news as we watch technology race ahead. Few today would argue against its value for Australia, the Indo-Pacific region and, in light of Russia’s war on Ukraine, globally.

Critical technologies are, of course, at the centre of strategic competition between states. Should society care? Absolutely, because those very technologies are already changing our daily lives. Knowing who is building and controlling the technologies we are using every day is vital. The security, prosperity, political stability and social harmony of nations—and coalitions of nations—will all depend on their ability to harness the promise and create clear parameters to guide the safe and responsible use of critical technologies.

Quantum computing, for example, offers the potential to solve in seconds complex problems that would take the fastest supercomputer thousands of years. Such a breakthrough could render existing security measures obsolete and shift social and geopolitical relations overnight. Artificial intelligence, meanwhile, has the promise of freeing us from both routine and complicated tasks, allowing individuals and organisations to focus on higher-level strategic and creative endeavours. But it also involves the risk of the AI not understanding important nuances in global geopolitics and human rights, for example, which can lead to mis- and disinformation. We are already seeing how AI can be weaponised to pollute our information environment with dangerous propaganda and deepfakes.

Beyond the technologies themselves, there are transformational shifts in how innovation is taking place and who is doing the innovating. During the Cold War, the research and development of critical technologies—particularly those relevant to security—were driven largely by huge government-funded programs in government-controlled facilities. Of course, only a small number of governments had the resources to generate cutting-edge technologies. Today, leading- and bleeding-edge technologies are emerging out of commercial R&D programs, which means they are available to a wider range of strategic players, large and small. This complicates national efforts; technology companies based in one country can, and in many cases do, work for the governments of other countries, even when the strategic goals of the two countries are misaligned.

Innovation is also increasingly achieving a bootstrapping effect. As former Google CEO Eric Schmidt has written of artificial intelligence: ‘Faster airplanes did not help build faster airplanes, but faster computers will help build faster computers.’

Regulatory and policy responses are struggling to keep up. No single sector, be it government, the tech industry or civil society, has all the answers. But democracies risk falling further behind as authoritarian regimes, without the same governance principles or accountability and transparency mechanisms, and with military–civil fusion, have surged ahead to the detriment of democratic ideals, social harmony and international standards. While we thought the internet and social media were uncontrollable, they harnessed new capability to wield greater power. With yet another tech revolution imminent, we cannot afford to be slow and passive again. We have to learn from the past or it will be a case of third strike.

That’s why the Sydney Dialogue was established in 2021. We need forums where the world’s leading thinkers and decision-makers—from government, industry, academia and civil society—can meet and exchange ideas and strategies, deepen cooperation and form new partnerships. While we’re attuned to the threats and challenges, we’ve very deliberately sought contributions from actors who have a positive and proactive vision for technology that upholds fundamental values such as human rights, openness and sovereignty.

Those players who share this vision must be the ones who shape the rules of the road. We don’t want to see a lawless space, and we certainly don’t want to see a technology ecosystem shaped by actors who are happy to use technology to spread disinformation, enable coercion or trample fundamental individual rights.

At the heart of the Sydney Dialogue is a discussion about how to harness the opportunities and mitigate the risks of technology, which will include the benefits of international collaboration, ways to protect sovereignty and build resilience, policies to foster investment and innovation, the next game-changers in technology, geopolitics and why technology lies at the heart of competition, and how we work together on governance and public policy.

Key technologies include AI, 5G/6G, robotics, biotech, nanotech, quantum, space, our information environment and of course cyber. The conversation brings in economies, climate change and energy, our societies, foreign policy, national security, defence capability, intelligence and ethics, and many other topics.

The goal of the Sydney Dialogue is to make a lasting contribution to ensuring these technologies grow in the right way—making us more prosperous, more equal, wiser, freer, safer and more secure. We aim to give the Indo-Pacific a new platform and a greater voice in global debates, while helping to build cooperation and global partnerships between government, industry and civil society, and between partners and allies.

Using automated systems, such as those enabled with artificial intelligence, to assist with recruitment can create efficiencies, but the increasingly complex interplay of machines, ownership and control of data, contract performance and human factors makes the effects difficult to predict. While solutions may be found by integrating hiring technology with deep knowledge of organisational needs and culture and adapting skills standards, recruitment is only one part of the workforce crisis facing Australia’s defence organisation.

These issues should be front of mind in the transition of the Australian Defence Force recruiting contract from Manpower to Adecco in the context of shortfalls in recruitment. Issues of the contract with Manpower have not been canvassed in public, but there are, without doubt, commonalities with contracting in other countries, such as the UK, which also has faced myriad recruitment challenges.

Recruitment methods, of course, rely increasingly on data, analytics and algorithms, and globally there are efforts to push legacy personnel systems into the information age.

The UK Ministry of Defence has released an AI strategy that has as one pillar transforming the armed forces into an ‘AI ready’ organisation by addressing human talent.

The UK strategy comes after a long period of underperformance in recruiting. A parliamentary committee inquiry criticised both the British Army and the contractor, Capita. The army achieved only 60% of its targets in 2018–19 and identified issues in recruitment in future skills and capabilities, such as digital technology and cybersecurity.

The inquiry found that Capita had underestimated the complexity of defence recruiting and the specialist expertise needed. It concluded that Capita’s off-the shelf commercial solution was not fit for purpose, and noted that a bespoke application took longer to develop than anticipated.

An overly prescriptive contract containing 10,000 specifications negatively affected innovation in the recruitment process. The recruitment system was not owned by the British Army and was hosted on the contractor’s servers.

Problems clearly remain. The system went offline for six weeks this year after a possible hack. The information of 120 recruits was found on the dark web.

The UK issues will no doubt resonate with those familiar with Australia’s problems.

The ADF and the Department of Defence are both below their budgeted personnel numbers. Deputy Prime Minister and Defence Minister Richard Marles said last month that the recruitment issue must be dealt with urgently. He said the ADF was almost 3,000 people below its allocated force strength and the Department of Defence was more than 1,000 people below its budgeted size.

The contract between the ADF and the new contractor, Adecco, will likely reflect the revised defence intellectual property strategy that outlined a new regime for collaboration with industry.

The ownership of data and online systems by contractors creates dependencies that may limit the scope for employers to gain insights from that data and to innovate. The experience of the British Army is instructive. Despite multiple years of missed targets, the British Army didn’t amend the contract with Capita to secure the intellectual property for the online recruitment system that it had co-developed and co-funded.

In recruitment systems, there are good reasons to quarantine candidates’ files—which may contain the results of aptitude tests and other personal information—from other defence systems. But air-gapped data creates other points of vulnerability—the ADF confirmed in October that hackers had attacked an external service provider used by military personnel and public servants.

During Australia’s first Covid-19 shutdown, defence recruitment applications surged, rising 42% in 2020 according to the ABC. But converting interest into boots on the ground has proved difficult. Digitising the recruitment process seems to have also driven an increase in applications from ‘unsuitable’ candidates.

The ADF’s three services have each apparently insisted on tailoring standards—including medical, physical and psychological assessments—based on job roles and improving the speed of application. They essentially want more control over who is deemed ‘suitable’.

This potential change to suitability criteria will undoubtedly attract internal and external criticism about supposedly declining standards. The inquiry in the UK said found that the British Army wasn’t changing its entry standards but widening its entry criteria. The ADF has already adapted its recruitment process to assess the physical differences between men and women more fairly while also relaxing its grooming standards to adapt to social norms. And as different defence requirements emerge—cyber is a classic example—defence has had to adjust.

Addressing retention is also a key part of solving the workforce problem, as ASPI’s Marcus Hellyer argued recently. While some elements of the defence workforce model will always remain, a cookie-cutter model for the ‘right’ sailor, soldier or aviator is no longer organisationally beneficial.

Data might be able to help Defence determine why candidates fail at different points, and equally where management is failing. But that information will be limited if partial details have been shared between the contractor and Defence, and if senior management isn’t monitoring performance, as was the case in the UK.

Equally, recruitment methods need to be adapted to the needs of different groups. Evidence of those involved in recruitment of Indigenous candidates, for example, suggests that advancement often occurs in contexts where they have strong peer support. Indigenous candidates haven’t had high rates of recruitment, so machine-learning systems trained on historical data could reinforce the same exclusions. The same is true for other groups.

In various contexts, AI or advanced analytics are being proposed to deliver a range of benefits. These include increasing the ability of organisations to conduct background checks; assess candidates’ values, beliefs and attitudes; and suggest possible behavioural patterns in relation to job fit and performance. But these systems can lead to perverse outcomes and legal risks, and in some instances can look like 19th-century-style pseudo-science. Efforts to identify ‘trustworthiness’ and ‘criminality’ in people’s facial structures are modern-day phrenology, no matter how many data points are being processed and how sophisticated the algorithm.

As Defence’s data-management systems become increasingly ‘optimised’ in line with its data strategy, we should be mindful that optimisation may not mean control and that contract complexity may stymie innovation. These are big questions in themselves, but the question of whether these enhancements will help tailor a force structure to align with likely workforce limits is even bigger.

Artificial intelligence is a general-purpose technology that is steadily becoming pervasive across global society. AI is now beginning to interest the world’s defence forces, but the military comes late to the game. Given this, defence forces globally are fundamentally uncertain about AI’s place in warfighting. Accordingly, there’s considerable experimentation in defence AI underway worldwide.

This process is being explored in a new series sponsored by the Defense AI Observatory at the Helmut Schmidt University/University of the Federal Armed Forces in Germany. Unlike other defence AI studies, the series is not focusing solely on technology but instead is looking more broadly across what the Australian Defence Force terms the ‘fundamental inputs to capability’. The first study examines Australian defence AI, and another 17 country studies have already been commissioned.

The ADF conceives of AI as mainly being used in human–machine teams to improve efficiency, increase combat power and achieve decision superiority, while lowering the risk to personnel. For a middle power, Australia is following a fairly active AI development program with a well-defined innovation pathway and numerous experimentation projects underway.

There is also a reasonable level of force structure ambition. The latest major equipment acquisition plan, covering the next 10 to 20 years, sets out six defence AI-relevant projects, one navy, one army, three air force and one in the information and cyber domain. Even in this decade, the AI-related projects are quite substantial; they include teaming air vehicles (with an estimated cost of $9.1 billion), an integrated undersea surveillance system ($6.2 billion), a joint air battle management system ($2.3 billion) and a distributed ground station ($1.5 billion).

Associated with this investment is a high expectation that Australian AI companies will have considerable involvement in the projects. Indeed, the government recently added AI to its set of priorities for sovereign industrial capability. The Australian defence AI sector, though, consists mainly of small and medium-sized companies that individually lack the scale to undertake major equipment projects and would need to partner with large prime contractors to achieve the requisite industrial heft.

There are also wider national concerns about whether Australia will have a large enough AI workforce over the next decade to handle commercial demands, even without Defence drawing people away for its requirements. Both factors suggest Defence could end up buying its AI offshore and rely principally on long-term foreign support, as it does for many other major equipment projects.

An alternative might be funding collaborative AI developments with the US. A harbinger of this may be the Royal Australian Navy’s new experimentation program involving a recently decommissioned patrol boat being fitted with Austal-developed autonomous vessel technology featuring AI. Austal is simultaneously involved in a much larger US Navy program fitting its system to one of the company’s expeditionary fast transport ships, USNS Apalachicola, currently being built. In this case, Austal is an Australian company with a large US footprint and so can work collaboratively in both countries. The RAN, simply because of economies of scale, is probably more likely to adopt the US Navy variant rather than a uniquely Australian version.

The outlier to this acquisition strategy might be the Boeing Australia Ghost Bat program that could see AI-enabled, ‘loyal wingman’ uncrewed air vehicles in limited ADF service in 2024–25, before the US. The US Air Force is running several experimentation programs aiming to develop suitable technologies, some of which also involve the Boeing parent company. There’s a high likelihood of cross-fertilisation between the Australian and US programs. This raises the tantalising possibility of a two-nation support system of a scale that would allow the Australian companies involved to grow to a size suitable for long-term sustainment of the relevant ADF AI capabilities. This possibility might be a one-off, however, as there seem to be no other significant Australian defence AI programs.

Australia collaborating with the US on AI or buying US AI products can ensure interoperability. But in seeking such an objective there’s always a tension between each Australian service being interoperable with its US counterpart or instead across the ADF. This tension is likely to remain as AI enters service, especially given its demands for task-related big data.

Interoperability and domestic industry support are traditionally important issues, but they may need to be counterbalanced by emerging geostrategic uncertainties and ADF capability considerations. Australia is worried about the possibility of conflict in the Indo-Pacific region given Chinese assertiveness coupled with the example of Russia’s invasion of Ukraine. To offset the numerically large military forces of the more bellicose Indo-Pacific states, some advocate developing a higher quality, technologically superior ADF that can help deter regional adventurism.

In being a general-purpose technology, AI can potentially provide a boost across the whole ADF, not just one or two elements within it. But such a vision is not what is being pursued. Defence’s current AI plans will most likely lead to evolutionary improvements not revolutionary changes. AI is envisaged as being used to either enhance, augment or replace existing capability; this approach means the future ADF will do things better, but it won’t necessarily be able to do better things.

A revolution in Australian military affairs seems unlikely under current schemes. For that, defence AI would need to be reconceptualised as a disruptive technology rather than a sustaining innovation. Embracing disruptive innovation would be intellectually demanding and, in suggesting the adoption of unproven force structures, could involve taking strategic risks. These are reasonable concerns that would need careful management.

Against such worries though, China looms large. The strategically intelligent choice for the ADF might be embracing disruptive AI.