Journal of Political Risk, Vol. 10, No. 1, January 2022

Helen Toner, Director of Strategy at the Center for Security and Emerging Technology (CSET) at Georgetown University.

Anders Corr, Ph.D.
Publisher of the Journal of Political Risk

The JPR interview with Helen Toner, the Director of Strategy at the Center for Security and Emerging Technology (CSET) at Georgetown University, was conducted via email between 4 January 2022 and 13 January 2022.

Corr: What are the national security risks and benefits of AI?

Toner: This is a huge question! AI is a general-purpose technology, meaning that—like electricity or the computer—its impacts will be felt across practically all industries and areas of society. Accordingly, it presents a huge range of potential risks and benefits from a national security perspective. One way of trying to summarize the possibilities might be as follows: the benefits will largely be in line with the kinds of benefits we have seen from increasingly sophisticated computing technology more generally: greater efficiency and accuracy, as well as the ability to perform tasks at scales impossible for humans (think: how Google search trawls the web). In terms of risks, one breakdown proposed by Zwetsloot and Dafoe is to think in terms of risks from accidents (i.e. unintended outcomes from using AI), misuse (i.e. the deliberate use of AI to cause harm), and structural changes (i.e. how progress in AI shapes surrounding systems and dynamics). I realize this is fairly abstract, but it’s impossible to enumerate specific risks without narrowing the scope to particular application areas, time frames, and actors.

Corr: What might some unintended outcomes of AI be?

Toner: Again, there are many possibilities depending on the time frame and application area we’re talking about. In the near term, we’re already seeing AI systems discriminate on the basis of protected classes (e.g. in loan applications or criminal sentencing), fail unexpectedly (e.g. crashes of autonomous vehicles), or create unexpected side effects (e.g. engagement-based algorithms on social media promoting radicalizing content). In the medium term, if AI continues to be deployed in a wider range of settings, we’re likely to see further, even more consequential failures of this kind—unless significant research advances are made in AI reliability, transparency, and fairness. In the longer term, if we continue to build systems that are increasingly capable of making decisions and pursuing goals, some scientists believe we may see far worse unintended outcomes, for instance if the systems we build learn to hoard resources, deceive their creators, or otherwise pursue undesirable means towards the ends we have programmed into them.

Corr: Not long ago, an AI jet fighter beat a human fighter in a simulation. If the adversarial relationship of the United States and China continues, are we bound to move towards AI armies built to defeat nation states? Are there unique dangers to this possible future scenario?

The current generation of AI systems, which is largely based on a set of techniques called “deep learning” or “deep neural networks,” is failure-prone, opaque, and easy to disrupt. These properties answer both parts of your question: On the one hand, they’re reasons that we’re unlikely to see the complete handover of all battlefield functions to machines any time soon. On the other hand, AI is already being used in a limited capacity to support various military functions, so deep learning’s weaknesses are likely to start to become visible there. Again, a lot depends on how much better we get at building reliable, secure, interpretable systems that can work towards nuanced goals. Current AI systems tend to fail in ways that are completely unintuitive to humans—for instance, mistaking a photo of a schoolbus for an ostrich—and could accordingly cause serious damage if used prematurely in high-stakes settings like military operations.

Corr: You have done extensive research into AI safety. How should we design better AI systems to ensure that these systems work in a reliable and safe manner?

Toner: Building AI systems that are safe, reliable, fair, and interpretable is an enormous open problem. Research into these areas has grown over the past few years, but still only makes up a fraction of the total effort poured into building and deploying AI systems. If we’re going to end up with trustworthy AI systems, we’ll need far greater investment and research progress in these areas. Organizations building and deploying AI will also have to recognize that beating their competitors to market—or to the battlefield—is to no avail if the systems they’re fielding are buggy, hackable, or unpredictable.

Corr: What research are you working on now?

Toner: I have a couple of papers in progress looking at different aspects of AI safety—one on how research into three different areas of AI safety (robustness, interpretability, and reward learning) has developed in recent years, and one introducing the concept of “uncertainty quantification” and all its concomitant challenges for a non-technical audience. More broadly, my role at CSET also involves higher level strategic planning and priority-setting for the organization as a whole, and on that front I’m excited about a new line of work we’re currently setting up that will focus on similar challenges—standards, testing, safety, accidents, and all the different ways AI systems can go wrong. We have our first couple of full-time researchers starting on that this winter.

Anders Corr, Ph.D., is the publisher of the Journal of Political Risk, the author of The Concentration of Power: Institutionalization, Hierarchy & Hegemony (Optimum Publishing International, 2021), No Trespassing: Squatting, Rent Strikes, and Land Struggles Worldwide (South End Press, 1999), and editor of Great Powers, Grand Strategies: The New Game in the South China Sea (U.S. Naval Institute Press, 2018).