PROJECT MAVEN: A Marine Colonel, His Team, and the Dawn of AI Warfare, by Katrina Manson

Science-fiction authors have long fantasized about wars where machines aim the weapons and pull the triggers. In “Project Maven,” the Bloomberg News tech and national security reporter Katrina Manson claims the era of “killer robots” is here. The fusion of artificial intelligence and modern warfare raises, as she puts it, “the biggest moral and practical question there is: who — or what — gets to decide to take a human life? And who bears that cost?”

Yet that question has barely been mulled by the officers managing the new weapons or the politicians funding them. It’s a familiar syndrome in military history: New weapons technologies are perfected and rushed into the arsenal before their implications — for the future of not only war but humanity — are thought through.

How this happened with A.I. over the course of barely a decade, much of it secretly plotted by a handful of clever obsessives, is the story that Manson tells in “Project Maven,” which is also the name of the Pentagon’s main A.I. program.

The U.S. military hasn’t yet fired a weapon with no humans in the loop, but we are poised on the thin edge of that evolution. The Defense Department regulation on autonomous weapons requires only “appropriate levels of human judgment over the use of force” — without specifying who defines “appropriate.” For many existing weapons, especially the latest drones, A.I. controls every stage from the launch up to the actual kill; it does so in a matter of seconds, and human troops trust the A.I. so fully that they pull the trigger, at the algorithm’s urging, with little hesitation.

Manson’s story — deeply researched and breezily written — begins when a Marine colonel named Drew Cukor was watching a 2017 documentary about an A.I. program that beat an 18-time Go champion in four out of five games. He realized that if A.I. was spreading everywhere, it would soon spread to the battlefield. Cukor knew China was adopting it. If the United States didn’t, he feared we might lose the next big war.

The Pentagon bureaucracy at the time had little interest in the subject; it barely had a presence in the cloud. The big arms manufacturers were still focused on Cold War-era weapons. Cukor, who had been a Mormon missionary in the 1980s, set out to change the world with what one Project Maven official describes as an “insurgency mentality.” He recruited a small team that “would come to regard itself as a scrappy and subversive cult,” Manson writes, exhausted yet inspired by its leader’s fanatical brilliance. A Project Maven contractor describes him as “a psychopath,” sometimes adding “in the best way.”

Cukor was not just a visionary but also, as a deputy defense secretary puts it, “a bureaucratic Ninja.” He scoped out, as an ideal contractor for A.I. weapons, a then-obscure software firm called Palantir (which is now the world’s most highly valued defense corporation). He ghost-drafted Project Maven’s list of requirements in a way that made Palantir seem uniquely suitable for the job. He schooled Palantir’s execs — whose “pernicious” tactics, Manson writes, had alienated the Army on a previous project — on how to pitch Pentagon insiders.

Finally, Cukor found and lobbied the two or three senior Pentagon officials who, once sold on the idea, could ram the project through the bureaucracy’s indifference or hostility.

Cukor kept his true purpose secret. He had sold Maven as strictly an intelligence-gathering system. However, he later admitted to Manson, his goal was “always” to develop an A.I. that could identify targets on the battlefield and calculate which weapons in the U.S. arsenal could most quickly destroy them — and eventually, it seems, take out the targets on their own.

Compressing the “kill chain” so completely was “precisely the thing that would most scare” the project’s skeptics, Manson writes. So Cukor never mentioned his true goal and, Manson reports, admonished his team not to talk about it either — until the program racked up successes, and then commanders and their troops found its possibilities irresistible.

The turning point of Maven’s fortunes came with Russia’s invasion of Ukraine in February 2022. President Joe Biden ruled against mobilizing U.S. troops (to avoid risking World War III), but, even before sending weapons, he gave the Ukrainians access to top-secret intelligence, and Maven was the main facilitator.

Almost immediately, a U.S. Army brigade, led by a Cukor enthusiast, set up an A.I. targeting system for the Ukrainians. Silicon Valley firms, including Palantir, sent technicians to tweak the database, making 60 refinements in the first few months.

In the end, Maven’s data stream let Ukrainian soldiers see and identify tens of thousands of targets — armored vehicles, missile launchers and various military facilities — and destroy hundreds of them a day, some inside Russian territory. To the extent Ukrainians beat back the much mightier Russian Army and mounted a counteroffensive, Maven played a major role.

A.I. targeting systems played a still bigger, if less publicized, role in Israel’s war in Gaza. After Hamas’s terrorist attack of Oct. 7, 2023, the Israel Defense Forces used A.I. to identify 15,000 targets — and proceeded to bomb many of them.

The results shined a harsh light on A.I. warfare’s “double-edged sword.” Some of the A.I. targets (10 percent of them, according to one estimate that Manson notes) were misidentified. The I.D.F. has said that it uses human analysts to cross-check the work of its machines, but even aside from any potential mistakes or ambiguities, A.I.-enabled weapons — which have been sold as a way to reduce civilian casualties — could make war more deadly. They give commanders “more targets, faster, and make it easier to strike them,” Manson writes. “It is hard to imagine this will lead to fewer strikes.”

The book was finished before this spring’s U.S.-Israeli airstrikes on Iran, which destroyed more than 13,000 targets, many of them at least in part civilian. Maybe the paperback edition will detail the extent to which Maven once again wreaked its awesomely efficient, and occasionally indiscriminate, powers in this theater too.

The dilemmas posed by A.I. weapons have triggered debates even among the technology’s creators. Google at first dropped out of Maven after its employees petitioned against doing any military work, but rejoined after the contracts grew lucrative and the cause — defending Ukraine — became appealing.

In February, Anthropic’s chief executive, Dario Amodei, publicly refused to work on completely autonomous weapons, but only because today’s A.I. systems “are simply not reliable enough” to support them. In his announcement, he touted “partially autonomous weapons,” like those used in Ukraine, as “vital to the defense of democracy” and allowed that fully autonomous ones “may prove critical” someday.

Even so, his critique, plus his refusal to work on domestic surveillance projects (which he said undermine “democratic values”), pushed the Trump administration into declaring Anthropic a “supply-chain risk,” which would bar it from all federal contracts. (President Trump has since begun to reconsider the prohibition.)

As with most spheres of life, it’s too late to ban A.I. from the battlefield, but Manson makes a case for continuing to challenge its use. Even Cukor — who retired from the military in 2021, never talked with any other reporters and only spoke with Manson after she begged him for nearly a year (and meanwhile interviewed his associates and critics) — is apprehensive about the revolution that he spawned. “We have all this tech,” he muses in the last of their many sessions. “Are we the best custodians of it?”

PROJECT MAVEN: A Marine Colonel, His Team, and the Dawn of AI Warfare | By Katrina Manson | Norton | 406 pp. | $31.99