Blab Tech
Senate Approves Human-In-the-Loop Requirement for U.S. Autonomous Weapons Amid AI Deployment Push
The Senate Armed Services Committee voted 18‑9 on Thursday to adopt a measure introduced by Arizona Democrat Mark Kelly that would require “ultimate human responsibility” in the use of force by autonomous weapons systems. The amendment, incorporated into the Senate version of the National Defense Authorization Act (NDAA) for fiscal year 2027, codifies the 2023 Department of Defense (DoD) Directive 3000.09, which mandates human oversight of lethal autonomous weapons.
The measure comes after President Donald Trump issued a presidential memorandum on June 5, 2024, directing the Pentagon to “eliminate unnecessary barriers to rapid deployment” of artificial intelligence (AI) and to provide an updated version of Directive 3000.09 within 90 days. Kelly’s amendment would effectively block that directive, if it survives the full‑Senate vote, by preserving human involvement in the “kill chain.”
Kelly, a former Navy combat pilot and astronaut, added the provision to the Senate text of the NDAA. He later voted against the bill, citing the $1.15 trillion price tag and a lack of transparency, saying, “We cannot write them another blank check.” The amendment was approved by the committee and forwarded to the Senate floor.
AI has increasingly entered U.S. military operations. In January, Anthropic’s large‑language model Claude was reportedly used in the capture of Venezuelan President Nicolás Maduro. The same model was also cited as being used on the first day of Operation Epic Fury in Iran, where a missile strike on a school near a military base killed 175 people, most of them young girls. Kevin Baker, a technology scholar, argued that the AI program used to identify targets—known as the Maven Smart System—was responsible for the error, noting that “People failed to update a database, and other people built a system fast enough to make that failure lethal.”
Maven, engineered by Palantir Technologies in 2017, analyzes intelligence data to recommend targets. A New Yorker article reported that it takes a person four clicks from target identification to target destruction, illustrating the system’s role in compressing the kill chain.
The amendment has drawn praise from some experts who see legal guardrails as a moral imperative. Paul Lushenko, a non‑resident expert at RegulatingAI and lecturer at George Washington University, said, “There’s a perception shared among the military and the public that there ought to be human oversight of these capabilities.” He added that while tools like Maven can be valuable, guidelines are needed to determine accountability.
Concerns about the DoD’s ability to reinterpret or waive directives without congressional approval were voiced by Zaza Tsotniashvili, a professor at Caucasus International University. He warned that “DoD directives can be reinterpreted, waived or quietly revised by the executive branch without congressional approval.” Trump’s memorandum does not specify who would conduct oversight or what it would entail, a point Tsotniashvili described as “ambiguity serves the interest of those deploying these systems.”
Senator Ruben Gallego sent a letter to Secretary of Defense Pete Hegseth demanding clarification on how the updated directive would protect U.S. servicemembers and civilians. The letter was issued on Monday, the same day the amendment was sent to the Senate floor.
Defining “human oversight” remains a challenge. Lushenko said the concepts of meaningful human control and appropriate human judgment are open to interpretation, especially when checks can be built into the design of autonomous weapons. Jovana Davidovic, a professor at the University of Iowa, said developers have a responsibility for the actions of any AI system they create, noting that “The more autonomous the system is, the more the decisions that developers make ultimately drive outcomes.”
Tsotniashvili argued that codifying Directive 3000.09 is necessary but insufficient, calling for specifics that ensure oversight is “meaningful rather than ceremonial.” He added, “The 2027 NDAA provision is a step forward, but without accompanying accountability mechanisms, it may succeed in preserving the appearance of human control while the substance of it continues to erode.”
The amendment is now before the full Senate. Its fate will determine whether the U.S. maintains a human‑in‑the‑loop requirement for autonomous weapons amid a broader push to accelerate AI deployment in the military.
The measure comes after President Donald Trump issued a presidential memorandum on June 5, 2024, directing the Pentagon to “eliminate unnecessary barriers to rapid deployment” of artificial intelligence (AI) and to provide an updated version of Directive 3000.09 within 90 days. Kelly’s amendment would effectively block that directive, if it survives the full‑Senate vote, by preserving human involvement in the “kill chain.”
Kelly, a former Navy combat pilot and astronaut, added the provision to the Senate text of the NDAA. He later voted against the bill, citing the $1.15 trillion price tag and a lack of transparency, saying, “We cannot write them another blank check.” The amendment was approved by the committee and forwarded to the Senate floor.
AI has increasingly entered U.S. military operations. In January, Anthropic’s large‑language model Claude was reportedly used in the capture of Venezuelan President Nicolás Maduro. The same model was also cited as being used on the first day of Operation Epic Fury in Iran, where a missile strike on a school near a military base killed 175 people, most of them young girls. Kevin Baker, a technology scholar, argued that the AI program used to identify targets—known as the Maven Smart System—was responsible for the error, noting that “People failed to update a database, and other people built a system fast enough to make that failure lethal.”
Maven, engineered by Palantir Technologies in 2017, analyzes intelligence data to recommend targets. A New Yorker article reported that it takes a person four clicks from target identification to target destruction, illustrating the system’s role in compressing the kill chain.
The amendment has drawn praise from some experts who see legal guardrails as a moral imperative. Paul Lushenko, a non‑resident expert at RegulatingAI and lecturer at George Washington University, said, “There’s a perception shared among the military and the public that there ought to be human oversight of these capabilities.” He added that while tools like Maven can be valuable, guidelines are needed to determine accountability.
Concerns about the DoD’s ability to reinterpret or waive directives without congressional approval were voiced by Zaza Tsotniashvili, a professor at Caucasus International University. He warned that “DoD directives can be reinterpreted, waived or quietly revised by the executive branch without congressional approval.” Trump’s memorandum does not specify who would conduct oversight or what it would entail, a point Tsotniashvili described as “ambiguity serves the interest of those deploying these systems.”
Senator Ruben Gallego sent a letter to Secretary of Defense Pete Hegseth demanding clarification on how the updated directive would protect U.S. servicemembers and civilians. The letter was issued on Monday, the same day the amendment was sent to the Senate floor.
Defining “human oversight” remains a challenge. Lushenko said the concepts of meaningful human control and appropriate human judgment are open to interpretation, especially when checks can be built into the design of autonomous weapons. Jovana Davidovic, a professor at the University of Iowa, said developers have a responsibility for the actions of any AI system they create, noting that “The more autonomous the system is, the more the decisions that developers make ultimately drive outcomes.”
Tsotniashvili argued that codifying Directive 3000.09 is necessary but insufficient, calling for specifics that ensure oversight is “meaningful rather than ceremonial.” He added, “The 2027 NDAA provision is a step forward, but without accompanying accountability mechanisms, it may succeed in preserving the appearance of human control while the substance of it continues to erode.”
The amendment is now before the full Senate. Its fate will determine whether the U.S. maintains a human‑in‑the‑loop requirement for autonomous weapons amid a broader push to accelerate AI deployment in the military.
