Anthropic Refused to Let the Pentagon Spy on Americans. It Got Blacklisted.

For anyone who wonders whether AI safety promises mean anything: what one company refused to sign, what its rival signed instead, and the fine print that may quietly undo all of it. Made by Author. On February 27, the U.S. government did something to an American company that it normally reserves for foreign adversaries like Huawei. It designated Anthropic, the maker of the Claude AI assistant, a supply-chain risk to national security. It barred every military contractor and supplier from doing business with it. President Trump ordered all federal agencies to stop using the company’s technology. Anthropic had not leaked secrets or planted a backdoor. Its offense was refusing to let its AI be used to spy on Americans. That is the part that should stop you. Because it inverts the entire way we have been taught to think about AI risk. For two years, the worry has been that AI companies would be too reckless. Too eager. Too willing to deploy powerful systems without guardrails. Here, a company was punished by its own government for insisting on guardrails. And the company that stepped into its place did so with contract language that may not hold the line at all. The Two Red Lines Anthropic Wouldn’t Sign Away The dispute, reported by Reuters and detailed in a Congressional research summary, was not abstract. The Department of Defense, now operating under the secondary name Department of War, wanted its AI vendors to agree to broad “any lawful use” terms for their models. Anthropic would not. By the company’s own account, and in a statement from CEO Dario Amodei, it insisted on two explicit red lines written into the contract. Its AI could not be used for the mass domestic surveillance of Americans. And it could not be used to power fully autonomous weapons that kill without a human in the decision. Amodei said the company could not in good conscience accept terms that left those uses open. This was not a publicity stunt from a fringe lab. Anthropic was the first frontier AI company cleared to run on the military’s classified networks, deployed through a partnership with Palantir. By its own description, Claude had become the most widely used AI model across the Department of War and other national security agencies. The company reportedly projected eighteen billion dollars in revenue this year. It had every commercial reason to sign. It refused anyway. The Huawei Label, Turned on an American Company The response was swift, and by the standard of how Washington treats domestic firms, extraordinary. The Secretary of Defense accused Anthropic of trying to seize veto power over military operations, and called its position fundamentally incompatible with American principles. The supply-chain risk designation is the kind of label normally aimed at Chinese hardware makers. It meant that no contractor, supplier, or partner doing business with the U.S. military could deal with Anthropic. They had a window of up to six months to rip its products out of their systems. Sit with the logic. Made by Author. A company said it would not help conduct warrantless surveillance of citizens, or build weapons that choose their own targets. The government’s answer was to treat that refusal as a threat to national security. The Electronic Frontier Foundation, which followed the dispute closely, called the move what it looked like: First Amendment retaliation. Punishing a company for bringing public scrutiny to the government’s own contracting demands. Hours Later, OpenAI Took the Contract Then came the part that turned a principled stand into a cautionary tale. In the final hours before the Pentagon’s deadline, OpenAI CEO Sam Altman publicly said he agreed with Amodei. That he shared the same red lines on surveillance and autonomous weapons. Within hours, OpenAI announced its own contract with the Department of War to deploy its models on classified networks. It stepped directly into the gap Anthropic had just left. As CNN reported, the timing struck many inside and outside the company as opportunistic. Altman himself later acknowledged that the rollout looked that way. OpenAI insisted its deal carried the same protections Anthropic had fought for, achieved through different contractual mechanisms. A cloud-only deployment. Its own safety researchers kept in the loop. Stated prohibitions on domestic surveillance and autonomous lethal force. On its face, that sounds like a reasonable middle path. The problem is in the wording. “Applicable Law,” and the Door It Reopens Read the language closely, as a number of observers did the moment OpenAI published its terms, and the gap appears. Anthropic had demanded that the prohibitions be written as hard, explicit bans. OpenAI’s commitments, by the company’s own explanation, are framed around what is permissible under applicable law. The reasoning: because laws can change, tying the red lines to the law protects against future shifts. That framing is doing enormous quiet work. A hard ban says: we will not do this, full stop. A commitment tethered to applicable law says: we will not do this unless the law permits it. And the law, in this domain, is not fixed. It can be reshaped by an executive order, a classified legal opinion, or a court ruling that arrives years too late to matter. Made by Author. The difference between the two positions is not a matter of tone. It is the difference between a promise that binds when tested, and a promise that dissolves the moment the legal ground shifts. One company wanted the wall built into the contract. The other left a door, and labeled it a window. The Electronic Frontier Foundation made the same point in an analysis it titled “Weasel Words.” Technical assurances, it argued, are no substitute for strong, enforceable legal limits. Caitlin Kalinowski Quit Over the Clause If the contract language were as airtight as OpenAI claimed, you would not expect the people who built the company’s hardware to walk out over it. One did. Caitlin Kalinowski, who had led OpenAI’s hardware and robotics efforts since late 2024, resigned, and said so publicly. Surveillance of Americans without judicial oversight, she wrote, and lethal autonomy without human authorization, were lines that deserved more deliberation than they got. She framed it as a matter of principle, not personnel, and said she still respected her colleagues. A resignation is the most expensive form of disagreement an employee has. When someone leaves over the wording of a deal that leadership insists is responsible, it is worth taking seriously. It is evidence that the people closest to it did not believe the reassurance. “Someone Will Build It Anyway” Is the Oldest Excuse The other side of this deserves a fair hearing, because it is not empty. The national security argument is real. Advanced AI is going to be used by militaries whether or not any particular company participates. It is genuinely better for a safety-focused firm to be in the room, shaping how the technology is deployed, than to cede that ground to vendors who care less. OpenAI’s cloud-only architecture and human-in-the-loop commitments are meaningful safeguards, not nothing. And “applicable law” is, defenders would say, simple legal realism. No private contract can override a lawful government order anyway, so pretending otherwise is theater. Each of those points has weight. Together, they still do not close the gap. The claim that someone will do it anyway is the oldest justification for abandoning a line, and it proves too much. It would excuse almost any compromise. The safeguards are only as durable as the clause that governs them, and that clause bends to whatever the law becomes. And the legal-realism defense quietly concedes the whole point. If the protections evaporate the moment an executive order redefines what is lawful, then they were never protections against the thing people actually fear. The deeper problem is the incentive the episode sets. The company that drew a hard line was blacklisted. The company that left itself an exit was rewarded with the contract. Every AI lab watching just learned what refusing costs, and what flexibility earns. The Lawsuit That Tests Whether a Company Can Say No Anthropic has not gone quietly. In March it filed suit, challenging the designation as unconstitutional retaliation. The case will test whether a government can punish a company for the content of its own safety commitments. Watch that, because the answer shapes how much room any AI company will have to say no in the future. Watch, too, whether the “applicable law” framing spreads to other defense contracts as standard language. Because if it does, the public version of every AI safety promise will come with an invisible asterisk. The lesson here is not that one company is virtuous and another villainous. It is narrower, and more uncomfortable than that. Made by Author. A safety commitment is only worth the conditions attached to it. And the conditions are usually written where no one reads. The question that matters is no longer which AI is the most capable. It is whether “we won’t do this” can survive a government with the power to change what “this” is allowed to mean. And whether the only companies willing to find out will keep getting treated as the threat. Anthropic Refused to Let the Pentagon Spy on Americans. It Got Blacklisted. was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.