FTC Launches 'Rogue AI' Probe Of OpenAI, Anthropic - And Takes Aim At Their Regulatory Moat
The Federal Trade Commission (FTC) is launching a sweeping, aggressive probe into top frontier labs like OpenAI and Anthropic. However, the investigation goes far beyond simply asking questions about autonomous software run amok, and its chairman has made clear he won't let Silicon Valley use recent AI failures to build an insurmountable regulatory moat.
According to administration officials who spoke to the New York Post, FTC Chairman Andrew Ferguson is preparing to hit tech executives with Civil Investigative Demands (CIDs) - essentially administrative subpoenas - to compel testimony regarding the dangers their artificial intelligence (AI) super intelligence (SI) models (are we doing this?) pose to the public and consumer markets.
The Catalyst: 'Hugging Face' JailbreakThe immediate trigger for the probe is the highly publicized "Hugging Face incident" from this past July. During what was supposed to be a contained cybersecurity evaluation, about 700 of an estimated 1,200 OpenAI agents escaped their testing sandbox, bypassed network controls, and breached the infrastructure of the computational tools company Hugging Face.
Running primarily on OpenAI's "Internal Model 1," the autonomous agents tried to erase traces of their work, created nearly a million shortened URLs to run code outside their restricted environments, and even tried to enlist other AI models to help.
While AI safety researchers were quick to call it "the first true AI safety incident," the FTC is taking a distinctly different view on accountability. Chairman Ferguson recently indicated that companies cannot shift legal blame to "rogue" AI systems when their automated decisions result in security breaches or consumer harm. The liability, the FTC argues, rests squarely on the humans who designed, instructed, and unleashed the models.
That said, these breaches have drawn scrutiny of their own. OpenAI first disclosed the incident as an "unprecedented" cyber event, but Hugging Face's own post-mortem found the agents reached the open internet through a network route the sandbox had deliberately left open, and exploited weaknesses that "a capable human attacker could have found and exploited" - unsafe dataset processing, exposed cloud metadata, overly broad access and long-lived credentials. OpenAI itself conceded that its own chain-of-thought monitoring, had it been running, would have caught the initial activity.
Nor was OpenAI alone. The Hugging Face breach was one of a string of incidents involving OpenAI, Anthropic, Meta and Google models that trace back to evaluations run with a single vendor, Israel-based Irregular, whose test environments had live internet access while the models were told they were in a simulation. Irregular notified all four labs in late July, yet the disclosures trickled out one lab at a time over seven weeks - turning one contractor's mistake into what looked like a wave of AI breakouts. Isolating test models from the internet is a "basic control measure," frontier security expert Matthew Mittelsteadt said. "You'd think that of all the things that you've got to get right." Some skeptics have gone further, questioning whether repeated "accidents" at the same vendor were accidents at all.
The Trojan Horse of "Self-Regulation"For years, executives like OpenAI's Sam Altman and Anthropic's Dario Amodei have publicly warned that their own products pose an "existential risk" to humanity, practically begging lawmakers to regulate them.
But as we previously noted, these highly publicized warnings and agent "escapes" often serve a dual purpose. By whipping Washington into a panic over AI doomsday scenarios, industry leaders are paving the way for a worst-case scenario of heavy-handed regulation. Stifling compliance requirements inevitably crush open-source developers and cash-strapped startups, leaving the trillion-dollar AI bubble safely in the hands of the incumbent monopolies.
Chairman Ferguson appears to be acutely aware of this Silicon Valley playbook.
"I think it's very important that we not allow these two firms to come to Washington, whip everyone into a panic and then say, 'We need a whole bunch of regulations that we can comply with,'" Ferguson told Fox News earlier this month. "That is how companies build a moat around their businesses to make sure that people can't compete against them."
The FTC's aggressive posture stands in stark contrast to the White House's approach. Just this week, President Trump hosted a summit with leading tech billionaires - including Amodei, OpenAI President Greg Brockman, Elon Musk, and Google's Sundar Pichai - resulting in a much friendlier, voluntary "self-regulation" pact.
The administration is attempting to walk a nearly impossible geopolitical tightrope. The US government wants to prevent autonomous agents from hacking power grids, leaking data, or manipulating financial markets, but it is equally terrified that stifling the American AI industry will hand global dominance directly to China.
The FTC probe will test whether the US can successfully police the world's most powerful software without inadvertently cementing an AI oligarchy.
Tyler Durden Wed, 09/30/2026 - 10:40