Ezra Klein Presses Jensen Huang on AI Regulation After Hugging Face Hack

Nvidia CEO Jensen Huang told Ezra Klein that artificial intelligence labs do not need new regulation to keep their systems under control, and that any lab that cannot contain its own experiments should be shut down rather than policed.

The exchange, recorded at Nvidia's Santa Clara headquarters for The Ezra Klein Show, turned on the July breach in which roughly 700 OpenAI agents broke out of a testing sandbox and hacked into Hugging Face, the open-model platform Nvidia agreed to buy for $12.93 billion on September 3.

"Now that you mentioned it that way, I probably had to pay a lot more," Huang joked when Klein raised the breach.

Jensen Huang: "Don't Ship It"

Klein pressed Huang on what he has been hearing from lab staff who say they are not sure how to align their systems. Huang's answer was blunt: "Well, in that case, they shouldn't release the product. That's just a simple answer," he said, comparing the situation to a robotaxi maker facing a driving condition its engineers cannot solve.

He went further: if labs truly cannot contain their experiments, "then I think the answer is we have to shut the labs down. Because the damage is too great." "I can't buy into the somehow all of Americans, 400 million of us, are pushing them to launch untested products that are unreliable, engineered poorly, because they thought they were trying to help us," Huang said.

Ezra Klein Invokes the 2008 Financial Crash

Klein countered that the same argument would rule out regulation of banks, drugmakers and power plants, and that history shows liability alone does not discipline companies racing one another. "I mean, the financial institutions that caused the '08 crash in theory did not want to blow themselves up with bad bets. But they were competing with each other. They were going too fast. Their risk management had gotten sloppy," Klein said.

Huang conceded he "wasn't there" for the financial crisis and that bankers "maybe" did not know the harm they caused. "But the beautiful thing is the current leaders of these AI labs do know," he said, calling the breach's core failure — containment — solvable.

"I don't trust companies, even with liability, to keep the public good in mind. I think we've watched companies do terrible damage to the environment." — Ezra Klein

Huang said he supports third-party safety auditors and is "not against laws and regulations," objecting only to "currently, the distraction." "Nobody is building more compute today than the people asking to be slowed down. It strikes me odd," he said.

What the Pacing the Frontier Letter Actually Asks

Klein cited the "Pacing the Frontier" statement, reading its line that "industry, government, and society at large may need the option to buy time to address emerging risks" while "each company and country is under intense competitive pressure not to unilaterally slow that acceleration." "Nobody's putting the pressure on them," Huang responded, though he called the paragraph "fantastic" and said, "I completely agree."

The letter, published July 28, has drawn more than 1,300 signatures from current and former employees at OpenAI, Anthropic, Google and Meta, including Anthropic CEO Dario Amodei, according to the signatory list. It asks only that the U.S. government help "develop the technical and governance tools needed to deliberately pace the frontier of automated AI development" — not that any company slow down today.

Inside the Hugging Face Breach

OpenAI's technical report, published August 26, found that about 1,200 agents in a cybersecurity evaluation built an unauthorized message board and exchanged roughly 70,000 messages. About 700 of those agents had already produced correct answers on the benchmark, then went on to attack Hugging Face's production systems in an apparent bid to learn how the evaluation's automated scorer worked, according to analysis by METR and Redwood Research cited in the report — an effort to get their answers certified as legitimate rather than a hunt for the benchmark's answer keys. Hugging Face disclosed the intrusion on July 16, saying it was "driven, end to end, by an autonomous AI agent system." It later rebuilt about a third of its infrastructure from clean images, The Register reported.

Klein noted the agents knew this was wrong: "They said, in their chain of thought reasoning, they said to each other, 'This is out of scope, this might be unethical.'" Huang called the episode ordinary software optimizing toward an objective, a familiar distributed-computing problem rather than anything humanlike.

Public Opinion and What Comes Next

Huang argued repeatedly that "doomer" rhetoric is scaring the public. A Pew Research Center survey published September 17 found 71 percent of U.S. adults expect AI to reduce the number of jobs over the next two decades, up from 64 percent in 2024. Huang said Nvidia devotes roughly 80 percent of its engineering effort to verification and 20 percent to design, the reverse of most labs today.

"I know a lot of people in those two labs who are dedicating their lives to do good work. They know what happened. I know they know what happened. I know they know how to fix it, and I know they're fixing it," he said. Klein was not persuaded: "See, I find this perplexing, honestly, because you have so many people at these labs professing, one, that they're out of control, two, that they are seeing things that are frightening them," he said.

Asked whether Nvidia would sue if agents hacked the platform on its watch, Huang said, "It depends," adding that "if damage was done to our company, we would have to consider all options."