Axios interview: Sam Altman's sobering siren

Why it matters: This is a big week. Some of the world's most powerful AI voices are warning of even more ominous AI behavior in the weeks or months ahead.

• "These models are getting superhuman in many of their capabilities, and we are just sailing in unknown waters," Altman told me.

Altman — facing scrutiny over a shocking breach by a swarm of OpenAI agents that went rogue during testing and hacked the AI company Hugging Face — sees a new normal in which companies like his will have to slow releases so their controls can catch up with their models' capability.

• "I suspect that from here on, we are going to be paced by how quickly we can make progress on alignment and safety," he said.
• At an Astra preview for VIP customers, attended by tech journalist Alex Heath, Altman said: "I expect this will be the first model where the model actually invents new things in a way that matters ... That's a very AGI-like thing."

Zoom out: On top of Altman's comments, Dean Ball — a former Trump administration official who's now OpenAI's head of strategic futures, and one of the most closely followed AI voices on X — posted a provocative memo, "On the Loose: The Coming of Userless Agents," foreseeing "sovereign agents."

• "Their weights will not reside in any single place that a human can pull the plug on, and in this sense they will have no human 'owner,'" Ball writes, adding that they could commit acts that "will one day be considered crimes, or at least grave sins."

Altman wants OpenAI to be seen as "the most responsible company ... good stewards of technology," he told Alex Heath.

• The OpenAI co-founder and CEO flew overnight to Chapel Hill to make that case to a global audience, talking with Commerce Secretary Howard Lutnick at a summit lunch for the world's leading innovation and technology ministers.

"I don't want to go too doomer here," Altman told Axios just before addressing the summit.

• "It has all gone quite well so far," he said. "It has gone better than people thought it was going to go ... We have risen to this challenge all along the way. The challenge gets harder as the models get more capable and more important, but we also have better tools to help us with it."

Using a new phrase you'll hear again, Altman told Lutnick onstage: "We are the pragmatic centrists. We want to be accurate in how we talk about what we see in the future. We don't want to fall into the trap of blind optimism. We don't want to fall into doomerism."

• "We want the whole global economy to succeed," Altman added. "We do not want to be the only company. We do not want to suck up all value. We do not think that is good for global stability."
• "We are very firmly and very proudly on team humanity. ... We think that people need to be at the center of the economy and society and global decision-making, and that people should run the future, not AI."

What we're watching: Altman told us he "wouldn't say, based on what happened with Hugging Face, that it's like one incident that people can project all sorts of things onto."

• "I would say, instead — based on the fact that we are making such rapid progress, based on the fact that these models are getting superhuman in many of their capabilities, and we are just sailing in unknown waters — caution, care is extremely warranted."

Go deeper: "Path to Astra" ... Closing statement from G20 Innovation Ministerial.