Sam Altman dishes on OpenAI's top token user
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Sarah Grillo/Axios
OpenAI CEO Sam Altman said the company's top token user is going through 100 billion tokens per month.
Why it matters: As people inside and out of AI labs are spending more and more on tokens, Altman acknowledges that cost concerns have become a "huge issue."
What they're saying: "The token leader at OpenAI uses about 100 billion tokens a month. To my embarrassment, that's not the token leader in the world. We found someone that used even more," Altman said on a livestream about OpenAI's enterprise adoption.
- The number is staggering when put into context: Over six years ago, the top token user at OpenAI blew through about 100,000 tokens per month, Altman said.
- That's a 1 million-fold increase in token usage.
Between the lines: AI demand is exploding.
- That's especially true given the tokenmaxxing culture that's led to things like internal token leader boards, or measurements of how many AI tokens employees are burning through.
- While that could sound like a positive for AI labs that benefit from token spend, the boom in demand also highlights "the infrastructure challenge ahead of us," Altman said.
- Compute constraints can prevent AI labs from being able to service all the inbound demand they're receiving, putting a ceiling on potential revenue growth.
Zoom out: Altman has talked about the AI costs problem several times this week.
- He said cost concerns are the second-most common issue he hears about from customers behind simplifying AI workflows. Cost never came up before but is now "all of a sudden a huge issue."
- "We want you all to be able to use AI and never worry about it being great and affordable," he added.
Follow the money: Figuring out how to make AI cheaper will be especially important given what Altman said is coming next from OpenAI: "constant running proactive AI."
- "If there's one thing to get ready for as the next phase over the next year, this is the one I would pick," Altman said, adding this will change how companies plan around AI usage and security.
- Autonomous AI that runs without being prompted can get expensive quickly.
- Just ask the CFO that accidentally oversaw a $500 million IT bill in one month.
What we're watching: How OpenAI handles the corporate cost reckoning.
