Lately, I’ve been spending a lot of time thinking about the AI inference market and how big it could get. Agents like Muse, Instinct, Town, and Grok Bot aggressively use browsers, run their own computers, and burn through far more tokens than a traditional chatbot. As more people put them to work (Muse is number two in the App Store), the demand for inference, or the computing needed to run these models, should grow enormously.
That’s why I was excited to talk with Tuhin Srivastava, the CEO and co-founder of Baseten, for episode three of the Sources podcast. His company builds the infrastructure that AI applications use to serve their customers. You may never have heard of Baseten, but Tuhin says that if you use popular AI apps, there’s a good chance some of your AI activity already runs through his platform.
Baseten’s growth gives a sense of what’s happening: The company raised funding at a $5 billion valuation in January and again at $13 billion in June. Tuhin tells me its token volume has grown 40 times year over year, with revenue up roughly tenfold over the last 12 months. He expects Baseten to become one of the largest users of compute on the planet within the next year or two.
Of course, growing demand doesn’t guarantee that an independent company like Baseten wins. It’s competing in a market the big clouds also want, where access to GPUs alone is a hard business to defend. I ask Tuhin what makes Baseten more than a reseller of someone else’s chips and why he thinks companies will increasingly want to own and improve their AI using their own data.
We also discuss Baseten’s recent acquisition to power agents working in browsers and virtual machines, why companies are embracing Chinese open models, and his ambitions for Baseten’s research lab. After my conversations with Sam Altman and Mark Zuckerberg, I wanted to hear from someone building the infrastructure behind this shift. Tuhin is betting that we’ve barely started to see how much of it we’ll need.
In addition to Substack, the Sources podcast is available wherever you get podcasts, including audio on Apple Podcasts and video on YouTube and Spotify.
A MESSAGE FROM MY SPONSOR
Here’s something for busy people like you
When you’re back-to-back all day, there’s no time to process. No time to write the follow-up. No time to turn “we should probably...” into, “ok done”.
Granola helps you become the person who actually does what they promised they’d do.
You can still make small notes in the meeting, or not, either way Granola quietly transcribes in the background, then turns those notes into clear summaries with actual next steps. After your call, share the notes with the team so everyone’s aligned. Or even chat with the actual notes to pull out exactly what you need to do next. No need to re-read the whole thing.
Download Granola and try it in your next meeting.
Episode highlights
Tuhin says the AI industry has done a “pretty terrible job” bringing the public along. The backlash against data centers affects Baseten’s ability to get the compute it needs, and he acknowledges that communities have valid concerns. In today’s episode, I press him on how the industry can explain AI’s benefits while its own researchers warn about the risks.
He thinks inference could be the only market left after AGI. His argument is that even if models become capable of doing everything, they’ll still need computing power and software to do it. “Without AGI, inference is going to be the biggest market ever,” he tells me. With AGI, “it’ll be the only market left.”
Baseten’s defense against becoming a thin GPU reseller is to help companies own their AI. Tuhin describes a “continual learning loop” in which customers run models, evaluate the results, improve them with their own data, and repeat. His bet is that helping companies control that process and protect their margins will make Baseten harder to replace.
AI agents are already deploying models on Baseten. Tuhin says agents are choosing which models to use and running them through the platform. Baseten also uses agents to investigate customer incidents, though it hasn’t given them free rein to make consequential changes in production. When I ask what he’ll never delegate to an agent, he says spending time with other people. “All we ever had,” he tells me, “is people.”
Sources is a newsletter and podcast by Alex Heath about the AI race, featuring conversations with leading founders and technologists. Every week, Sources reaches the inboxes of thousands of decision-makers in tech, finance, policy, and media.
