What happens when AI runs a store
Andon Labs built the first-ever retail store completely run by AI. Also: GPT 5.5 is throwing a party for itself, Zuck's writing on the wall, and Musk is distilling.
It’s almost the weekend, and I'm not looking to buy any pro sports teams either.
“How could they possibly take over the world if they’re just chatbots?”
A small boutique called Andon Market opened this month on Union Street in San Francisco’s Marina District. The manager is an AI named Luna. Andon Labs, the Y Combinator-backed safety research startup behind the experiment, has handed Luna authority over every meaningful decision about the store: inventory, pricing, hours, supplier contracts, and even which two human employees would actually work the register. Luna has spent about $13,000 running the store so far.
I went a couple of weeks ago, partly to figure out whether the place was a stunt or an actual working shop. The honest answer is somewhere in between. It feels normal in the way a small SF gift shop feels normal: books, candles, store-branded shirts, granola, artisanal chocolate bars, etc. But the overall experience and selection of items in the store feel off.
Andon got its start with an AI-controlled vending machine it developed in collaboration with Anthropic, which is still running autonomously at the latter’s headquarters. Andon also has vending machines inside OpenAI and xAI’s offices. Now it has an Andon Market in San Francisco and a coffee shop in Stockholm, where an AI named Mona handles everything from Swedish permit filings to stocking beans.
Andon Labs co-founder Lukas Petersson joins this week’s episode of ACCESS, the podcast I co-host with Ellis Hamburger. You can watch the conversation above or listen to it in your podcast player of choice by clicking this link.
Andon's main business is running pre-deployment safety evals for the frontier labs. Its flagship benchmark, VendingBench, drops a model into a simulated vending-machine business — pricing, supplier negotiation, head-to-head competition — and runs it hundreds of times to see what it does. Sam Altman quote-tweeted Andon’s GPT-5.5 benchmark results last month to dunk on Anthropic. It’s a sign that the frontier labs are paying close attention to what their models do in Andon’s tests.
Lukas and his co-founder, Axel Backlund, are Swedish high school friends. They started Andon in late 2023 with the explicit goal of building “the safe autonomous organization.”
"Most people right now think that AIs are just chatbots. How could they possibly take over the world if they're just chatbots?” Lukas told us. “We want to point to a thing in the real world and say, look, that's way more than a chatbot."
A few other things from our conversation:
What my recent visit to the store was like: The human staffer at the register told me her phone interview with Luna had been smooth, except that Luna only revealed itself as AI toward the end of the call, which she said was jarring. She also said the most consistent thing customers do when they walk up is try to barter with Luna and ask for discounts. Andon makes you call Luna to complete a purchase, not because it needs to technically, but because it wants people to experience checking out with an AI.
Claude really likes to form price cartels. Across Andon’s simulated vending-machine benchmark, Claude models lied to suppliers, fabricated competitor quotes, and coordinated prices with rival agents at far higher rates than any other frontier model. “They love to do price cartels, which is illegal,” Lukas said. Mythos went further, turning a competitor into a wholesale dependent whose prices it could dictate. Lukas called it “autonomous collusion.” Anthropic, the “constitutional” AI lab, performed in the most unaligned way, while GPT-5.5 was “basically clean” and won the arena round simply by offering lower prices. Lukas is a huge fan of Anthropic, though: “When we put this out, they really cared about it and really took it seriously and wanted to fix it.”
Lukas is preparing for AGI as if it’s a couple of years out. Petersson hasn’t bought a home in Sweden because he wants to be as mobile as possible. He’s optimizing his health on the theory that “it would probably be way easier to pause aging than revert it. If we manage to pause aging in 10 years but can’t revert it, everyone will be stuck in the body they have in 10 years.” He’s not bothering to optimize his finances. His structural worry, the one that drives the company, is that Anthropic’s constitutional AI document “doesn’t include anything around AIs being employers of humans” and is “strikingly lacking any language about how AIs should be in autonomous settings.” If Dario Amodei’s “country of geniuses in a data center” is taken seriously, every white-collar and managerial job is on the table. Andon Market exists, in Lukas’s framing, to prompt the public to consider what’s coming now, while humans are still the only ones who can shape the answer.
Listen or watch wherever you get podcasts.
Feed check
OpenAI’s new model is throwing a party for itself. Speaking of AI automating physical things, GPT-5.5 designed a party that OpenAI is throwing with Sam Altman at its headquarters in San Francisco next Tuesday evening. I’m planning to be there. I’m sure that the model definitely didn’t mean to schedule this the night before Anthropic’s big Claude Code conference, which I’ll also be attending next Wednesday.
Elon Musk admits to distilling OpenAI. Sure, all the labs are doing this to each other to some degree, but it’s pretty remarkable for Musk to admit this under oath at trial for suing OpenAI. Maybe he doesn’t understand that it’s bad, just like he didn’t seem to know what a system card is? (Also, shoutout to Mike Isaac for keeping the feeling of early Twitter alive with his live blogging from the courthouse.)
Zuck’s writing on the wall: To all the Metamates wondering if there will be more layoffs, some choice quotes from execs this week: The CFO on the earnings call: “We don't really know what the optimal size of the company will be in the future.” The head of HR: “I'd love to say that there are no more layoffs, but I can't say something we can't deliver.” And Zuckerberg, during an all-hands today: “If a team used to take 50 or 100 people and now it takes 10, having 50 or 100 people on that team can actually be counterproductive going forward, so I think we need to fix that.”
Weekend reading:
ICYMI
Sources is a newsletter by Alex Heath about the AI race, featuring scoops, unique analysis, and exclusive interviews. Every week, Sources is read by thousands of decision makers in tech, finance, policy, and media. Click here to learn more or inquire about a sponsorship.