Scale wants to make the Oscars for AI

Scale's head of research: “Evaluation is falling behind the development of model capabilities."

This year’s Scale model awards that are being sent out to the AI labs.

Scale AI is toying with holding an actual Oscars-style ceremony for AI models next year, complete with a live audience.

For now, the company announced its first-ever “Models of the Year” awards on Tuesday, based on a year’s worth of evaluations from its Safety, Evaluations, and Alignment [SEAL] research team. Models released after December 5th missed the cutoff, which notably excludes OpenAI’s GPT-5.2.

I spoke with Bing Liu, Scale’s head of research, about why the company is doing this and where he sees AI evaluation heading. The short version: the industry needs a trusted, independent authority on model performance, and Scale is betting it can be that resource.