On November 9, 2026, Scale Labs added Anthropic’s new Claude Fable 5.2 model to the Remote Labor Index, a benchmark built from real, paid freelance projects. Fable 5.2 completed 41.3% of the projects in the index to a standard that expert graders judged a paying client would have accepted. When the index launched at the end of October 2025, the best AI agent managed 2.5%.
AI agents are AI systems that work on their own: they open files, use software and hand in finished work. The index is run by the Center for AI Safety (CAIS), a nonprofit research group, and Scale Labs, part of the company Scale AI. On the leaderboard, Fable 5.2 nearly doubles the runner-up, Anthropic’s Opus 5.5 (21.25%), and more than doubles its predecessor, Fable 5.1 (17.92%).
Anthropic released Fable 5.2 the same day (our coverage of the Fable 5.2 launch). Its launch post lists the same 41.3% in its benchmark table, measured at the model’s “High” effort setting.
Real projects, judged the way a client would
The authors of the paper that introduced the index paid 358 freelancers with verified accounts on Upwork, a freelance marketplace, to sell existing samples of their work. Of the 550 projects collected, 240 passed review, spanning 23 categories of work such as game development, product design, architecture, data analysis and video animation.
Each project comes with the client’s brief, the input files and the freelancer’s finished work. At the median, a project took 11.5 hours and was worth $200, based on the prices the freelancers reported. The biggest jobs pull the averages up to 28.9 hours and $632.60: the most expensive cost more than $10,000, and the longest took more than 100 hours. The projects total $143,991.
A trained human evaluator opens the AI’s files next to the freelancer’s and decides whether the AI’s version “would be accepted by a reasonable client as the commissioned work.” Each project gets a 1 (fails), a 2 (meets the standard) or a 3 (exceeds it). The headline “automation rate” is the share of projects scored 2 or 3.
Three evaluators review each deliverable, and the majority vote counts. Scale Labs reports 94.4% agreement among them. The leaderboard scores agents on 230 projects kept private, so future models are less likely to have seen them in training.
CAIS has not managed to automate the grading. In a July 1 post, it reported that an AI grader rated OpenAI’s GPT-5.5 about 2.9 times too high and Anthropic’s Opus 4.8 about 2.3 times too high.
To judge a deliverable, a grader has to open it in tools such as the 3D program Blender or CAD (computer-aided design) software. Using those tools properly is the skill current agents handle worst. One GPT-5.5 bathroom “render” had been faked with an image generator, which only a look at the actual 3D model revealed.
From 2.5% to 41.3% in about a year
At launch, the paper concluded that “AI agents perform near the floor.” The best of them, the agent Manus, completed 2.5% of the projects. “The current automation rate is less than 3%,” Dan Hendrycks, a senior author of the paper, wrote on X on October 29, 2025.
On July 1, 2026, CAIS reported 15.8% for Fable 5, an earlier model in the same family. Opus 4.8 reached 8.3% and GPT-5.5 reached 6.3%. The previous record, 4.17%, belonged to Anthropic’s Opus 4.6 running in Claude Cowork, an Anthropic agent environment.
CAIS wrote that the best score had “more than quadrupled in under eight months.” Four months later, Fable 5.2 reached 41.3%, more than 16 times the launch score.
By our count, the latest step is also the largest: Fable 5.2 leads the runner-up, Opus 5.5, by 20 percentage points. All earlier gains in the record, from 2.5% at launch to Opus 5.5’s 21.25%, added up to 18.75 points.
CAIS’s index site puts the pace at one doubling “every ~3.3 months” over the past 12 months. Our own arithmetic gives a similar figure. Going from 2.5% to 41.3% takes a little more than four doublings. Spread over the 375 days between October 30, 2025, and November 9, 2026, that is about one every three months.
Where Fable 5.2 improved and where agents fail
Scale Labs said on X on November 9 that Fable 5.2’s biggest gains came in audio and music, architecture and interior design, and 3D modeling. The model also made “significant progress on long, complex projects,” according to the same thread. Nearly 50 of the projects Fable 5.2 completed had never been completed by an earlier agent. Industrial product design and animation “remain far from solved,” Scale Labs wrote.
The launch paper’s analysis shows how agents failed at the time, and one deliverable could fail in several ways. Of the failed deliverables, 45.6% were of poor, “child-like” or amateur quality, and 35.7% were incomplete, such as an 8-second video for a brief that asked for 8 minutes. In 17.6%, files were corrupt, empty or wrongly formatted. In 14.8%, the work contradicted itself, such as a house whose design changed between 3D views.
In July, CAIS also found that the time a project took a human did not predict whether an agent would succeed. Some quick tasks for a professional, such as transcribing music or playtesting a real-time game, stayed out of reach. Digital art or coding that takes a person hours was done in minutes.
What the 41.3% leaves out
The 41.3% is a share of the projects in the index, which covers only part of remote work. It leaves out work that needs the client in the loop, such as tutoring, and team jobs, such as project management. It also excludes physical work and work judged only after a long wait, such as search engine optimization (SEO). The paper warns that an AI scoring 100% “may still underperform humans on types of work that we do not evaluate.”
Each model gets a single score. “There is no confidence interval for Automation Rate,” the leaderboard states, so readers get no margin of error. Nor does it say how often a success repeats when an agent tries the same project again.
The test setup also changed during the year. At launch, agents ran in standard setups with a budget cap of $30 per project. Since July, CAIS has run each model inside the strongest coding agent for its family, such as Claude Code for Anthropic models and Codex CLI for OpenAI models. These are the coding agents developers use day to day.
CAIS added a tool to operate graphical programs. Each project gets a Linux desktop with more than 30 professional applications, up to 24 hours and a GPU (graphics chip) when needed. A second agent critiques the work before it is handed in. The default budget is $50 per project, raised to $150 for Fable 5 because of its higher prices.
Neither the leaderboard nor Anthropic’s post gives Fable 5.2’s budget. CAIS described the July results as “paired with stronger agent scaffolding,” the software that lets a model act. The gains since 2025 therefore mix better models with a better setup, and the published scores do not separate the two.
The AI Futures Project, a group that forecasts AI progress, cites the index as a sign that AI’s other skills seem to lag behind its research skills. Its November 10 forecast update draws on METR’s audit of AI-driven research at Anthropic. METR is a nonprofit that evaluates AI models.
The group also quotes Anthropic’s system card, the safety and capabilities report Anthropic published with the new models. In it, Anthropic writes that Mythos 5.2, which runs on the same underlying model as Fable 5.2, “does not seem close to being able to substitute for our Research Scientists and Research Engineers.”
What it means
The Remote Labor Index measures something closer to paid work than most AI benchmarks. People judge whole projects, with the files a client would receive, against what a professional delivered. On that measure, the best agent went from almost nothing to about four projects in ten in a year.
The result applies to well-specified projects, each given to the agent with a brief and input files and graded once. It leaves out client calls, rounds of revisions and teamwork, and it does not measure employment.
In the labor data, US employers cited AI for 1,265 job cuts in October, the lowest monthly figure of 2026, down from 38,579 in May. At Anthropic, the company’s own study found that juniors fell from 27% to 9% of technical hires.
The public record still lacks three things: what each run cost, how often a success repeats on a second attempt, and how much of the rise since 2025 comes from the July setup change.
CAIS says it will keep adding major model releases to the index. Its blog carried a detailed write-up with examples for Fable 5 in July but had none for Fable 5.2 as of November 11. The next entries will show whether the pace of one doubling every three months holds as agents run into the categories Scale Labs calls far from solved, industrial product design and animation.
Sources
- Remote Labor Index: Measuring AI Automation of Remote Work, Mantas Mazeika, Alice Gatti, Cristina Menghini et al. (Center for AI Safety, Scale AI), arXiv, October 30, 2025
- Remote Labor Index (RLI), Scale Labs leaderboard, retrieved November 11, 2026
- @AnthropicAI’s Fable 5.2 takes the top spot on our Remote Labor Index with @CAIS, Scale Labs on X, November 9, 2026
- A Significant Increase in Digital Labor Automation, Mantas Mazeika, Center for AI Safety, July 1, 2026
- Remote Labor Index, Center for AI Safety and Scale AI, retrieved November 11, 2026
- Introducing Claude Fable 5.2 and Claude Mythos 5.2, Anthropic, November 9, 2026
- Q3.5 Timelines Update: Looks Like the Intelligence Explosion Will Happen in 2027 After All, AI Futures Project, November 10, 2026
- Can AI automate jobs? We created the Remote Labor Index, Dan Hendrycks on X, October 29, 2025
Header image: chart by MACCUS, data from Mazeika et al. (Center for AI Safety, Scale AI), arXiv 2510.26787, October 30, 2025; Center for AI Safety, July 1, 2026; Scale Labs, November 9, 2026
