CryptoSlate
Anthropic's Fable 5.2 leads Scale's Remote Labor Index with 41.3% of its freelance deliverables judged to meet a professional reference standard, the highest score on the Nov. 12 leaderboard. The next-highest model, Opus 5.5, sits at 21.25%. Anthropic reported the result in its Nov. 9 release announcement, labeling the score "High effort" with safeguards enabled.
The RLI, developed by researchers from the Center for AI Safety and Scale AI, tests complete freelance projects rather than isolated questions. Its dataset spans 240 projects across 23 categories, from documents and audio to video, 3D models and CAD files. Trained human reviewers compare each AI deliverable against both the client brief and a professional human-reference product, and a project passes only when the work clears that reference standard. A useful prototype or partial result can still fail, which makes 41.3% a demanding measure of end-to-end capability.
Why it matters
Fable 5.2's lead is a real jump in practical AI capability, more than doubling the closest competitor on a benchmark built around finished, inspectable work. But the same number means the model fails 58% of evaluated projects, and the study itself excludes work that requires client interaction, teamwork, or physical labor. Anthropic's release page also does not specify per-project spending or the amount of human support involved, leaving the cost side of the equation open.
Market impact
For anyone pricing AI against freelance labor markets, RLI offers the strictest yardstick so far: an assignment carried through to a product a reasonable client would accept. A 41.3% pass rate is far from a replacement threshold, yet the pace of improvement on a leaderboard that only months ago was measuring models at far lower levels suggests the gap between prototype-grade and client-grade AI output is closing fast. Watch whether rival labs publish their own RLI scores and whether Anthropic discloses execution costs in future runs.
Frequently asked questions
-
What is the Remote Labor Index (RLI)?
A benchmark from Center for AI Safety and Scale AI researchers that tests AI on complete freelance projects across 23 categories, from documents to 3D models. A project passes only when human reviewers judge the deliverable as good as a professional reference standard.
-
How did Anthropic's Fable 5.2 score on the RLI?
Fable 5.2 scored 41.3%, ranking first on Scale's Nov. 12 leaderboard. Anthropic labeled the result "High effort" with safeguards enabled, ahead of the next-highest model, Opus 5.5, at 21.25%.
-
What does the 41.3% score actually mean?
It means 41.3% of evaluated freelance projects produced deliverables that met a professional acceptance standard, while the remaining 58% failed. Useful prototypes or partial results count as failures because they fall short of client-acceptable, finished work.
-
What work does the RLI exclude?
The study excludes projects that inherently require client interaction, teamwork, or physical labor, as well as projects whose outcomes must be awaited before evaluation. Those exclusions limit how broadly the results apply to the wider remote-work economy.
-
What details did Anthropic leave out about its RLI run?
The release page specifies "High effort" but does not establish the precise execution setup, spending per project, or the amount of human support involved, leaving the cost side of the capability picture unresolved.
Crypto News
WatcherGuru
CoinTelegraph
WuBlockchain
Altcoin Daily
TheBlock