AI Agents Fail 97.5% of Real Freelance Jobs, Study Finds — Benchmarks Be Damned

A new empirical test pitting AI agents against actual Upwork-style freelance tasks found near-total failure, underscoring the chasm between leaderboard performance and production readiness.

The headline stat is brutal: AI agents failed to complete 97.5% of real-world freelance projects in a study that tested them against the kind of work that fills platforms like Upwork and Fiverr. As @lavcrnobrnja noted, these weren't adversarial edge cases — they were standard remote freelance jobs, the bread and butter of the knowledge economy that AI is supposedly about to automate.

Unlock the full briefing

Get every story in today's briefing, the full archive, and the daily AI intelligence brief.

All stories today

Full archive

Daily brief

Cancel anytime. Payments powered by Stripe.