Vision-Language Models Can't Track a Ball Under a Cup — VET-Bench Exposes Fundamental Spatial Reasoning Gaps
A new benchmark based on the shell game finds state-of-the-art VLMs performing at random chance (33%) on object tracking tasks, revealing that video 'understanding' remains largely illusory.
Researchers behind VET-Bench designed a deceptively simple test for vision-language models: track an object through a shell game. As @HuggingPapers reported, current state-of-the-art models perform at 33% accuracy — statistically indistinguishable from random guessing among three cups.
Unlock the full briefing
Get every story in today's briefing, the full archive, and the daily AI intelligence brief.
All stories today
Full archive
Daily brief
Cancel anytime. Payments powered by Stripe.