March 18, 2026
Who tests the robot IQ testers?
Measuring progress toward AGI: A cognitive framework
$200k “Robot IQ Test” contest launches — commenters ask: who decides what’s smart
TLDR: A new “cognitive” framework and $200k Kaggle contest aims to build AI report cards for skills like learning and social savvy. The crowd’s split: fans cheer real metrics, skeptics ask who gets to define “smart” and whether tests measure true understanding or just fancy pattern tricks.
AI just got a report card — and the internet immediately argued over the grading curve. A new paper drops a 10-part “mind map” for machines and a Kaggle hackathon with $200k in prizes to build the tests, targeting tricky stuff like attention, learning, social savvy, and even metacognition (aka “knowing what you know”). Submissions run now through April 16, winners June 1.
But the comments? Spicy. One camp is hyped that we’re finally measuring progress toward Artificial General Intelligence (think: human-level smarts across tasks). The other camp is side-eyeing hard. “Two guys from Google get to set the rules?” asks one skeptic, demanding a way to score wisdom and exceptions, not just trivia — dropping a YouTube rant for flair. Another compares the whole thing to rebranding community challenges as “AGI benchmarks,” quipping it’s like when we named an index “GDP” and called it science.
Meanwhile, jokesters dubbed it the “Robot SATs” and “Kaggle Hunger Games,” while pragmatists cheer: if models beat human baselines on clear tasks, that’s real progress. The core drama: can you test understanding, or just pattern-matching? And if an average person might flunk these hoops, do they measure “general” intelligence — or just make machines look smart?
Key Points
- •A new paper introduces a cognitive taxonomy to evaluate AI systems’ general intelligence using ten cognitive abilities.
- •The framework proposes a three-stage protocol: task-based AI evaluation, representative human baselines, and mapping AI performance to human distributions.
- •A Kaggle hackathon launches to crowdsource evaluations for five gap areas: learning, metacognition, attention, executive functions, and social cognition.
- •Participants can build and test benchmarks using Kaggle’s Community Benchmarks platform against frontier models.
- •The hackathon offers $200,000 in prizes, with submissions open March 17–April 16 and results announced June 1.