October 29, 2025
Grading bots like reality TV
Show HN: Automate Robot Data Quality Improvement
Robots get report cards: score the demos, skip the junk, show the gains
TLDR: A new tool grades robot training videos and filters out bad ones to boost learning. Early reaction: smart idea, but the community wants proof it improves results—and they’re wary of AI-powered grading and over-filtering that could delete valuable weird cases.
A new open-source toolkit just dropped to grade robot training videos like a fussy teacher, and the early vibe is cautiously hyped. It watches for blurry footage, jerky moves, and crash-y moments, then gives each episode a 0–1 score. There’s even an optional check using Google’s Gemini (an image-and-text AI) to judge if the robot actually did the task. One early commenter called it a “pragmatic” fix for messy data, and the crowd’s collective eyebrow is up: great idea, but does it actually make robots smarter?
That’s the headline drama: builders love the promise of auto-cleaning bad demos; skeptics want receipts. Expect demands for hard before/after charts showing training gains—baseline vs filtered. Also brewing: side-eye over depending on Gemini’s API limits and costs, and whether filtering out “weird” episodes might erase valuable edge cases. The vibe is very “please, no more vibe-based data”—score it, sort it, ship it.
Humor isn’t far behind either. With features reading like a robot report card—“visual clarity,” “smoothness,” “collision spikes”—folks joked about robotic Gordon Ramsay yelling “BLURRY!” at shaky clips. The repo is here for brave souls to try it: score_lerobot_episodes. Bottom line: the community’s ready to be convinced—show the gains, or this grade gets a redo.
Key Points
- •Toolkit scores robot demonstration episodes across multiple dimensions and aggregates scores from 0–1.
- •Combines computer vision heuristics (blur/exposure, smoothness, collisions) with optional Gemini VLM checks for task success.
- •Supports filtering low-quality episodes to improve downstream training and enables visualizing score distributions.
- •Provides CLI with arguments for dataset source (Hugging Face), output paths, thresholds, vision method (OpenCV or Gemini), and training options.
- •Installation requires Python 3.8+, pip, GitHub clone, and optional Google API key for Gemini due to rate limits.