Show HN: Automate Robot Data Quality Improvement

Robots get report cards: score the demos, skip the junk, show the gains

TLDR: A new tool grades robot training videos and filters out bad ones to boost learning. Early reaction: smart idea, but the community wants proof it improves results—and they’re wary of AI-powered grading and over-filtering that could delete valuable weird cases.

A new open-source toolkit just dropped to grade robot training videos like a fussy teacher, and the early vibe is cautiously hyped. It watches for blurry footage, jerky moves, and crash-y moments, then gives each episode a 0–1 score. There’s even an optional check using Google’s Gemini (an image-and-text AI) to judge if the robot actually did the task. One early commenter called it a “pragmatic” fix for messy data, and the crowd’s collective eyebrow is up: great idea, but does it actually make robots smarter?

That’s the headline drama: builders love the promise of auto-cleaning bad demos; skeptics want receipts. Expect demands for hard before/after charts showing training gains—baseline vs filtered. Also brewing: side-eye over depending on Gemini’s API limits and costs, and whether filtering out “weird” episodes might erase valuable edge cases. The vibe is very “please, no more vibe-based data”—score it, sort it, ship it.

Humor isn’t far behind either. With features reading like a robot report card—“visual clarity,” “smoothness,” “collision spikes”—folks joked about robotic Gordon Ramsay yelling “BLURRY!” at shaky clips. The repo is here for brave souls to try it: score_lerobot_episodes. Bottom line: the community’s ready to be convinced—show the gains, or this grade gets a redo.

Key Points

  • Toolkit scores robot demonstration episodes across multiple dimensions and aggregates scores from 0–1.
  • Combines computer vision heuristics (blur/exposure, smoothness, collisions) with optional Gemini VLM checks for task success.
  • Supports filtering low-quality episodes to improve downstream training and enables visualizing score distributions.
  • Provides CLI with arguments for dataset source (Hugging Face), output paths, thresholds, vision method (OpenCV or Gemini), and training options.
  • Installation requires Python 3.8+, pip, GitHub clone, and optional Google API key for Gemini due to rate limits.

Hottest takes

“pragmatic way to tackle the data quality problem in robotics” — marshavoidance
Made with <3 by @siedrix and @shesho from CDMX. Powered by Forge&Hive.