July 12, 2026
Zip drama > tech demo
Autoresearch, Claude and Constrained Optimization
AI tries to invent a better zip file, and the comments instantly turn into a metric war
TLDR: A developer used Claude to try building a better file compressor as a test of whether AI can handle bigger jobs on its own. Commenters were far more interested in roasting the way success was measured, with critics saying the benchmark itself may have made the whole result hard to trust.
A developer set out to test a big, flashy claim: can an artificial intelligence assistant do serious work with only light supervision? Instead of asking it to write a cute app or summarize notes, he gave Claude a simple-sounding challenge with clear rules: shrink files as much as possible, make sure they open perfectly afterward, and don’t let the process run forever. On paper, it’s a neat real-world experiment in letting AI chase a goal without constant hand-holding. In the comments, though, the real sport was dragging the benchmark.
The loudest reaction was basically: nice idea, messy scorecard. One commenter zeroed in on the “objective function” — the way success is measured — and argued that this is where these AI experiments live or die. They even invoked a spicy Mitchell Hashimoto X thread, only for another commenter to swat it down as a “very generous interpretation of that absolute stinker of a tweet.” Ouch.
Then came the compression nerd pile-on. One user bluntly said it was “not the greatest benchmark,” arguing the comparison between common tools may be misleading because default settings can make one program look better or worse unfairly. The extra sting? They pointed out the author previously wrote about avoiding bad metrics, which commenters treated like a deliciously ironic own goal. The vibe was half serious debate, half internet side-eye: is this a clever AI experiment, or a fancy way to accidentally compare apples to badly zipped oranges?
Key Points
- •The article describes an experiment by Elliot Smith to test whether an AI agent can handle a constrained optimization task with limited supervision.
- •Smith chose file compression as the test problem because success could be measured objectively by output size.
- •The experiment imposed two pass-fail constraints: decompression had to reproduce the original file exactly, and both compression and decompression had to complete within 300 seconds.
- •Smith says the project is intended as a viability test for the approach, not as a benchmark of any specific model or an attempt to build a top-tier compression algorithm.
- •The methodology section states that the code is available on GitHub and that the experiment used Claude Code with default settings on Sonnet 4.6, with an initial project scaffold written in Rust.