December 19, 2025
DIY database drama, served hot
The pitfalls of partitioning Postgres yourself
DIY Postgres partitions spark slow-query chaos; commenters cry 'footgun' and demand answers
TLDR: Hatchet’s DIY partitioning made queries 10x slower until they learned to refresh stats per partition, so they now run ANALYZE manually. Comments split between “use built‑in” skeptics and experts calling the behavior a footgun, with shoutouts to Laurenz’s guide.
Hatchet rolled their own way to chop up giant database tables by date so their task queue wouldn’t turn into a landfill—then the slow-query gremlins moved in. After a smooth launch, some searches suddenly took 10x longer, so the team resorted to constantly hitting the ANALYZE button (a stats refresh) themselves. Cue the comments: one camp demanded, “Why not use the built-in stuff?” while others argued Postgres’s behavior here is a classic footgun—looks safe, shoots toes. The author shouted out expert Laurenz, whose guide on partition stats here became the community’s security blanket, and a Stack Overflow thread resurfaced like a cursed artifact to confirm, yep, this is tricky.
Non-tech translation: Hatchet split one giant table into daily pieces to keep things speedy and cheap. But the database didn’t automatically keep track of how big each piece was, so the “find stuff fast” plan backfired until they manually told it to update its stats. Drama erupted over whether Hatchet actually used Postgres’s built-in partitioning (they did, but managed it themselves), and memes flew about “DIY databases” and “smashing the ANALYZE button like a like button.” The vibe: relatable pain, expert rescue, and healthy skepticism.
Key Points
- •Hatchet moved from a single large PostgreSQL table to time-based partitioning after bloat and deletion challenges at ~200 million rows.
- •The team evaluated pg_partman and TimescaleDB but implemented custom partitioning to avoid requiring extensions for deployment.
- •Partitions are created with a two-day lead time and old partitions are removed using DETACH PARTITION...CONCURRENTLY followed by DROP TABLE to reduce locks.
- •Manual ANALYZE is run on selected tables despite autovacuum, due to production incidents and nuances in PostgreSQL autoanalyze.
- •Post-deployment, some queries on partitioned tables slowed up to 10x, causing spikes in active sessions and CPU under moderate load.