Designing DB partitions you don't have to babysit

Stop making your database need a nanny, says internet as the comments spiral

TLDR: The article’s main point is simple: don’t organize a big table by date if it forces every app query to remember that date forever; use the main record ID and automate the upkeep instead. Commenters instantly turned it into a tech showdown, with some cheering the idea and others bragging that different database tools make this whole problem look silly.

This database post started with a painfully relatable workplace horror story: a team split up its giant orders table by date, then discovered even simple “find order #12345” lookups got slower because the app didn’t include the magic date filter. Suddenly the fix wasn’t really a fix at all — it was a new lifestyle. Every query had to remember the date, every code review became a hall monitor, and the database design quietly turned into a rule humans had to babysit forever. The author’s big pitch? Partition by the order’s main ID instead, and let a background service quietly adjust the boundaries based on real growth so developers don’t have to keep feeding the beast by hand.

But the real fireworks were in the comments, where readers immediately split into camps. One side basically said, “Cute idea, but why are you still doing this the hard way?” with drive-by flexes about kdb+ handling a billion rows a day and ClickHouse crunching 500+ billion rows daily on a cheap server. Subtle? Absolutely not. Another crowd got philosophical, questioning whether “ID” should even be the main thing a database is organized around if users never search by it. And then came the practical nerds waving Snowflake IDs and uuid7 like they were alternate-universe cheat codes. The vibe was classic tech-thread chaos: one solid article, three different religions, and at least two comments that felt like benchmark-shaped bragging rights.

Key Points

  • The article says partitioning an OLTP table by `created_at` can cause queries that filter only by `id` to scan every partition instead of pruning to one.
  • The article describes how adding date filters to application queries can restore partition pruning but causes the partition key to leak into application code and review processes.
  • According to the article, PostgreSQL and MySQL require the partition key to be included in primary keys and unique constraints on partitioned tables.
  • The article states that changing from `PRIMARY KEY (id)` to `PRIMARY KEY (id, created_at)` means the database no longer enforces uniqueness on `id` alone.
  • The recommended design is to partition by the primary key and let a background service adjust partition boundaries based on observed growth.

Hottest takes

"1bn rows a day is par for the course" — leprechaun1066
"querying over 500+ billion rows daily without any issues" — djfobbz
"Why for example is the id a good primary key?" — yasaheblasa
Made with <3 by @siedrix and @shesho from CDMX. Powered by Forge&Hive.