SQLite improving performance with pre-sort

SQLite gets a surprise speed glow-up, but the comments want receipts

TLDR: Sorting random data before inserting it into SQLite made writes about 2 to 2.5 times faster in the test. Commenters were split between impressed applause, benchmark skepticism, and a warning that one table setting can backfire badly with bigger data.

A tiny database trick just turned into a full-on comment section showdown. The big reveal: if you take a giant batch of totally random IDs—basically scrambled digital labels—and sort them before saving them, SQLite suddenly looks a lot less sluggish. The benchmark in the post shows insert times dropping from roughly 11 seconds to about 4.3 seconds at 10 million rows, which is a pretty dramatic makeover for a change that sounds almost suspiciously simple.

But the real fun starts in the replies. One camp was instantly impressed, with the internet equivalent of a polite standing ovation: "Neat optimization" and a genuine "wait, why haven’t I seen this before?" vibe. Another commenter came in with the classic skeptical energy every benchmark post eventually summons: are we sure the speedup is from sorting, or is this secretly just a batching win wearing a fake mustache? In other words, the community wants receipts, not just vibes.

Then came the practical war story that gave the whole thing extra spice. One reader warned that a specific table setting, WITHOUT ROWID, can actually wreck performance for large records, based on their own painful experience. That turned the discussion from "cool trick!" into "cool trick, but don’t copy-paste this blindly." Even the author’s aside about writing offline because search engines are bad now added a little indie-hacker comedy to the mood. The result? A wholesome-but-chaotic thread where curiosity, caution, and low-key database trauma all got equal screen time.

Key Points

  • The article benchmarks SQLite inserts using 160-bit random `BLOB` primary keys generated with `SecureRandom` in a `WITHOUT ROWID` table.
  • Inserting unordered random keys in batches of 1,000,000 produced reported times from 4,927 ms to 11,103 ms as the table grew from 2 million to 10 million rows.
  • The post attributes slower performance to ordered B+ tree structures, where random inserts cause page thrashing, page splits, and rebalancing.
  • The proposed optimization is to sort each batch before insertion using an unsigned comparison of the first 8 bytes of each 20-byte ID.
  • With pre-sorting, reported insert times improved to 2,846 ms to 4,332 ms per batch, which the article says is about 2x to 2.5x faster despite sorting overhead.

Hottest takes

"Neat optimization, I've never seen that before" — linolevan
"Could the performance improvement be mostly explained by batches?" — gandreani
"WITHOUT ROWID really tanked performance" — Loranubi
Made with <3 by @siedrix and @shesho from CDMX. Powered by Forge&Hive.