October 29, 2025
Buckets vs. Buckets of Cash
A Fork in the Road: Deciding Kafka's Diskless Future
Fans split: go full cloud and ditch the disks or play it safe to cut the bill
TLDR: Kafka is deciding whether to push data straight to cloud storage to cut cross‑zone costs, choosing between a bold redesign or safer tweaks. Commenters split between ‘this is years overdue’ and ‘diskless just means S3,’ joking about buckets while debating performance, simplicity, and Kafka’s future identity
Kafka just hit soap-opera mode: three proposals to go ‘diskless’ by writing straight to S3 (aka cloud buckets) to slash cross–data center bills and simplify ops. The comments were the real show. Old hands rolled their eyes — one summed it up as obvious years ago — while others cheered finally tackling the cloud bill monster. The fight boils down to two vibes: revolution (rewrite for cloud-native, elastic everything) vs evolution (bolt S3 on, keep the classic feel, save on network fees).
Fans worried aloud: will Kafka lose its low-latency soul if it becomes bucket-first? Or is clinging to disks like insisting on DVDs in the age of streaming? A reality check landed hard: as one commenter put it, diskless doesn’t mean magic — it just means your disk lives in someone else’s warehouse. Meanwhile, memes flew: ‘just put it in the bucket,’ ‘latency stans vs elasticity gang,’ and ‘cut the cloud bill diet.’
There’s extra spice with one proposal seen as unlikely, fueling whispers of politics and strategy. Stakes feel huge: this decision could set Kafka’s next decade. The crowd’s chorus: make it bold or keep it simple — but for the love of budgets, stop the cross-AZ fees and don’t make ops a nightmare. Grab popcorn, this vote matters
Key Points
- •Apache Kafka is evaluating how to integrate S3-backed topics to cut cross–availability zone replication costs.
- •Three KIPs (KIP-1150, KIP-1176, KIP-1183) address the same challenge, with different design implications.
- •Two strategic paths are presented: a revolutionary direct-to-S3 design versus an evolutionary reuse of existing components.
- •The revolutionary path offers elasticity and simpler operations but requires maintaining two topic models and higher implementation effort.
- •KIP-1183 is described as non-viable, proposing AutoMQ’s storage abstractions without implementation, and the community is urged to choose a long-term direction.