January 26, 2026
One kernel to rule them all
Compiling models to megakernels
One giant GPU job: fans cheer, skeptics say it’s old tricks
TLDR: Luminal wants to fuse an AI model’s whole step into one huge GPU job to cut waiting and boost speed. The crowd is split: some say it’s just old-school compiler “inlining,” others think it’s a real win for squeezing more out of pricey hardware—without buying more GPUs.
Luminal wants to squash an AI model’s entire “think about this” step into one megakernel—basically, one massive GPU job—to stop the hardware from sitting around waiting for the next task. Fewer pauses, less starting and stopping, and more non-stop number crunching. They’re riffing on ideas like Hazy Research’s hand-built mega-kernel for Llama, but aiming to automate it so you don’t need a wizard writing GPU spells by hand. The pitch is simple: cut the idle time, load the next weights early, and keep every GPU core busy like a well-oiled pit crew.
The comment section, of course, turned the hype into a comedy club. The top snark: isn’t this just “inlining”—aka gluing functions together—dressed up in AI clothes? One user joked it’s “compiler 101” masquerading as a moonshot. Performance nerds still popped confetti, cheering anything that keeps GPUs from twiddling their thumbs. Practical folks asked the awkward question: if you cram everything into one mega-task, does debugging become a horror movie? Memes landed fast—“one kernel to rule them all,” “GPU babysitters,” and “speedrun your forward pass.” Love it or roast it, the vibe is clear: Luminal promises fewer stalls and more speed, the crowd argues whether that’s genius or just finally reading the manual.
Key Points
- •Luminal identifies three bottlenecks in GPU inference: kernel launch overhead, wave quantization across SMs, and initial weight-loading stalls.
- •CUDA Graphs reduce kernel launch overhead (e.g., dummy kernel from 2.1 µs to 1.3 µs) but do not eliminate it.
- •Wave quantization causes some SMs to idle when work cannot be evenly distributed across the GPU.
- •Weight-loading at the start of each kernel leaves tensor cores idle, and device-level techniques like Programmatic Dependent Launch only partially mitigate this.
- •Fusing an entire forward pass into a single megakernel can remove launch latency, mitigate wave quantization by redistributing work per SM, and overlap weight prefetching with computation.