July 15, 2026
Basement box, internet brawl
Running Gemma 4 26B at 5 tokens/SEC on a 13-year-old Xeon with no GPU
A basement relic just ran a giant AI, and the comments instantly turned it into a flex war
TLDR: A hobbyist got a huge modern AI model running at reading speed on a 13-year-old server with no graphics card, proving old hardware still has surprising life. Commenters turned it into a mix of amazement, bragging, and jokes after the author revealed the machine first produced smooth-sounding nonsense.
A man dragged a 13-year-old server out of basement retirement, spent under $300, and somehow got it to run a modern Google AI model with no graphics card at all. That alone would be enough internet catnip, but the real party started in the comments, where the reaction split neatly into awe, one-upmanship, and gloriously nerdy chaos. One camp treated it like a sneak peek at the future, with one commenter calling it “a peek into the future” for what old machines might still do. The vibe there was pure techno-optimism: if this junk-box can do this now, what counts as “obsolete” anymore?
Then came the flexers. Almost immediately, someone jumped in with the classic online energy of “actually, mine is faster”, claiming 8–12 tokens per second on another ancient processor. Suddenly the wholesome basement success story became a mini benchmark showdown. Another commenter dropped a whole gist of results, because no internet argument is complete until somebody arrives with receipts.
But the funniest twist came from the author himself, who explained the scariest bug: the machine wasn’t just failing loudly, it was producing “fluent-looking multilingual gibberish” thanks to uninitialized memory. In plain English, the AI was confidently speaking polished nonsense — which, let’s be honest, several readers probably found a little too relatable. The fix is now open upstream, turning this from basement hack into community-approved legend.
Key Points
- •Ryan Findley reports running Google’s Gemma 4 26B-A4B model on a repurposed HP StoreVirtual server with dual 2013 Xeon E5-2690 v2 CPUs, DDR3 memory, and no GPU.
- •The reported performance is about 5.2 tokens per second for decoding and roughly 16 tokens per second for prompt evaluation using a Q8_0 model format.
- •The project was inspired by an earlier post that ran Gemma 4 on a 2016 Xeon using ik_llama.cpp and CPU-focused inference optimizations.
- •The initial build failed because the optimized kernels assumed AVX2 and FMA3 support, while the tested Ivy Bridge CPUs provide only AVX1.
- •The article says Claude helped identify the instruction-set mismatch and complete fallback changes so the inference code could run on pre-AVX2 hardware.