August 8, 2026
Rust never sleeps, and neither do haters
Building a Rust Inference Engine That Matches Llama.cpp
Rust remake of a beloved AI tool sparks hype, shade, and instant trust issues
TLDR: Ferrox is a new from-scratch tool for running AI models locally, and its creator says it can keep up with the famous llama.cpp in speed tests. The comments immediately split between impressed Rust fans and critics who mocked AI-assisted coding and warned that trendy new projects can be hard to trust.
A developer just rolled out Ferrox, a brand-new AI model runner built entirely in Rust, a programming language with a near-religious fan club. On paper, this is impressive stuff: it runs open-source chat models on laptops, works with Apple chips, Nvidia graphics cards, and regular processors, and in some tests it nearly matched — and occasionally beat — the hugely respected llama.cpp, one of the biggest names in local AI. For fans, that alone was enough to pop the confetti.
But the comments? Pure tech-forum reality TV. One camp was delighted, basically saying, finally, a Rust version of this thing, and asking the big future question: will it stay a copy of llama.cpp, or grow into something weirder and more ambitious? Another camp was instantly suspicious, with one blunt jab reducing the entire project to: "written in Rust™ (thanks to Claude)" — a not-so-subtle swipe at AI-assisted coding and the internet’s current obsession with shipping first and asking questions later.
Then came the full doomposting. One commenter declared that "the entire internet is becoming slop", while another warned that "vibe coded" projects are basically security risks in a trench coat. So while the article itself is a victory lap about proving speed with receipts, the real community drama is about trust: is this a serious fresh alternative, or just another flashy AI-era clone? In classic internet fashion, Ferrox launched straight into a food fight over hype, quality, and whether writing it in Rust is genius, branding, or both.
Key Points
- •Ferrox is a pure-Rust local LLM inference engine built from scratch without bindings to llama.cpp or ggml.
- •It supports GGUF models and can run dense and Mixture-of-Experts models on CPU, Apple Metal, and CUDA.
- •The project provides both a CLI with llama.cpp-compatible flags and a server exposing an OpenAI chat-completions API-compatible interface.
- •The article reports benchmark methodology designed for reproducibility, including same-machine comparisons, warm runs, multiple repetitions, and JSON-backed result receipts.
- •Reported results show near parity with llama.cpp on Llama-3.1-8B and higher throughput on several smaller models, which the article attributes to fused dequantization and architecture-specific GPU kernels.