April 22, 2026
Two bits and a thousand hot takes
Approximating Hyperbolic Tangent
The Great Curve-Off: Speed Freaks vs Accuracy Police
TLDR: A survey shows faster shortcuts for the tanh S‑curve used in AI and audio, swapping small errors for big speed. Comments erupted into a speed‑vs‑accuracy showdown—calls for clearer basics and benchmarks, bold claims of better curves, and crafty hardware tricks that squeeze results from just a few bits.
A simple post about making the tanh “S‑curve” faster—handy in AI and music effects—sparked a full‑blown speed vs. accuracy scuffle. The author tours quick‑and‑dirty shortcuts (polynomials, fractions of polynomials, and Lego‑style piecewise curves), plus cheeky tricks that lean on how computers store numbers to get big speed with “good enough” accuracy.
Cue the comments: one reader huffed that the piece buried the basics—“start with a definition!”—while link‑droppers swooped in with receipts. A commenter pointed to an analysis and upgrade of a famous fast‑exp trick at typ.dev, basically saying, “if you make exp() faster, tanh gets faster too.” Then came the plot twist: a well‑known audio dev claimed his square‑root‑flavored “sigmoid” (another S‑curve) tuned with a small polynomial likely beats these quick hacks on worst‑case error—translation: “my curve’s cleaner when things get ugly.” Benchmarks? Not in the thread, but the gauntlet felt thrown.
And the hardware heads brought the spice: one suggested a two‑bit lookup table hustle—peek at a couple of bits inside the number, hit a tiny table, then blend—making exp (and therefore tanh) cheaper in silicon. The vibe: pragmatists chanting “fast enough for real‑time,” purists demanding error bounds and proofs, and jokesters quipping “two bits to rule them all” while someone inevitably typed “Padé? I barely knew her.”
Key Points
- •The article surveys fast tanh approximations for performance-critical tasks in ML and audio processing.
- •A Taylor-series polynomial (up to x^11) with saturation beyond |x| > 1.365 provides a simple, fast approximation.
- •A [7/6] Padé approximant adapted from JUCE improves accuracy, recommended for inputs within [-5, 5], with saturation outside.
- •A spline method (three cubic segments over [0, 18]) from Simos and Tsitouras prioritizes speed with acceptable error.
- •The survey notes IEEE‑754 floating-point tricks as an additional class of fast approximation techniques.