May 16, 2026
Hessian Impossible: Rewrite Protocol
After 8 years, I rewrote my open-source PyTorch curvature library
The code comeback has people cheering, squinting, and arguing over who this rewrite is really for
TLDR: A developer rebuilt an eight-year-old open-source AI research tool so it works better on today’s larger models without eating huge amounts of memory. The community reaction is a mix of applause for the comeback, curiosity from researchers, and low-key dread from anyone worried the new version might break old setups.
An old-school AI tool just got a very 2025 glow-up, and the vibe in the community is a mix of respect, relief, and nerdy side-eye. pytorch-hessian-eigenthings, first released back in 2018, has been rewritten after eight years to help researchers peek inside how machine-learning models behave while training. In plain English: it gives developers a way to inspect a model’s "shape of learning" without blowing up memory, which is exactly the sort of thing that makes deep-learning people nod gravely and say, “finally.”
The loudest reaction is simple: people love a maintainer who comes back. Open-source users treated the rewrite like a beloved cult project getting a surprise reboot, with extra applause because the new version works on modern big-name model stacks like Hugging Face. But there’s also delicious little drama: the old 0.x setup is gone, and anyone depending on it is being told to pin an older version. Translation for non-coders: yes, some people are thrilled, and some are already bracing for “my workflow broke” posts.
The funniest running joke is the library name itself, which sounds like either a prog-rock album or a wizard spell. Even the creator, Noah Golmant, leaned into the niche nature of the project, calling it a specialized tool for model curvature analysis. That honesty won points. The hot take energy? This is either a heroic resurrection of a beloved research utility, or the most elegant possible way to restart ancient arguments about whether “flat” AI models are actually better at all.
Key Points
- •The article announces `pytorch-hessian-eigenthings` v1.0.0a1, a rewritten alpha release with the old 0.x API removed.
- •The library computes eigendecompositions of the Hessian and related curvature matrices using iterative methods that rely on Hessian-vector products, avoiding the quadratic memory cost of forming the full Hessian.
- •Supported algorithms include Lanczos, stochastic power iteration, Hutch++ trace estimation, and Stochastic Lanczos Quadrature for spectral density.
- •The package supports multiple interchangeable operators, including Hessian, Generalized Gauss-Newton, and empirical Fisher, and can be used with PyTorch models including HuggingFace and TransformerLens transformers.
- •For large-vocabulary language-model workloads, the article describes a fused cross-entropy Hessian-vector kernel using Triton on CUDA or `torch.compile`, with reported speedups and lower peak memory use.