December 9, 2025
3n+1 hot takes, 0 chill
Transformers know more than they can tell: Learning the Collatz sequence
AI aces a weird number game — but commenters call the title clickbait and ask why they stopped short
TLDR: Researchers trained transformers to predict steps in the Collatz number game, hitting near-perfect accuracy in some number systems but failing when “counting loops.” The comments roast the vague title, demand ELI5s and architecture fixes, and revive old theory debates about neural nets’ limits
An AI study claims transformers can predict big jumps in the Collatz sequence (the internet’s favorite “take a number: if even, halve it; if odd, triple it + 1” loop). The twist: performance depends on how you write the numbers. In some number systems it’s almost perfect, in others it falls apart. The authors say the models mostly get the math right but mess up the counting of loops, not the calculation — and almost never “hallucinate.” But the comment section? On fire. The top vibe was “ELI5, please,” with readers begging for a plain-English breakdown while the Title Police questioned what, exactly, these models “know but can’t tell.”
Then came the spicy takes: one reader blasted the paper for stopping “at the most interesting part” — if “base conversion” is the bottleneck, why not fix the architecture to handle it better? Another pulled the emergency brake with an old-theory throwback: what about proofs that neural nets can’t model arbitrarily long sine waves? Meanwhile, interpretability fans cheered the forensic analysis of failures. The community split into two camps: “show me the tweak that makes this work” vs. “understand it first.” Cue memes about “loop counters,” “number bases,” and “3n+1 hot takes” outpacing the math itself
Key Points
- •The study evaluates transformer accuracy on predicting long Collatz steps, finding performance depends strongly on numeral base encoding.
- •Accuracy reaches about 99.7% in bases 24 and 32, but drops to roughly 37% and 25% in bases 11 and 3.
- •Models learn classes of inputs sharing the same residual modulo 2^p, achieving near-perfect accuracy on these and under 1% elsewhere.
- •The learning pattern aligns with a property that loop lengths in the Collatz computation can be inferred from the input’s binary representation.
- •Most errors involve correct arithmetic with misestimated loop lengths; hallucination is rare, occurring in under 10% of failures.