Billion-Parameter Theories

Napkin science vs black‑box beasts — and the comments are on fire

TLDR: The essay says simple laws won’t cut it for tangled problems like climate and poverty, so giant AI-sized “theories” may be our new tools. Comments split between simplicity purists, pragmatic big‑model fans, and a surprise Chomsky grudge match—making this a must‑watch fight over how we explain the world.

Sean Linehan’s new essay argues our neat “F=ma on a napkin” era is over for messy, real‑world stuff like poverty and climate. The gist: simple laws tamed the merely complicated, but truly complex problems may need billion‑parameter “theories” (read: giant AIs). The crowd? Divided and dramatic. One camp cheers the shift, saying big models give theories more “reach,” even if that word feels squishy. Another camp claps back hard: user js8 insists small theories still rule, using climate as a poster child—yes to huge simulations, but simple models can explain the basics without a data center.

Then the plot thickens. harperlee jumps in with a peace‑treaty twist: even giant models might secretly be governed by a few key directions—“find the right perspective and compress the beast.” Meanwhile, us‑merul throws shade at ivory‑tower perfectionism: the more we chase the “correct” solution, the easier it is to dismiss real people and their messy lives. And in the spiciest subplot, curuinor lights up a classic language‑theory feud, saying connectionist AI folks are “explicitly pissed off” at Noam Chomsky—cue the popcorn.

The memes write themselves: “From napkin to server rack,” “F=ma vs FOMO,” and “press X to compress.” Whether you want elegant equations or billion‑knob oracles, this thread has your weekend debate covered.

Key Points

  • Historical scientific success often relied on terse, elegant theories that were operable within human cognitive limits.
  • The article distinguishes complicated (decomposable) systems from complex systems with dynamic interactions and emergent behavior.
  • Examples of complex domains include poverty, climate change, health-related behaviors, ecosystems, and financial markets.
  • The Santa Fe Institute’s interdisciplinary work identified descriptive patterns in complex systems but struggled to produce prescriptive tools.
  • A linguistics parallel suggests universal principles can be correct yet non-operational, with practical modeling (e.g., language modeling) providing usable capabilities.

Hottest takes

"it is always possible to come up with reasonably small theories" — js8
"Even for billion-parameter theories, a small amount of vectors might dominate the behaviour" — harperlee
"explicitly pissed off about Chomsky's assertion" — curuinor
Made with <3 by @siedrix and @shesho from CDMX. Powered by Forge&Hive.