Simulacrum of Knowledge Work

Shiny reports, messy reality: AI writes, AI reviews, who’s accountable

TLDR: Article says AI makes work that looks right but isn’t, pushing offices into Goodhart’s law—chasing the metric, not truth. Comments split: doomers see “AI writing for AI,” skeptics say check the real work, a few tout verifiable training as a fix.

An essay claims we’re living in a “simulacrum” of work: shiny reports, shaky reality. The comments erupted. One camp nodded hard, with balamatom dropping the line of the day: “We’ve automated ourselves into Goodhart’s law—when you game the yardstick, the yardstick stops meaning anything. Their twist: progress isn’t dead; it’s just happening in ways today’s internet culture can’t read. Translation: the work might be better under the hood, even if the gloss looks too perfect. Cue dread of bosses skimming “Looks Good To Me” while chatbots crank out confident copy.

Others pushed back. rowanG077 called the premise lazy: you can check real quality if you bother; judging by typos is just bad management. firefoxd painted chaos: AI writes, AI reads, AI replies, and when a real customer screams, nobody knows where it broke—office telephone turned into a factory. mrtesthah offered hope: train models with verifiable rewards for right answers (math you can check) instead of applause. And bensyverson reminded everyone humans long shipped pretty-yet-empty work. The meme of the thread: “robots grading robot homework” as everyone chases the token leaderboard.

Key Points

  • Knowledge work is frequently evaluated using surface-level proxies (e.g., formatting, typos, labeling) because verifying substantive quality is costly.
  • LLMs can produce outputs that look highly polished, weakening the reliability of traditional proxy measures for quality.
  • In software engineering, AI-generated code and AI-driven code reviews can uphold process rituals without ensuring actual robustness.
  • LLM training methods, including likelihood-based objectives and RLHF, optimize for plausible or pleasing outputs rather than truth or utility.
  • These dynamics create an automated Goodhart’s law scenario, where optimizing for appearance leads to a simulacrum of work and reduced substantive scrutiny.

Hottest takes

"We’ve automated ourselves into Goodhart’s law." — balamatom
"Everybody’s output is someone else’s input." — firefoxd
"If you accepted work purely based on superficial proxy measures you were not fairly evaluating work at all." — rowanG077
Made with <3 by @siedrix and @shesho from CDMX. Powered by Forge&Hive.