March 20, 2026
Random? Or just random drama
Randomization in Controlled Experiments
Readers cry foul: “Where’s the placebo?” as critics poke holes in the ‘gold standard’ advice
TLDR: The article champions randomization as the key to trustworthy experiments, but commenters roast the examples, insisting human trials need placebos and tighter design. Why it matters: sloppy “random” tests can mislead big decisions, and the crowd wants clearer guidance—and a statistician—before calling anything gold standard
Terence Kelly’s ode to randomized experiments set out to crown randomization the gold standard and warn that sloppy methods can turn science into junk. But the comment section lit up like a Vegas marquee. One camp shot back that the opening example—TV-watching turkeys vs. no-TV turkeys—sounds fine for birds, but for humans it’s classic “A vs A+B”: one group gets extra attention, the other gets nothing. Cue bias. As one commenter put it, even a chat with someone in a white coat can lift outcomes—so use a placebo or it’s not a fair fight.
Then came the academic clapback. Another reader slammed the piece, saying the ACM should’ve run it by a statistician, calling out “bad experimental design” and suggesting the author misunderstands how to assign people to groups. The thread split into Team “Randomization Saves Science” vs. Team “This Needs Nuance,” with memes about turkeys bingeing nature docs and jokes about lab coats being performance-enhancing gear. Behind the snark is a serious point: everyone agrees randomization matters, but commenters say the article glossed over real-world traps. Verdict from the crowd? Great reminder, shaky examples—bring placebos, bring rigor, and maybe bring a statistician to the edit table
Key Points
- •Randomization errors in controlled experiments can invalidate causal conclusions and mislead significance tests.
- •A proper control group is necessary to isolate treatment effects; relying on intervention alone obscures causality.
- •Random assignment prevents confounding from self-selection or experimenter bias, enabling credible treatment-control comparisons.
- •Significance tests of the null hypothesis are valid only when assignment is correctly randomized.
- •Randomized controlled trials are recognized as the gold standard; leading medical journals require meticulous documentation of randomization.