Incident March 30th, 2026 – Accidental CDN Caching

Tiny bug, big side‑eye: users question mixed messages and missing details

TLDR: Railway accidentally cached pages for some sites that had caching turned off, potentially showing responses to the wrong people for 52 minutes before purging and adding safeguards. The crowd’s split: praise for transparency vs. heat over confusing wording, “vanity” stats, and demands for hard numbers and clearer root cause explanations

For 52 wild minutes (10:42–11:34 UTC), Railway flipped the wrong switch and its “off” setting for a CDN — a speed‑up network that stores pages closer to users — briefly turned “on” for a sliver of sites. Translation: some page responses from simple “view” requests might have been shown to the wrong person. Railway purged the cache, apologized, called it a “trust boundary violation,” and promised more tests and slower, staggered rollouts.

But the comments? Absolute chaos. One camp is fuming about confusing language: how can “authenticated” responses be cached without cookies, and why does the blog read like a riddle? Another camp actually credits the team for being upfront, but wants receipts. A top critique blasts the post for skipping the juicy root cause — enabling “Surrogate Keys” — which the status page reportedly explains, while the blog dances around it. And that neat “0.05% of domains” line? The crowd’s calling it a “vanity metric,” demanding the real stat that matters: how many cross‑user responses were mis‑served.

Elsewhere, armchair architects pitched fixes (like unique URLs per session), reliability folks asked what QA missed, and jokesters dropped memes: “Oops, all cache,” “Cache me outside,” and “Surrogate Keys unlocked New Game+.” One commenter even wondered if a separate outage (Stripe) was connected — not proven, but the rumor mill is working overtime.

Key Points

  • A CDN configuration update on March 30, 2026 (10:42–11:34 UTC) unintentionally enabled caching on ~0.05% of Railway domains with CDN disabled.
  • Cached HTTP GET responses may have been served to users other than the original requester during the 52-minute window.
  • Cache-Control headers were honored and Set-Cookie responses were not cached, but most GET responses without explicit cache headers were cached by default.
  • The change was reverted at 11:34 UTC and all cached assets were purged globally; affected users will be notified by email.
  • Railway implemented added caching behavior tests and more aggressive sharding of CDN rollouts, and labeled the issue a trust boundary violation.

Hottest takes

"This write up doesn’t make sense" — stingraycharles
"\"0.05% of domains\" is a vanity metric" — varun_chopra
"Good that they are transparent about it" — sebmellen
Made with <3 by @siedrix and @shesho from CDMX. Powered by Forge&Hive.