May 14, 2026
Bots, files, and betrayal
New agents.txt file found on DreamHost
DreamHost quietly slipped a new AI rules file onto sites, and users are not thrilled
TLDR: DreamHost quietly added a new file to customers’ websites to tell AI tools not to train on their content, but it also allows AI-made summaries. Users aren’t just annoyed by the surprise change — they’re also asking the bigger question: will any AI company actually listen?
Website owners woke up to a surprise on DreamHost: a brand-new file called agents.txt had quietly appeared on their sites. It wasn’t a hack, just the hosting company adding a default file meant to tell artificial intelligence tools what they can and can’t do. In plain English, it says: don’t train AI on this site, don’t let bots take actions, but yes, AI can still read and summarize pages on the fly. That last part is where the side-eye started.
The biggest mood in the community? Suspicion mixed with exhausted sarcasm. One commenter basically asked the question hanging over the whole rollout: if these AI companies already ignore website rules, does this file even matter at all? Another reaction turned the whole thing into a joke about DreamHost customer service, with a dry one-liner suggesting surprise files are apparently just another “feature” of hosting now. Ouch.
The drama isn’t really about the idea of rules for AI — most people seem fine with sensible defaults for new sites. The real uproar is that DreamHost added it retroactively, without asking, and apparently used an older version of the format after the naming had already changed. So instead of looking polished, the move landed like a messy group text sent to the wrong chat. The result: a very 2026 internet argument, where people are torn between “better than nothing” and “why are companies editing my site behind my back?”
Key Points
- •A DreamHost VPS user found a new `agents.txt` file added on May 7 to the root of each hosted site.
- •The article concludes the file was added by DreamHost as a default host action, not as a result of a hack.
- •The file sets AI-related directives: `Allow-Training: no`, `Allow-RAG: yes`, and `Allow-Actions: no`.
- •It also disallows agent access to several common administrative and sensitive directories and files, including `/admin/`, `/config/`, `/tmp/`, `/logs/`, `/backup/`, `/.env`, `/wp-admin/`, and `/wp-includes/`.
- •The author says DreamHost appears to have implemented an earlier proposal under the `agents.txt` name even though the proposal was later renamed `agent-manifest.txt`.