July 12, 2026
Bot swap, comment war
Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper
AI website maker ditches its old fave, but commenters are already fighting about the results
TLDR: Ploy switched its website-making AI to GPT-5.6 after seeing faster speeds and lower costs, a big deal for a tool that builds real business pages. But commenters instantly split into camps: some praised the useful write-up, others mocked the AI-sounding prose, and one bluntly said the old model’s designs looked better anyway.
Ploy says it pulled off a big AI swap: its website-building bot now runs on GPT-5.6 instead of Claude Opus, and the company claims the new setup is 2.2 times faster and 27% cheaper while still matching or beating the old one on finished pages. In plain English, the bot helps create marketing websites by planning pages, writing code, making images, checking its own work, and deciding when it’s done. Ploy says GPT-5.6 was the first challenger in months to finally knock Claude off the throne.
But the real entertainment is in the peanut gallery. One camp was genuinely impressed, with one commenter basically giving the post a gold star for being a rare useful, practical AI write-up. Another camp immediately smelled corporate robot perfume: one reader mocked the article’s polished phrasing as obvious LLM-style writing, zeroing in on lines like “Numbers like that buy a model a real migration effort” as peak AI-speak. Ouch.
And then came the taste-war. One commenter threw the biggest punch: who cares if it’s cheaper if the pages look worse? For marketing sites, they argued, the only thing that matters is what actual humans prefer and what gets more clicks or sales — and they personally liked Claude’s examples better. Meanwhile, another commenter sidestepped the whole GPT-vs-Claude cage match to brag that their own setup makes requests “practically free,” which is classic comment-section energy: nice benchmark, but have you considered my superior stack?
Key Points
- •Ploy migrated its production website-building agent from Claude Opus 4.8 to OpenAI’s GPT-5.6 Sol after internal head-to-head evaluations.
- •Ploy says GPT-5.6 Sol delivered homepage redesign results that were 2.2x faster, 27% cheaper, and slightly higher in visual score in the sample shown.
- •The company’s agent performs end-to-end website tasks including planning pages, reading codebases, writing components, generating imagery, taking screenshots, and deciding completion.
- •Ploy found that about one-third of initial cross-model failures came from evaluation harness assumptions, such as tool-call budgets, lack of batched file-read support, and a hidden `minScore` threshold.
- •The migration required changes beyond model swapping, including fixes to the eval harness, tool schemas, caching behavior, and reasoning replay handling.