March 8, 2026
They put a dev in a .md file
We Turned Our Wireshark Wizard into a Markdown File
Dev team turns a grumpy network nerd into an AI sidekick — and the internet has feelings
TLDR: Checkly launched “Rocky AI,” a tool that uses an engineer’s documented troubleshooting skills to automatically explain why tests and network checks fail. The community is split between praising it as finally-useful AI and mocking it as “copy-pasting a human expert into a file” that might one day replace them.
Developers at Checkly say they basically downloaded a senior engineer’s brain into a markdown file so their new “Rocky AI” can read giant error logs and tell you what went wrong. The tech crowd lost it. One camp is cheering, calling it “Wireshark wizardry in a can” and saying this is the first AI feature that doesn’t feel like a gimmicky chatbot. The other camp? Screaming “you turned a human expert into a checklist” and joking they’ve just invented the world’s most overqualified intern.
The moment they casually mentioned “we literally codified our Wireshark guy into markdown,” comments exploded. People imagined the poor engineer being CTRL+C, CTRL+V’d into a text file, with memes of brain progress bars, “Senior Engineer.md,” and “Patch Notes: Fixed human.” Others dragged the team for admitting model-swapping was a disaster, with one user calling it “multi-cloud, but for disappointment.” Their jab at a “dreadful GPT-5” sparked its own mini-war between model fanboys.
Meanwhile, non-technical readers loved the idea of skipping painful debugging, while cynics warned this is how you quietly replace specialists with an AI that “kind of knows what it’s doing.” In classic internet fashion, the debate ends where it started: half the crowd asking for access, the other half asking if Rocky pays union dues.
Key Points
- •Checkly has released Rocky AI, an AI-based root cause analysis agent integrated into its SaaS monitoring product after 6–8 months of development.
- •The team refocused from broad AI ideas to a narrow, high-value problem: using LLMs to triage failed Playwright Check tests by analyzing large Playwright trace files.
- •They address large diagnostic artifacts (e.g., 100MB+ Playwright traces, big PCAP files) by pre-parsing, filtering, and summarizing them via tool calls before sending key data to LLMs.
- •Expert troubleshooting skills (such as Wireshark ICMP and PCAP analysis) are codified into semi-structured markdown files that guide the LLM’s root cause analysis across multiple check types (Playwright, HTTP, TCP, DNS, ICMP).
- •Model upgrades (e.g., OpenAI GPT-4.1 to GPT-5.1, Opus and Gemini versions) significantly improved performance, while attempts at Bring-Your-Own-Model support revealed that swapping between OpenAI, Gemini, and Anthropic models is difficult to do while maintaining quality, even with wrapper SDKs.