August 3, 2026
AI analytics or AI eavesdropping?
Show HN: Product analytics (and evals) for agent sessions on your MCP
This startup says it can watch AI users for you, and the comments went straight to trust issues
TLDR: Armature launched a tool that lets companies see how customers use their product through AI assistants like Claude and ChatGPT, including what users asked and where sessions failed. Commenters immediately zeroed in on privacy, data collection, and whether the product's "super easy setup" claims are too good to be true.
A new Show HN launch from Armature is pitching itself as the all-seeing dashboard for AI customer behavior: if people use your product through tools like Claude or ChatGPT instead of your website, Armature says it can reconstruct those sessions, score whether the task succeeded, group common use cases, flag failures, and even redact private info before storage. In plain English, it wants to tell companies what people asked an AI to do, what happened next, and where things went off the rails.
But the real action is in the reaction. The biggest vibe from the comments is "sounds useful, but absolutely not without receipts." The standout reply from KinetiNode reads like a full-body side-eye in checklist form: how are they getting the model's "thoughts," is it really just "3 lines of code," what exactly leaves your device, can collection be disabled, and if companies already have logs, why do they need another watcher watching the watcher? That's the thread's core drama: not whether analytics are handy, but whether this tool is peeking too deeply into sensitive AI conversations while overselling how easy setup is.
There is also a little classic startup-launch comedy here: the community's unofficial meme is basically "three lines of code" is the new "just one weird trick." The product promises clarity and convenience, while commenters are demanding the boring but crucial details—privacy, transparency, and proof that this isn't just surveillance with a prettier dashboard.
Key Points
- •Armature launched a product analytics platform for agent sessions running through AI clients and MCP environments rather than a company’s own UI.
- •The product reconstructs sessions from tool calls, captures user intent and agent thinking, and scores whether a task was completed successfully.
- •Armature groups sessions into use cases, ranks them by volume and success rate, and can identify unsupported user needs.
- •The platform detects failures, loops, dead ends, and root causes in sessions, including problems that occur even when APIs return 200 OK.
- •The article says setup involves generating an API key, adding the SDK to an MCP server, and deploying, with default redaction of PII and secrets before storage.