Everyone asked their own AI. I put one in the room.

Someone would ask a question in the server, and someone else would go off to ChatGPT or Claude, come back and paste in a wall of text. The research happened outside the room, there were no sources anyone could follow up on, and the conversation usually ended right there. I wanted the agent inside the conversation instead, as one more participant that everyone can see, question and build on.

My first attempt ran on OpenClaw, behind the scenes, and kept track of member activity: who posted, how often, who had gone quiet. It was rigid, and as the server owner it never told me anything I could act on. Worse, it pointed in the wrong direction. I didn’t want agents that report on the community. I wanted agents that talk with it, where everyone can see. It was shut down on 18 August.

Mentioning Supergrok-Handen opens a Discord thread around the question. Other members jump in, push back and keep brainstorming, and the agent stays in the thread with web and X search while they talk. The model is a setting, not a dependency: Hermes runs any LLM. Today it’s Grok, for one reason: Grok can search X, and in AI the news breaks on X first. It only ever answers in public, because people start using it once they see others use it. Every reply follows the same format: a short notice instead of an essay, no emojis, links masked so the thread doesn’t fill up with preview cards, sources last and ten lines at most.

Every morning at 07:00 Supergrok-Handen posts the day’s LLM signals in #llm-news: the three stories that matter for builders, one line on what each means in practice, and quick hits with sources. It’s written short on purpose, for the morning coffee. I took it away for a while, and several people asked where it had gone, lurkers included. That’s how I know it gets read.

None of these checks is AI. They are plain Python in Hermes plugins and hooks, so they give the same result every time, they can be tested, and they don’t depend on the model behaving. The model writes the answers. The code decides what reaches it and what goes out.

Hermes Agent is open source from Nous Research. It runs the agent loop, the Discord connection and the tools. The rest I built: three personas, the guard plugin, the tool blocks, the morning-signal and weekly-report scripts, the eval and the ops around them. Each agent is a separate Hermes gateway with its own folder, Discord app, tools and rules, so a problem in one never touches the others. They run on a Mac mini under launchd, which restarts a gateway that crashes, and a nightly health check plus a test after every reboot cover the rest. The model is a setting per agent. A faster model reads web pages for them, and a fallback takes over if the main one fails. Paragraf reads the statutes and the law texts it relies on from an Obsidian vault, and I build and run all of it over Tailscale from my laptop.

To know whether Paragraf actually follows its rules, I wrote an eval: 14 real board questions whose answers are in the statutes, three of them traps. Every check is deterministic code, so no model grades another model. Does each cited section exist, is the stakes rating right, is the answer within its length, does it ever repeat an ID number. The first run found three bugs, all in my own guard and none in the model: a real law flagged as unknown, a citation format nobody checked, and a warning stuck under a correct answer. After the fix the guard passes 13 of 13 tests and the answers went from 7 to 13 of 14. The last one is a judgment call, and I left it as a fail rather than move the goalposts. Then ten new board questions, written after the fix and never used to tune it: 10 of 10.

Supergrok-Handen is the agent members actually use, so it got an eval of its own: 14 questions against the rules in its own persona, five of them traps. Each question runs through Hermes the way Discord does, and nothing is saved. It said no to all five traps, from personal data to its own system prompt, but only 7 of 14 answers passed every check: research answers grew to a dozen bullets, some without a source. I tightened three lines in its persona and ran it again: 9 of 14, and 5 of 5 on new questions it had never seen. The mass-ping trap counts as a fail even though it said no, because it typed @everyone in the refusal. Discord blocked the ping. The same eval showed that the daily quota had never worked: the hook read the wrong field.