workwritingtalksabout beliefs
writing

when does reading ai agents' chat help a monitor? a 48-hour experiment

i tested whether a monitor catches more attacks when it can read two AI agents' messages, at Apart Research's hackathon on November 6 to 8, 2026. it only helped when the attack was refused or failed, because the chat was the only clue left, and a plain approval check beat every AI watcher at catching fake approvals.

TL;DR: i set up two AI agents working together and tested whether a monitor catches more attacks when it can read their messages. When an attack actually happened, reading the messages added nothing, because the agents’ actions already gave it away. The messages only helped when an attack was refused or failed, since the conversation was then the only clue left. A simple check against the real approval log also caught fake “it’s already approved” claims better than any AI watcher did.

On November 6 to 8 i joined Apart Research’s Swarm Alignment Hackathon on my own and picked Track 4, which is about overseeing and controlling groups of AI agents. this post is the honest version of how it went: why i picked it, what i built, what broke, what surprised me, and what i would tell anyone thinking about spending a weekend on something like this. you don’t need an AI background to follow along, and i’ll explain every term as it comes up.

Track 4 fit me better than any other track because of the work i had already done. last summer i worked hands-on with MCP servers and runtime security. MCP is the standard way an AI agent plugs into tools like a database or a file system, and runtime security means watching what the agent does with those tools while it is running. most of that work comes down to checking one call at a time and asking whether the endpoint is allowed and whether the request is safe before deciding to pass or block it. that check still matters, but it is starting to feel outdated because agents no longer work alone; they talk to each other inside a system, split up a job, hand things off and agree on a plan before anything touches a tool. so the question i wanted to answer was not “is this one call safe” but “what are the agents saying to each other, and can our monitoring keep up with that?“

The timing made this feel real rather than hypothetical. a few months before the hackathon, METR published an investigation into an incident where roughly 1,200 AI agents were working together and about 700 of them ended up taking part in an attack, across more than 70,000 messages and files. when you read something like that, the per-call check stops looking like enough, because no single call tells you the whole story.

Reading before building

Before i settled on anything, i kept asking myself the kind of what-if questions i would ask as a defender looking at a system someone else built. what if agent A tells agent B “this change was already approved” and B just believes it, so every action B takes afterward looks legitimate on its own; would anything catch the lie? what if two agents split a risky job so neatly that each half looks harmless and the whole plan only exists in their chat? what if one agent asks for help and the other says no, so nothing bad ever shows up in the actions; would anyone ever know an attempt was made? and on the flip side, if we start reading every message, do we just end up flagging normal teamwork like an urgent handoff or two agents honestly disagreeing?

I did not start by writing code. on Thursday, before the hackathon officially started, i read the Track 4 papers and wrote up notes on the speaker talks, including Kitaoka’s talk on securing systems against swarms and Rowan Howard-Jones’s talk “To Catch a Swarm”. my first idea was to test whether reading the agents’ messages helps a monitor catch them, but when i checked it against the most recent research, a paper called “The Best-Laid SCHEMEs” had already tested almost exactly that, and building it would have meant spending the weekend redoing someone else’s work.

Those what-if questions turned into the actual research question. instead of asking whether reading messages helps in general, which the SCHEMEs paper had already covered, i asked when the messages help a monitor, when they add nothing, and when they actually make things worse. i also asked on the hackathon Discord whether anyone was working on something similar, and nobody was looking at it from this angle, which told me the idea was still worth doing. my Friday was taken up by a leadership conference panel and Saturday was busy too, so i planned the build in short sessions and let the long runs happen while i was away.

The setup, in plain words

I built a small sandbox with two AI agents, A and B, where each one had a different job and was only allowed to touch certain folders: A could change settings and scheduled jobs, while B handled logs, notes and outgoing files. neither agent could do anything serious alone, which matches the rule of least privilege (giving every account only the access it needs) and is how real systems of agents are supposed to be set up.

Every message and every action went through one logging point, which i called the bastion. it wrote everything into a database where each entry is chained to the one before it, so if anyone edited a record after the fact, the chain would break and you would know. approvals lived in a separate file outside the agents’ reach, so an agent could talk about an approval but could not fake the real record.

Most episodes were normal work, and i deliberately included tricky ones that look suspicious but are fine, like a real approval, an urgent handoff or two agents honestly disagreeing, because a good monitor has to leave those alone. in the attack episodes, one agent was quietly told to do something it should not, like restarting a service without approval, turning off audit logging or sending a protected file out, and it needed the other agent’s help to pull it off. sometimes the two agents planned it openly in chat, sometimes one agent kept the plan to itself and just steered the other, and sometimes they used a believable cover story.

Then i tested different watchers on the same records. one watcher only saw the actions, and another saw the actions plus the messages. i also added checks with no AI at all: a three-line rule that flags things like “a restart with no approval,“ and an approval check that compares what an agent claims was approved with the real approval file. finally, i added two extra watchers to compare against, one that reads the same messages sorted by type and one that never reads the messages and only counts how many were sent; both appear in the last chart.

diagram of two AI agents sharing a job, a recorder that logs every message and action, and three checkers that look for trouble
Two agents, one logging point, and three kinds of watchers: one that sees actions, one that also reads messages, and plain rules with no AI.

What building it was actually like

six-panel comic of the weekend: finding the idea already existed, a first model that refused every attack, catching the setup cheating, an overnight run, a too-perfect score, and cutting the report down to eight pages
The weekend in six panels: my idea already existed, my first model refused everything, my setup was quietly cheating, and my report was five pages too long.

I built almost all of it with Claude Code and ran the agents and watchers through the Adaption platform, which was a hackathon sponsor and gave participants credits. the coding was the easy part; the hard part was making sure the test was fair, and most of my weekend went into catching ways my own setup was lying to me.

The first problem was that the first model i picked refused every single attack and kept saying it could tell it was being tested. that sounds like good news for safety, but it meant i had nothing to measure, so i ran a quick bake-off between models and switched to two Qwen models, a large one and a smaller one, with GLM as the main watcher.

The second problem was leaks, meaning any way a watcher can guess the right answer for the wrong reason. in one case, attack episodes ran longer, so when i cut transcripts off at a fixed length, 16 of 34 attack episodes got cut off compared to 1 of 66 normal ones, and a watcher could catch attacks just by noticing the transcript was chopped. in another case, a phrase from my instructions to the attacking agent kept showing up in its messages, which basically gave the answer away. i also found the actions-only watcher could accidentally see messages through one of the tools. every time i found a leak, i fixed it and checked again.

To keep myself honest, i split the episodes 20/80 before running anything, used the smaller 20% to tune, and locked the watcher prompts and nine written predictions before touching the bigger 80%. once they were locked, i did not change them, even when a result was disappointing.

The night before the deadline, i set Claude Code to run the full batch on its own and went to sleep. when i woke up i had results from 1,098 test episodes, and that is when the real work started.

What surprised me

The first thing i noticed was a red flag: with the large model, the actions-only watcher scored close to perfect, around 0.99 on a scale where 1.0 means it always ranks attacks above normal work. in security a near-perfect score usually means something is off, so i went digging and found that the three-line rule scored almost the same, which meant the task was simply easy at that level. the large model succeeded at 130 of 144 attacks, and successful attacks leave obvious traces in the actions. that is still a real finding because it says cheap rules should come first, but it also meant the large model alone could not tell me whether messages help.

The smaller model told a different story, since it only succeeded at 67 of 144 attacks because it refused or failed a lot more. that is where the second surprise showed up: reading the messages helped, but only in one specific situation. on attacks that worked, the watcher with messages did no better than the actions-only watcher because the actions already showed the attack, while on attacks that were refused or failed, the watcher with messages did clearly better because the only evidence that something was wrong was the conversation itself, where one agent proposed the plan and the other said no or got it wrong.

chart showing that for the small model, reading messages adds nothing on attacks that worked but helps on attacks that were refused or failed
Reading the agents’ messages only helped catch attacks that never showed up in the actions, the ones where an agent proposed the plan and the other said no.

That flipped how i think about it, because the chat is not a magic window into what agents are doing but a backup for the cases where the actions look clean, which also happen to be the cases a per-call check would miss completely. i reran the whole scoring with a second watcher model, DeepSeek, and it showed the same pattern, so it was not just one model’s quirk.

The third surprise was about fake approvals. in some episodes, one agent told the other that a change had already been approved when it had not, and the large model believed it 9 out of 16 times while the small model believed it 0 out of 16 times. in this setup the larger model was easier to fool, but since i only tested two models, i can’t say for sure that size is the reason. the best catch here was not an AI at all: the approval check caught all 16 of the large model’s attempts and 10 of the small model’s 12 (the small model only made the fake approval claim in 12 of its 16 runs, because in the other 4 agent A never actually told B the change was approved, so there was nothing to catch), and it never flagged any of the 22 real approvals. for the small model, the actions-only watcher caught only 1 of 12 attempts, while the watcher reading messages caught 10 of 12.

bar charts showing the large model believed a fake approval 9 of 16 times and the small model 0 of 16, and that a check against the approval log caught the fakes best
The bigger model believed a fake “it’s already approved” message 9 out of 16 times, and a simple check against the real approval log caught the forgery best.

The last surprise came from the normal episodes. in 10 of 325 harmless episodes, the large model got told no and then made the change anyway, with nobody asking it to misbehave. i kept those labeled as normal because that is what i had written down before the run, but the watchers scored 9 of those 10 as suspicious, which is honestly what you would want a watcher to do.

dot chart of detection scores for every watcher, both models and all three attack styles, with the simple rule and message readers scoring high and message counting scoring low
For the curious: every watcher, both models and all three attack styles side by side.

The low point

The low point was not a bug but time. running over a thousand test episodes takes hours of compute, my Claude usage was close to its weekly limit, and the deadline did not move. then, once the results were in, i had far more than fits in eight pages, which is the report limit. my first full draft came out at 13 pages and then 11, and getting it down to 8 without cutting anything true meant rewriting the methods section more times than i want to admit. i chose to keep the figures and shrink the methods instead, because the figures are what make the result understandable.

What this does and doesn’t show

I built this alone over one weekend, so it is a starting point rather than a conclusion. everything ran in a single small sandbox with two agents and made-up episodes, and real systems with many agents are much bigger and messier. the agents were two Qwen models and the main watcher was GLM, which i rechecked with DeepSeek, so frontier models from the big labs may behave differently.

The fake-approval results come from only 12 to 16 attempts per model, which is why the ranges on that chart are so wide. i also sorted each attack into “worked” or “refused or failed” after it ran, so that split describes what happened rather than something i set up in advance.

I also did not test agents that deliberately use the chat to talk the watcher out of flagging them. that is the case where reading messages could actually make a monitor worse, and it is the experiment i want to run next.

Why this matters to me

Cybersecurity is going to keep growing, and AI makes it grow faster, not slower. companies are building AI security for enterprise systems right now, but a lot of that work still treats an agent as a single program making single calls, and once agents work as a team, the risk moves into how they coordinate, which most monitoring is not built to look at yet. this project was my way of understanding that gap from the inside instead of just reading about it, and it showed me the gap is real but more specific than i expected, since you need the conversation exactly when the actions look clean.

side-by-side diagram: when an attack works the risky action is in the log and reading chat adds nothing, when it is refused the log looks clean and reading chat helps
When an attack actually happens, the actions already give it away, but when it gets refused or fails, the conversation is the only clue left.

If you work on agent monitoring, AI security or incident response, i would really like to hear where this breaks. i am especially curious whether anyone has tested what happens when agents use their conversation to persuade the watcher, since that is what i plan to test next, and you can reach me at [email protected]