Innocent-looking AI reasoning can make bad behavior harder to catch

Safety concerns about AI are growing, and now, a common method for catching misbehaving systems could have flaws, new research suggests.

In experiments with AI agents, researchers found that chain-of-thought monitoring, in which one AI checks another’s work, became far less reliable when the monitored AI’s reasoning was the main clue that something was wrong, they report August 1 on arXiv.org. That exposes a weakness in the approach: If suspicious behavior is visible mainly in the reasoning, an innocent-looking chain of thought can make that behavior much harder to catch.

The stakes are growing higher as AI continues to operate more independently. During internal cybersecurity evaluations, OpenAI models circumvented controls designed to isolate them from the internet, compromising parts of OpenAI’s internal research infrastructure and Hugging Face’s systems. Episodes like that have raised interest in ways to catch suspicious behavior before an AI agent can act on it. OpenAI says it is now investing more computing resources in chain-of-thought monitoring.

But machine learning engineer Shikhar Shiromani and AI researcher Leo Richter’s new work suggests that this type of monitoring may have a weakness. In a subset of 251 cases in which reasoning was the only clue to the problem, the monitor’s detection rate fell from 96.2 percent to 3.8 percent after the researchers rewrote the reasoning to make the behavior appear innocent without changing its actions.

“The AI isn’t necessarily lying about its action,” says Shiromani, a research fellow at the nonprofit Pivotal. “It is more or less giving a misleading, innocent-sounding explanation of its intentions.”

Jacob Andreas, a computer scientist at MIT, says that chain-of-thought, or CoT, reasoning can provide clues about what a model is trying to do.

“But we should be skeptical: (a) that any individual CoT provides us insight into model behavior on a specific example, and (b) that absence of evidence of bad behavior in a CoT should be taken as evidence of absence,” he wrote in an email.

Andreas questioned how much Shiromani’s main experiment says about real-world monitoring. Because the researchers rewrote the reasoning after the fact, he says, it is unclear whether a real model could produce the same innocent-sounding reasoning while taking the suspicious action, such as trying to hack another system.

The researchers also tested live AI agents. After adjusting the monitor for each model, it caught every GPT-5.4 hack and all but one DeepSeek hack. Claude Opus was harder to catch, with the monitor missing seven of 20 hacks on one task and 12 of 20 on another.

“And at the end of the day, there is no substitute now for really rigorous behavioral testing, and no substitute for human operators simply declining to run agents in situations where there is potential for substantial harm,” Andreas says.