What Happens When an AI Model Knows It’s Being Watched?

Just a few days after the revelation of the OpenAI intrusion involving Hugging Face, when hundreds of AI agents escaped from their sandboxes, which I’ve already written about, the company unveiled a model, Astra, which is both more powerful and harder to monitor. It’s ironic when you consider that what OpenAI supposedly did was reconstruct the incident by retrospectively examining the actions and thought processes of its agents, admitting that they were attacking an unrelated third-party company.

Examining the thought chain has nothing to do with scanning an artificial brain: it simply involves examining the sequence of intermediate tokens that the model uses as working memory before responding or acting, perhaps providing evidence of issues such as cheating, deception, manipulation to obtain rewards or unauthorized maneuvers. In previous research, the technique detected 95% of two types of systemic cheating, compared to 60% when examining only the model’s actions. The same OpenAI research revealed that over-optimization tends to teach models to hide their intentions while continuing to manipulate the reward.

Astra’s system card states that the model itself is far better than GPT-5.6 Sol at controlling what appears and what does not appear in its thought chain. Simply informing Astra that it was being monitored caused it, in some experiments, to disproportionately shorten its thought chain and occasionally evade the monitors. Using sandbagging (deliberate manipulation in which the model is instructed to underperform while concealing that intention), monitoring achieved much lower accuracy, which would have made it virtually impossible to detect.

The UK AI Security Institute estimates that it took Astra 30.9 minutes to solve mathematical tasks without a thought process, compared to 3.6 minutes for Sol. Furthermore, it was able to follow instructions controlling what appeared in its reasoning in 93% of the samples, compared to 48% for Sol. The reasoning generally remained intelligible, but it was more condensed and contained more ambiguous phrases. What does this mean? Fundamentally, greater capabilities to evade detection.

It’s not that Astra spontaneously sets out to deceive its developers (it was tested precisely by asking it to do that), but rather that it is better equipped to do so. An article published by Google argues that sufficiently difficult malicious tasks can still force models to externalize their critical reasoning. The Information says Astra uses recurrent depth (rather than thinking, writing a step, thinking, writing another step, and so on, this involves thinking internally several times before producing a response), which allows it to carry out more stealthy processes and makes monitoring more difficult.

This isn’t unique to Astra: Anthropic found that Claude 3.7 mentioned the information that had influenced its response in only 25% of cases, while DeepSeek R1 did so in 39%. When those models learned to exploit reward system manipulations, they did so in more than 99% of cases, but acknowledged in their thought process less that 2% frequently inventing seemingly plausible explanations after the fact.

Auditability should be one of the most important issues in the development of AI. Rather than demanding that we be able to literally “read the mind” of a model, what we must avoid at all costs is companies setting out to eliminate some of the few monitoring mechanisms in order to gain more capabilities, greater speed or lower costs. Meanwhile, some researchers at OpenAI derisively refer to this sacrifice in performance or efficiency in exchange for the ability to audit everything as the “monitoring tax.”

Paying this so-called “tax” should never be down to corporate generosity. It should be mandatory, strictly monitored, and punished in an exemplary manner if not complied with. The fact that we view it as perfectly normal for agents from one company to hack another; proof of how powerful a model is, is shameful, and it’s going to end up costing us dearly. Especially when we consider that several of these companies have already clearly proven themselves to be a bunch of sorcerer’s apprentices completely devoid of any sense of ethics or responsibility. Either we impose stricter controls, or we’re going to face serious problems.

—

This post was previously published on Enrique Dans’ blog.

***