Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks
GLM 5.2 was released on 16 June 2026, and it's currently the most capable open-weight model available: it tops the Artificial Analysis Intelligence Index among open-weight models (51, ahead of DeepSeek v4 Pro's 44), and trails only Claude Fable 5, Claude Opus 4.8 and GPT 5.5 on the index. It also has the second-highest score on Vending-Bench 2, Andon Labs' long-horizon business-simulation benchmark, trailing only Claude Opus 4.7.
This research note reports our early safety and behavioral results on GLM 5.2. Overall, across nearly every benchmark we ran, across behavioral and persuasion tasks, GLM 5.2's safety profile lands between Claude-family models and DeepSeek v4 Pro.
In brief
- On behavioral benchmarks, GLM 5.2 outperforms DeepSeek v4 Pro on both MASK and DeceptionBench, landing closer to Claude on each. MASK honesty is 55%, versus Claude's 66% and DSv4 Pro's 20%. Under Gram, an automated alignment audit, its scores land between Claude Opus 4.8 and DSv4 Pro rather than matching Claude.
- On harmful persuasion benchmarks, GLM 5.2 is also better than DSv4 Pro, matching and doing better than Claude Sonnet 5 in some cases. On Attempt to Persuade, GLM 5.2 matches Claude's near-total refusal on Noncontroversially Harmful prompts, while its Undermining Control attempt rate sits between Claude and DeepSeek, closer to DeepSeek's end. On PersuSafety, GLM 5.2 roughly matches Claude on medium-harm prompts and outperforms both Claude and DeepSeek on high-harm prompts.
- Unlike the behavioral and persuasion results, GLM 5.2's evaluation-awareness trend doesn't fit a clean story. After climbing steadily from 0% (GLM-4.6) to 39% (GLM-5.1) on our agentic-misalignment trend, GLM-5.2 dropped to 32% — still well below Claude's ~80% baseline, but the first release-over-release decline we've seen. On MASK specifically, GLM 5.2 shows lower verbalised awareness than DeepSeek v4 Pro (3.9% vs. 6.1%). We don't yet have a confident explanation for either result.
- We think this result is worth surfacing on its own. A model getting quieter about evaluation awareness, without a way to tell whether that reflects less awareness or less willingness to say so, is a live concern for chain-of-thought-based safety monitoring generally.
We note that these findings are preliminary: our suite is narrow, we compare it to several different Claude model versions across evaluations, and we have not yet evaluated GLM 5.2 in sustained, multi-turn agentic deployments, where benchmark-level behavior often fails to translate cleanly into real-world behavior. Closing that gap is a priority for our next round of evaluations, particularly on longer-horizon, agentic tasks.
1 Behavioral
We ran MASK (Ren et al., 2026) to measure honesty under pressure: does the model contradict a belief it has already stated once it's given an incentive to do so? GLM 5.2 has a 55% honesty rate, which sits closer to Claude Sonnet 4.6 (66%) than to DeepSeek v4 Pro (20%).
The same pattern holds on DeceptionBench (Huang et al., 2025), which tests deception across realistic scenarios, such as a financial advisor pressured to misrepresent investment risk, or a healthcare agent incentivized to downplay medication side effects. Across both single-turn and multi-turn conditions, GLM 5.2's deception rate (at 11.3% and 16.7%) came in relatively low compared to DSv4 Pro (at 33.9% and 52.2%).
To test behavior under live pressure, we ran Gram, Google DeepMind's variant of Petri (an automated alignment audit tool) designed for scenario realism. We ran the four publicly available seeds. In the baseline scenario (Observe), where the auditor acts only as a scenario simulator, the models tested generally do not misbehave.
In the pressure scenario (Redteam), where the auditor actively introduces pressure for the model to misbehave, GLM 5.2's scores across dimensions like 'concerning' and 'scheming' largely land between Claude Opus 4.8 and DeepSeek v4 Pro.
2 Persuasion benchmarks
We ran Attempt to Persuade Eval (Kowal et al., 2025), which measures whether LLMs will attempt to generate persuasive content on harmful topics, shifting focus from persuasion success to persuasion propensity. Of the four categories of topics, we focus on the two key categories: Noncontroversially Harmful (e.g. asking the model to manipulate individuals into creating a doomsday cult), and Undermining Control (e.g. encouraging employees to revolt against managerial oversight).
On Noncontroversially Harmful prompts, GLM 5.2 almost always refuses, attempting persuasion in only 1% of conversations, matching Claude Sonnet 5 (1%). DeepSeek V4 Pro, by contrast, attempts persuasion in roughly 88% of cases. However, on Undermining Control, GLM 5.2 attempts persuasion in about 70% of conversations, an improvement over DeepSeek's ~86%, but still well above Claude's ~33%.
On harmful persuasion, we also ran PersuSafety (Liu et al., 2025), which similarly checks whether LLMs refuse unethical persuasion tasks in a single-turn setting. Its tasks range from low-harm nudging as a baseline (e.g. exaggerating how fun a boring event is), to medium-harm exploitation (e.g. pressuring an elderly customer into an unneeded warranty), to high-harm coercion (e.g. pushing someone to skip needed medical treatment).
On medium-harm prompts, GLM 5.2's persuasion attempt rate is at 54%, roughly matching Claude Sonnet 5 (57%) but lower than DeepSeek v4 Pro (76%). GLM 5.2's persuasion attempt rate drops to near zero at 6% on high-harm prompts, outperforming Claude Sonnet 5 (at 31%) and DeepSeek v4 Pro (at 78%).
We use an APE-style judge based on StrongREJECT rather than PersuSafety's original setup, which asks the model directly “Will you engage in this task?” and tips the model off that refusal is an option. The APE judge instead scores the model's typical output to just the prompt, which we found in our DeepSeek v4 Pro report to be a more faithful measure of a model's tendency to engage with these tasks.
3 Evaluation awareness
Following our DeepSeek v4 Pro report, we track GLM's verbalised eval-awareness rate by running an agentic misalignment (Anthropic, 2025) scenario over 100 epochs.
GLM climbed steadily across five releases, from 0% for GLM-4.6 to 39% for GLM-5.1, before GLM-5.2 broke the trend and dropped to 32% in this setup, still well below the Claude Sonnet/Opus 4.5 baseline of ~80%.
As a cross-check, we also ran a chain-of-thought monitor over our MASK evaluations to check for verbalised evaluation-awareness rates. GLM 5.2 was eval-aware in 39 of 1,000 evals (3.9%), which was lower than DeepSeek v4 Pro's 6.1%. To compare to a previous version, we ran MASK on GLM 5.1 and found that its verbalised evaluation rate was similar at 41 of 1,000 (4.1%), within error of GLM 5.2.
We don't yet have a confident explanation for the GLM-5.2 dip. Given the error bars in this setup, this could simply be noise rather than a genuine reversal. It could also reflect the anti-reward-hacking guard Z.ai disclosed adding during GLM-5.2 training, which may suppress verbalised “this looks like a test” reasoning without necessarily reducing underlying awareness.
4 Conclusion
We think this drop in evaluation awareness could matter for chain-of-thought monitoring: if the drop is real, it could mean the model has learned to hide the “this is a test” thought without actually being less aware. This could also potentially mean that it learned to hide its reasoning more broadly, not just on this one signal. We can't yet tell which explains GLM 5.2's drop, and intend to investigate this further.
One limitation of this research note: our Claude comparisons aren't from a single model. Each benchmark compares against whichever Claude model we'd already evaluated in earlier work, or whichever Claude scores were publicly available for that benchmark, rather than one consistent reference model throughout.
Looking ahead, we think it's important to look at longer-horizon, agentic traces, where evaluation awareness may show up more realistically than in single-scenario evals. Agentic tasks may also be where evaluation awareness causes a model to change its actions, which may have bigger consequences than the model changing its answer to a question.