Research report: Paired response final divergence signatures of jailbreak vulnerability in large language models

Researchers have developed a new way to peer inside large language models and see, layer by layer, what happens when a jailbreak prompt successfully slips past a model's safety training. The study, published in Discover Artificial Intelligence, analyzed 15 open-source models and found that the internal fingerprints of jailbreak vulnerability are strikingly consistent within model families, and largely independent of the alignment strategy a model was trained with. The core idea behind the new framework is elegantly simple.

When a model receives two versions of the same harmful request, one asked directly and one wrapped in a jailbreak template, the model produces two completed responses. The researchers extracted the hidden-state vector at the final generated token of each response, across every transformer layer, and measured the cosine distance between the paired representations. If both framings lead to refusal, the closing representations should look similar.

If the jailbreak framing successfully redirects generation, the two closing states should drift apart. This paired response-final divergence thus provides a geometric readout of how deeply a jailbreak penetrates a model's processing. The evaluation covered 15 open-source models spanning several families, four alignment strategies, and four attack types: roleplay persona, emotional appeal, authority framing, and context manipulation.

The paired adversarial dataset combined 520 stratified prompts from AdvBench with 59 deduplicated jailbreak templates, plus a 630-prompt generalization set from SORRY-BENCH, yielding 5,502 prompt pairs per model. Responses were judged by an evaluator LLM running at temperature zero, producing compliance labels, a Harm Severity Score from 0 to 100, and an Information Leakage Index counting actionable items. The automated judge was validated against three independent human annotators, achieving a mean Cohen's kappa of 0.880, indicating strong agreement.

The behavioral results revealed a sharp family-associated contrast in jailbreak amplification, measured as the change in attack success rate between jailbreak and direct prompting. Five LLaMA-family models, spanning 1B to 13B parameters and three alignment strategies, showed attack success rate changes within a narrow band of minus 3.6 to plus 3.6 percent, with four of the five showing statistically significant negative or near-zero amplification. Non-LLaMA models fared dramatically worse: Vicuna-13B amplified by 57.4 percentage points and Vicuna-7B by 54.0.

Perhaps most telling, Qwen2.5-7B amplified by 31.2 percent while its sibling Qwen2-7B, under the identical SFT plus DPO alignment, amplified by only 12.3 percent, demonstrating that alignment labels and parameter scale alone cannot explain vulnerability differences within this sample. The representational analysis mirrored these behavioral patterns with remarkable fidelity. Mean paired cosine divergence ranged from 0.14 for Gemma-2B to 0.57 for Vicuna-13B, with LLaMA-family models clustering between 0.17 and 0.39.

Family-averaged layer profiles exposed three distinct shapes. The Vicuna family separated from other families at shallow transformer depth, exceeding a mean cosine distance of roughly 0.60 by normalized depth 0.3 and sustaining that elevation through later layers, an early-onset signature of representational displacement. The LLaMA family instead showed moderate, gradually increasing divergence, rising from about 0.17 at depth 0.1 to 0.40 at depth 0.9, a pattern consistent across all five LLaMA variants regardless of alignment method or size.

The Gemma family showed the lowest divergence overall. A validation check confirmed that divergence tracks genuine jailbreak penetration rather than superficial wording differences. When divergence was stratified by the judge's compliance outcome, the expected ordering held for genuinely aligned models: divergence was lowest when the jailbroken response remained a refusal, intermediate for partial compliance, and highest for full compliance.

Crucially, this ordering broke down in exactly the models with high baseline compliance, Mistral-7B-v0.1 at 87.4 percent baseline attack success and Falcon-7B at 40.8 percent, where direct prompts already frequently comply. In those cases, high divergence reflects pre-existing broad compliance rather than jailbreak-specific displacement, a distinction the authors treat as diagnostic rather than problematic. Because raw divergence measures magnitude but not direction, the researchers added a complementary measure: difference-in-means refusal-direction suppression.

Building on prior mechanistic work showing that refusal behavior is mediated by a single linear direction in the residual stream, they computed a unit refusal direction from the mean hidden states of refused versus complied jailbreak responses at each layer, then compared how strongly direct and jailbreak response-final states project onto that direction. This suppression summary, averaged over the middle 10 to 90 percent of normalized depth, correlated with jailbreak amplification across all 15 models at r = 0.753 (p = 0.001, 95 percent confidence interval 0.205 to 0.934), the largest cross-model association among the representational measures evaluated. Given the small cross-model sample of 15, the team subjected this central correlation to an unusually thorough robustness battery.

Rank-based measures corroborated the Pearson result: Spearman rho of 0.586 with permutation p = 0.028 and Kendall's tau-b of 0.486 with exact p = 0.011, alongside a permutation test on the Pearson statistic itself (p = 0.0024). A leave-one-model-out analysis kept the correlation positive and significant under exclusion of any single model, ranging from 0.642 when the most influential point, Vicuna-13B, was removed, to 0.825 when Yi-6B was removed. Theil-Sen robust regression yielded a slope of 3.85 with a confidence interval from 0.857 to 5.687, and excluding the two ceiling-compromised models left the correlation essentially unchanged at r = 0.742.

Notably, that same exclusion strengthened the weaker global-divergence correlation from a nonsignificant 0.391 to 0.640, suggesting baseline saturation had masked rather than inflated the secondary relationship. One of the study's most practically consequential findings concerns attack-type specificity. When divergence was correlated with amplification separately for each attack category, authority framing produced the strongest association (r = 0.836, p = 0.001, n = 11), followed by context manipulation (r = 0.712, p = 0.009, n = 12).

Roleplay persona, the most heavily used template type, did not discriminate at all (r = 0.329, p = 0.250, n = 14), because it produced uniformly high divergence across both resistant and vulnerable families. After Holm adjustment for the four comparisons, authority framing and context manipulation remained significant while emotional appeal and roleplay did not. The implication for safety evaluation is direct: benchmark suites dominated by roleplay templates risk under-detecting the very family-level vulnerability patterns that matter most for deployment decisions.

Synthesizing divergence and suppression evidence, the authors propose a three-profile representational taxonomy. Structurally resistant models, exemplified by the LLaMA family, show flat, low divergence across all depths and attack types, near-zero suppression, and a tight compliance-stratified ordering. Early-displaced vulnerable models, exemplified by Vicuna, show sharp divergence onset at shallow depth that persists through the network, uniformly across attack types.

Ceiling-compromised models, such as Mistral-7B-v0.1 and Falcon-7B, show high divergence driven by baseline compliance saturation, negative amplification, and a breakdown of the compliance ordering. The authors are careful about scope: the analysis uses only response last-token hidden states, covers 15 open-source models, excludes proprietary systems and multi-turn jailbreaks, and the reported associations are correlational rather than causal. Most families are represented by a single alignment variant, so architecture and alignment remain only partially separable.

Still, within the evaluated set, the message is clear: the geometry of a model's closing representations, not its alignment label, may be the more faithful guide to how it will behave under attack.

Subject of Research: Representational signatures of jailbreak vulnerability in large language models

Article Title: Paired response final divergence signatures of jailbreak vulnerability in large language models

Article References: Ismail, H. (2026). Paired response final divergence signatures of jailbreak vulnerability in large language models. Discover Artificial Intelligence, 6(1), Article 1596. https://doi.org/10.1007/s44163-026-02593-2

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02593-2

Keywords: large language models, jailbreak attacks, AI safety, hidden states, refusal direction, mechanistic interpretability, model families, attack success rate, alignment, adversarial prompts, representational divergence, safety evaluation