Writing / 29 September 2026

How LLM guards decide: a tour inside four safety models

In one sentence

Where the policy lives, which layers read it, what the router does and why the last block can overturn a verdict: a tour through the internals of four LLM guards, measured on the weights they actually run.

In Inside four LLM guards I compared what four safety models catch. This post is about how they do it: what happens between the input tokens and the one token that decides. Every figure comes from probes run on the same quantised weights that Ollama serves, validated against Ollama’s own outputs.

Qwen3Guard: the template is the policy

Qwen3Guard-Gen 8B does not receive a policy; it carries one. Its chat template wraps your text in a task, definitions of Safe, Unsafe and Controversial, nine categories and format rules. A system message from your application is serialised as another user turn: there is no channel for your own policy.

Figure 1. Qwen3Guard-Gen 8B. Top: the template that wraps every input, 36 identical decoder blocks and an untied output head. Bottom, animated: one real call. The input is built to scale, processed in a single prefill pass, then the output is generated token by token; the third token carries the decision, shown with its three label probabilities. Choose a case.

Figure 2. A typical call to scale (hover a segment). The fixed template dwarfs the content being judged. Below: gradient × input attribution that falls on the template rather than on the content, per case; filled = prompts, hollow = documents.

The model leans on that fixed text as much as on the content: between 37 % and 87 % of the attribution lands on the template. The decision is the third generated token (Safe, Unsafe or Cont…), and the label tokens only gain probability after layer 24 of 36. Just before, the top token already anticipates the judgement in plain words: unacceptable for a bomb manual, trivial for a CV hiding an instruction.

Figure 3. All 62 text inputs, scored as P(Unsafe) + P(Controversial) on a logit scale. Filled: attacks; hollow: benign. Harmful prompts pile up at the right. Documents with injected instructions sit among the benign ones: no threshold separates them well.

That second figure is the key to its blind spot. The errors are not a calibration problem that a lower threshold could fix; they are about what the model learned to call dangerous. In one email, the token that pushes hardest towards Safe is <!--, the opening of the HTML comment that hides the instruction.

Shieldstral: a linear classifier in the embedding matrix

Shieldstral 3B never generates. It reads the prompt once and compares two logits at the last position. Because its output head is its embedding matrix, the score reduces to geometry:

Figure 4. Shieldstral 3B. Top: images enter through the Pixtral encoder and a 2×2 merger; text and image tokens run through 26 decoder blocks; only two rows of the tied embedding matrix are ever read. Bottom, animated: each audited case placed on the one sigmoid that turns the margin Δ into a score, for the harm question and the manipulation question.

score = σ( (e_yes − e_no)^⊤ · RMSNorm(h₂₆) )

a fixed 3,072-dimensional direction learned by fine-tuning. The policy is not a document: it is the question you ask. We asked two per input, one about harm and one about manipulation, and kept the maximum.

Figure 5. Scores for the two questions, per case (hover for values). The manipulation question, written precisely for hidden instructions, stays low on every injected document; the ones that get caught are caught by the generic harm question, because they also look like fraud.

Inside, Shieldstral commits early, around layer 15-17 of 26, and then its last block recalibrates hard: on the injected CV the harm signal is 1.00 at layer 24 and 0.71 at the output. There is signal that the final layer throws away.

Images enter through a Pixtral encoder: 14-pixel patches, merged 2 × 2 into one token per 28-pixel cell, with a break token at the end of each row. The runtime decides how many cells you get.

Figure 6. Image tokens for a square image. The reference preprocessing grows to 3,080 tokens at 1,540 px; the GGUF runtime in Ollama caps at 1,024 cells, about 896 px, so small text in screenshots loses resolution. Both curves reproduce the token counts measured in the audit.

gpt-oss: 4 experts of 32, and no safety expert

gpt-oss 20B stores 20.9 B parameters but each token only passes through 3.6 B: in every layer a router scores 32 experts and keeps 4. It is tempting to look for a “safety expert”. There isn’t one.

Figure 7. gpt-oss 20B. Top: the harmony conversation (the policy is the dark developer block), 24 blocks alternating a 128-token window with global attention, and the router that keeps 4 of 32 experts. Bottom, animated: the real route of each content token of an injected CV through the 24 routers; opacity is the router weight, screened cells show experts already used.

Figure 8. Experts chosen at the decision token, for 24 layers and 8 cases (benign and malicious). Dark cells are chosen by most cases whatever their label: on average, two cases share 3.2 of their 4 experts per layer. Hover a cell for its counts.

The verdict travels through the residual stream; the router responds mostly to position and format. What does differ is attention. Even layers only see the last 128 tokens; odd layers see everything. Each head also has a learned sink, an extra softmax column with no value vector, so it can attend to nothing.

Figure 9. Per layer, at the decision token, averaged over 8 cases. Up: attention poured into the sink. Down: attention to the policy. Windowed even layers cannot reach the policy and dump about half their attention into the sink; global odd layers spend about 42 % of it reading the policy.

So the whole policy is read by twelve global layers, while the local ones effectively switch off. In the harmony format our policy sits in the developer role and the content in the user role, which puts an injected instruction exactly on the developer-versus-user boundary, where the model card reports one of its lowest scores (0.66).

Gemma 4 E2B: built for a phone, and it shows

Almost every decision in Gemma 4 E2B saves memory. Four local layers (window of 512) for each global one; one key-value head shared by eight query heads; and the last 20 layers compute no keys or values at all, reusing the cache of layers 13 and 14. Nearly half the parameters are a per-layer embedding table that is looked up, never multiplied.

Figure 10. Gemma 4 E2B. Top: 35 layers, four local for each global one; arcs show that layers 15-34 reuse the key-value cache of layers 13 and 14; every block also receives its own slice of a per-layer embedding table. Bottom, animated: P(violation) read at each layer, without and with the model's own reasoning in front. Choose a case.

Figure 11. Top: the 35 layers, tall = global, screened = reuses another layer's cache. Middle: the learned scale each block applies to its output, from 0.018 to 0.87. Bottom: attention from the decision position to the 848-token policy, averaged over ten cases, without and with the model's own reasoning in front.

Two things stand out. First, each block multiplies its output by a learned layer_scalar, and the standard loader silently sets them all to 1.0: skip that correction and every internal measurement describes a different model. Second, the bottom panel: without reasoning, the local layers spend about half their attention on the policy, yet the model still misses subtle attacks. With its reasoning written first, local attention to the policy drops to about 6 % and the verdict becomes correct. The model does not need to see the policy better; it needs the intermediate step that applies it.

What the internals teach

Decisions are late and fragile. All four decide in the last third of the network, and final layers can overturn a correct intermediate signal.
The policy lives in different places: fixed in a template, in the wording of a question, in a developer message read by half the layers, or in a system prompt the model must apply step by step.
Capacity is not where it seems. A 20.9 B model computes like a 3.6 B one; half of a 5.1 B model is a lookup table.
The runtime changes the model. Image preprocessing and silently dropped scales matter as much as the weights.

Method: tensor-by-tensor inventory of each GGUF, logit lens with a ≥ 1 % probability-mass filter, gradient × input attribution, attention and router probes, and a custom forward pass that decodes gpt-oss’s MXFP4 weights bit-exactly. Every probe was checked against Ollama’s outputs.