Do language models understand risk? A rigorous cross-model study of internal risk representations in Qwen-7B and Mistral-7B.
We tested whether two 7-billion-parameter language models (Qwen2.5-7B-Instruct and Mistral-7B-Instruct-v0.3) have internal representations of risk attitude that go beyond simply encoding what a lottery looks like. We presented each model with 13,100 binary choices between lotteries and sure things, extracted hidden-layer activations, and ran a hierarchy of five increasingly demanding tests.
The central finding is a paradox. Neither model has orderly behavioral risk preferences — their choices are inconsistent, position-biased, and violate basic rationality. Yet both models contain a detectable internal representation of risk attitude in their activations — one that survives confound controls and, in Qwen's case, is causally connected to the model's choices without affecting non-risk decisions.
The representation is roughly 10-dimensional, not a single "risk direction." It transfers across different stake levels but not across probability regimes. Qwen's risk direction is specific to risk decisions; Mistral's is not — it disrupts factual questions too.
Methodological correction (v2). An earlier version of this analysis extracted activations at each lottery's switch-point stimulus — the trial where the model flips preference. Because the switch-point's sure-thing amount sk nearly determines RP (via RP ≈ EV − sk), a probe could trivially recover RP by reading the displayed number off the activation, without encoding any risk attitude. This analysis now uses lottery-mean activations (averaged over all 20 trials per lottery), which eliminates the sk confound. A diagnostic analysis confirms the old switch-point residual collapses to ρ ≈ 0 when sk is added to the confound matrix. The corrected residual values are lower but remain significant.
We organized the study as a hierarchy of claims, where each level depends on the ones before it. The pipeline tests increasingly strong assertions about the model's risk representation — from "does the model have consistent risk preferences?" up to "can we causally steer the model's risk behavior by intervening on its activations?"
Both models fail Level 0 and Level 3. The paradox: despite failing the behavioral baseline, the internal representation passes Levels 1–2 and partially passes Level 4.
| Parameter | Value |
|---|---|
| Model A | Qwen2.5-7B-Instruct (28 layers, hidden dim 3,584) |
| Model B | Mistral-7B-Instruct-v0.3 (32 layers, hidden dim 4,096) |
| Precision | FP32 (critical for activation geometry) |
| Exp 0 stimuli | 1,100 binary choices (400 EV-matched, 240 transitivity, 60 dominance, 200 retest, 200 order-swap) |
| Exp 1 stimuli | 12,000 binary choices (600 lotteries × 20 sure-thing levels) |
| Position randomization | Lottery randomly assigned to Option A or B (~50/50 split) |
| Target variable | Risk premium (RP = EV − CE), model-free, non-parametric |
| Permutation tests | 1,000 label shuffles for every prediction claim |
| Probing model | Ridge regression with nested cross-validation (α selected per fold) |
Neither model can make consistent risk choices. But both encode risk information internally — and in Qwen, we can use that encoding to causally steer behavior.
We asked: does the model have orderly risk preferences at all? We presented 400 EV-matched pairs (where both options have the same expected value, so an EV-maximizer would choose randomly), 80 transitivity triplets, and 60 stochastic dominance checks. We randomized which option appeared as "A" vs "B" to control for position bias.
Behavioral metrics for both models. Dashed lines show the pass threshold for each metric. The only metric both models pass is risk aversion rate (significantly different from 50%). Qwen passes dominance; Mistral is near chance (53%).
Both models fail Level 0. Neither has stable, position-independent risk preferences. Qwen's 50.5% order consistency means flipping which option is labeled "A" vs "B" produces essentially random changes in its choice. Mistral is even worse at 32.5%, and its dominance accuracy (53%) means it fails to prefer clearly better lotteries about half the time.
For 600 lotteries, we presented 20 sure-thing values spanning each lottery's outcome range and identified where the model's preference switches from lottery to sure thing. This switch point estimates the model's certainty equivalent (CE) — the guaranteed amount it treats as equally attractive to the risky lottery. The risk premium (RP = EV − CE) measures how much expected value the model sacrifices to avoid risk.
Qwen produces clean, monotonic preference schedules for 78% of lotteries. Mistral only manages 43% — over half its choice patterns have too many inconsistencies to estimate a meaningful certainty equivalent. Both models skew strongly risk-averse (88% and 85% of valid lotteries have positive risk premiums).
For each lottery, we average the model's hidden-state activations over all 20 binary-choice trials. This lottery-mean representation captures the model's encoding of the risky option while eliminating any single displayed sure-thing value. We ask: can these averaged activations predict the lottery's risk premium?
We use ridge regression with 10-fold cross-validation and test significance with 1,000 random label shuffles. Crucially, we also test residual prediction: after statistically removing everything predictable from the lottery's parameters (EV, variance, win probability, outcomes), can the activations still predict the remaining risk premium?
Why lottery-mean, not switch-point? An earlier version used the activation at the switch-point trial (where the model flips preference). But the switch-point stimulus displays a sure-thing amount sk that nearly determines RP via RP ≈ EV − sk. A diagnostic test confirms this confound: adding sk to the confound matrix collapses the switch-point residual to ρ ≈ 0 (p > 0.1) for both models. The lottery-mean method averages over all 20 sure-thing values, washing out sk-specific encoding.
Prediction accuracy (Spearman ρ) across transformer layers. Solid lines: total prediction from lottery-mean activations. Dashed lines: residual prediction after removing stimulus-parameter confounds. All points are significant (p<0.001 by permutation test). Qwen peaks at layer 27 (residual ρ = 0.60); Mistral peaks at layer 18 (residual ρ = 0.26).
Both models pass Level 1; Qwen passes Level 2. Qwen's lottery-mean activations predict risk premium (ρ = 0.84) and the signal survives residualization (ρ = 0.60, p<0.001). Mistral's residual signal is weaker (ρ = 0.26) — still significant, but substantially lower than the uncorrected switch-point estimate (0.60). Over half of Mistral's original "risk representation" claim was an artifact of the sk confound.
How many dimensions does the risk representation need? We reduced activations to k principal components and measured prediction accuracy for each k.
Prediction accuracy vs. number of PCA dimensions used. A single direction (k=1) captures almost nothing — ρ is near zero for Qwen. Performance jumps sharply between k=5 and k=10, then plateaus. The risk representation lives in a roughly 10-dimensional subspace, not a single "risk direction."
A potential objection: maybe the activations just encode what the model is about to say ("A" or "B"), and that correlates with risk premium. To rule this out, we force-feed a fixed response — always "A" or always "B" — and extract activations from these teacher-forced sequences. Any remaining variation must come from the prompt context, not the response.
Even with the response text held constant, activations still predict risk premium (p<0.001 for both models). The Cohen's d values are large (>1.0), meaning the effect is practically significant, not just statistically so. The risk signal comes from the model's understanding of the problem, not from its planned answer.
Does the risk direction generalize? We trained the probing model on one type of lottery and tested on a structurally different type.
| Split | Qwen residual ρ | Qwen p-value | Mistral residual ρ | Mistral p-value |
|---|---|---|---|---|
| Small → Large stakes | 0.427 | 0.001 | 0.455 | 0.001 |
| Unlikely → Likely probability | −0.071 | 0.799 | Skipped (insufficient data) | |
| Gains-only → Mixed outcomes | −0.068 | 0.927 | Skipped (insufficient data) | |
| Grid → Random lotteries | 0.461 | 0.001 | 0.159 | 0.013 |
The direction transfers across stake levels (small → large) in both models, and across structural types (grid → random) in Qwen. But it does not transfer across probability regimes or outcome types. The risk representation is partially, not universally, general.
The strongest test: can we change the model's risk behavior by directly injecting the risk direction into its activations during a forward pass? We selected 100 lotteries near their indifference point and applied steering coefficients from −50 to +50.
Mean logit shift (change in log-odds of choosing Option A over B) as a function of steering strength. Negative coefficients push Qwen toward risk-seeking (choosing the lottery); positive toward risk-averse. The relationship is tightly monotonic (ρ = −0.95). Mistral's response is non-monotonic and collapses at extreme coefficients.
Flip rate on risk questions vs. non-risk factual questions. Qwen's risk direction only affects risk decisions (0% non-risk flips across all coefficients). Mistral's direction is not specific — it disrupts 58% of factual questions at the highest coefficient, meaning it isn't really a "risk" direction.
Qwen passes Level 4. Its risk direction is causal (65% flip rate), monotonic (ρ = −0.95), and specific (0% non-risk disruption). Mistral partially passes — it does flip choices, but the direction is not risk-specific. This suggests Mistral's "risk representation" may encode something more general, like decision uncertainty.
Are models using the same direction across different levels of risk aversion? We split lotteries into quartiles by risk premium, fit a separate direction in each quartile, and measured how aligned they are.
Both models show significantly more alignment than random (3–4× the null expectation), but far from perfect alignment (which would be 1.0). The risk encoding uses partially overlapping but distinct directions at different risk levels. This is consistent with the 10-dimensional subspace finding: different risk regimes may activate different combinations of those 10 dimensions.
The most striking result is the disconnection between behavior and representation. These models cannot reliably rank options by risk — they flip their choices when you swap the labels, and one in five transitivity tests produces a cycle. By any behavioral standard, they lack coherent risk preferences.
Yet their activations encode risk-relevant information that is separable from what the lottery looks like, stable across random data splits, and — in Qwen's case — causally connected to the choice the model makes. The model has the concept but cannot deploy it consistently.
One interpretation: the pre-training corpus teaches the model what risk-averse behavior looks like (it sees enough human text about risk to form a representation), but the instruction-tuning or decoding process introduces noise that prevents this representation from driving orderly behavior. The "concept" exists at the representation level but fails at the decision level.
| Dimension | Qwen | Mistral |
|---|---|---|
| Behavioral quality | Moderate (100% dominance, 82% retest) | Poor (53% dominance, 73% retest) |
| Position bias | Mild (A-rate 37%) | Severe (A-rate 71%) |
| Valid CE estimates | 469/600 (78%) | 258/600 (43%) |
| Peak residual ρ (corrected) | 0.60 (layer 27) | 0.26 (layer 18) |
| Steering specificity | Risk-specific | Not specific |
| Steering monotonicity | ρ = −0.95 | ρ = 0.63 |
Qwen is the stronger model by every metric. Its risk direction is specific (only affecting risk questions) and monotonic (more steering = more shift). Mistral's direction is a blunter instrument — it shifts the model's behavior, but it also disrupts unrelated decisions, suggesting it may encode something like "override default answer" rather than "adjust risk attitude."
The sk confound. The original switch-point probing methodology had a critical confound: the displayed sure-thing amount at the switch point nearly determines RP. The corrected lottery-mean methodology eliminates this, but at the cost of lower residual signal — particularly for Mistral, where the corrected residual ρ (0.26) is less than half the original claim (0.60). Mistral's Level 2 claim is now marginal rather than strong.
The cross-domain transfer (Level 3) fails for both models in the probability and outcome-type splits. We cannot claim a universal risk representation — the encoding appears partially tied to the structural features of the lottery. Additionally, Mistral's low CE yield (43%) limits the statistical power of its downstream analyses. The study uses only greedy decoding; sampling-based generation might produce different behavioral profiles.
Three directions are most promising: (1) test whether larger models (70B+) exhibit the same paradox or whether scale closes the gap between representation and behavior; (2) investigate whether the 10-dimensional structure aligns with known economic features (probability weighting, loss aversion, diminishing sensitivity); and (3) try fine-tuning the model to use its own risk representation more consistently, potentially closing the representation–behavior gap.