Behavioral Compositionality of Sparse Autoencoder Representations
Haotian Ma, Ziye Mei, Frederic Sala
Sparse autoencoder features are additive for reconstruction, but their downstream behavioral effects are not.
Do sparse autoencoder (SAE) features combine additively to produce model behaviors, or are there significant non-linear interactions?
Sparse autoencoders (SAEs) decompose model activations into linear features, but these features often interact in complex ways once they pass through the rest of the model. This means that simply summing individual feature effects fails to predict the joint behavior of a circuit or intervention. The authors introduce SAE Additivity Residuals (SAR), a framework that measures behavioral variation unexplained by a given interaction order. By sampling subsets of features and observing the downstream response, they quantify how much interaction exists beyond individual effects and whether pairwise terms can recover that missing structure. Pairwise modeling consistently improves rank fidelity and reduces ordering error across all 18 tested model and behavior settings, demonstrating that while SAE coordinates are comparatively compositional, faithful joint explanations require accounting for feature interactions.
Paper Primer
The problem is that SAE features are optimized for linear reconstruction, not for the non-linear downstream computation that determines model behavior. When researchers combine features to steer a model or explain a circuit, they often assume individual effects sum linearly, but the model's remaining layers can suppress or reinforce these features in ways that break this assumption.
The authors treat the model as a black-box function that takes a subset of active SAE features as input and returns a behavioral score. They define SAR as the fraction of behavioral variance unexplained by an order-$r$ approximation: SAR$_1$ measures the residual after accounting for individual features, while SAR$_2$ includes pairwise interactions. The core move is to sample feature coalitions, fit low-order models to the resulting behavioral responses, and evaluate them on held-out subsets to quantify the "interaction energy" required for a faithful description.
Pairwise interactions capture substantial behavioral structure that individual features miss.
Across 18 model and behavior settings, pairwise models consistently outperformed first-order models in rank fidelity and behavior-weighted ordering error. In a combined steering experiment, pairwise modeling raised rank fidelity from 0.5874 to 0.7790 and reduced ordering error from 0.1444 to 0.0606.
To ensure these results aren't just an artifact of the chosen SAE basis, the authors compared learned SAE coordinates against sum-preserving random rotations of the same activation subspace. Learned SAE coordinates consistently supported a lower-order behavioral description than the random rotations, suggesting the SAE basis itself is more naturally compositional than arbitrary alternatives.
Why does this problem matter for someone building interpretability tools?
If SAE features are not behaviorally additive, then "feature circuits" or joint explanations built by summing individual feature contributions will be systematically inaccurate. This paper provides the metrics needed to determine when a low-order explanation is sufficient and when it will fail to capture the true joint effect.
Is this a new architecture or a diagnostic framework?
It is a diagnostic framework. The authors do not propose a new SAE training objective; instead, they provide a method to evaluate the behavioral compositionality of existing SAE representations by treating them as inputs to a controlled intervention game.
The authors emphasize that reconstruction quality (how well the SAE recovers the activation) and behavioral compositionality (how well the features sum downstream) are distinct properties. In their matched SAE comparisons, reconstruction metrics failed to track behavioral fidelity, suggesting that "good" SAEs for reconstruction may not be the "best" SAEs for joint behavioral explanation.
Researchers should stop assuming SAE features are additive for behavior. Future work on multi-feature circuits must explicitly account for pairwise interactions to ensure their explanations and steering interventions are faithful.
Introduction and Motivation
We expose the gap between sparse‑autoencoder features and truly additive behavior.
Sparse autoencoders (SAEs) encode model activations as sparse, linearly decoded features, but it is unclear whether the downstream effects of those features combine additively. The paper asks: if we know each feature’s individual effect, can we predict the joint effect simply by summing them? Existing SAE evaluations measure reconstruction, sparsity, and interpretability, yet they do not reveal when independently characterized features can be faithfully composed into joint explanations or interventions.
SAR quantifies how much of a model’s downstream response cannot be explained by adding up the individual contributions of selected SAE features.
**Figure 1.** Behavioral compositionality in SAE coordinates. (a) Selected decoded contributions are retained or removed while all other activation content is fixed, and the patched activation is propagated through the frozen upper layers to obtain $g_x(S)$. (b, c) Nearly additive and interacting responses, showing the first order fit, pairwise interactions, and variance decomposition. $SAR_1$ is the residual beyond individual features and $\rho_2 = \Delta_2/SAR_1$ its pairwise share.
The paper tests whether SAE features are truly additive.
Defining Behavioral Compositionality
We detail the experimental protocol for measuring compositionality of SAE representations.
To probe compositionality we treat the activation at a chosen layer as a controllable set‑function. By fixing all but a selected coalition of SAE features we can isolate the behavioral effect of that coalition and then quantify how well low‑order additive models explain the observed responses.
We remove a chosen group of SAE features (a coalition) from the activation while keeping everything else unchanged, then observe how the downstream behavior shifts.
How does coalition‑based ablation differ from simply zeroing out each feature individually?
Zeroing a single feature removes only its contribution, while a coalition removes all selected features simultaneously and preserves the residual activation $h_x^{0}$. This joint removal can reveal interactions that are invisible when features are ablated one‑by‑one.
First‑order effects assume each feature contributes independently; pairwise interaction adds a term that captures how two features jointly influence the response.
Why do we need pairwise interaction terms if first‑order effects already explain most of the variance?
Even a small $\text{SAR}_1$ can hide systematic biases: pairwise terms reveal whether the remaining variance is structured (e.g., due to feature synergy) rather than random noise. The metric $\rho_2$ tells us exactly how much of that unexplained variance is captured by pairwise interactions.
Select the top $K$ active SAE features $N_x$ for a given prompt $x$.
Form the decoder matrix $D_x$ whose columns are the decoded contributions $\delta_i(x)$ for $i\in N_x$.
Sample a collection of coalitions $S\subseteq N_x$ from the uniform distribution (equivalently $p=\tfrac12$).
For each sampled $S$, compute the modified activation $h_x(S)=h_x^{0}+D_x\,s(S)$ and evaluate the downstream scalar $g_x(S)=\psi(F_\ell\!\leftarrow h_x(S)\big)(x)$.
Fit the best order‑$r$ additive model $\hat a_r\in\mathcal{A}_r(N_x)$ on a subset of the sampled coalitions (least‑squares regression).
Evaluate $\hat a_r$ on a held‑out set of coalitions, compute mean‑squared error $\hat L_r$, empirical variance $\hat V$, and derive $\widehat{\text{SAR}}_r=\hat L_r/(\hat V+\varepsilon)$.
Report additional predictive metrics: Spearman rank fidelity $F_{\text{rank}}$, sign error, and selection regret $R(r)$ on the held‑out set.
Construct the selector vector $s(S)=(1,0,1)^\top$.
Compute the modified activation $h_x(S)=0.5 + D_x\,s(S)=0.5 + (0.2+0.3)=1.0$.
Evaluate the downstream scalar $g_x(S)=\psi(F_\ell\!\leftarrow 1.0\big)(x)=1.2$ (using the frozen model).
Fit a first‑order model on the other two coalitions $\{1\}$ and $\{3\}$, obtaining weights $\theta_1=0.4$, $\theta_3=0.6$.
Predict $\hat a_1(S)=\theta_1+\theta_3=1.0$, yielding residual $g_x(S)-\hat a_1(S)=0.2$.
This tiny trial shows how $\text{SAR}_1$ captures the leftover variance (0.2) after accounting for individual feature effects, and how pairwise terms would further reduce that residual.
Experimental Results
Key compositionality findings across behaviors, coordinates, depth, and pairwise modeling.
Pairwise interaction models raise rank fidelity from 0.9782 to 0.9959 across all 18 model‑behavior settings.
Observed for Gemma‑3 4B, Mistral‑7B, and Llama‑3.1 8B on six behaviors.
All experiments used 1 000 (or 100, 800, 150) prompts per model‑behavior, selected the top $K\!=\!16$ active SAE contributions, and sampled 384 coalitions (256 for fitting, 128 for evaluation).
Fidelity quantifies how well a coalition’s predicted effect matches the true model behavior.
How does explanation fidelity differ from reconstruction quality?
Explanation fidelity evaluates downstream behavioral predictions (rank ordering, selection regret), whereas reconstruction quality (e.g., FVU) measures how accurately the SAE reproduces raw activations. The two capture distinct aspects of a representation.
**Figure 2.** Prompt median $\widehat{SAR}_1$ for six behaviors in Gemma-3 4B layer 22, Mistral-7B layer 24, and Llama-3.1 8B layer 21. Error bars show 95% prompt bootstrap intervals.
**Figure 3.** Paired first order SAR for learned SAE coordinates and sum preserving random rotations of the same selected activation subspace.
**Figure 4.** First order explanation fidelity across $\widehat{\text{SAR}}_1$ deciles within each model.
**Figure 5.** Held out coalition ordering for first order and pairwise approximations.
**Figure 6.** Depth profiles for simple copying, CounterFact, and IOI.
**Figure 7.** Simple copying at the earliest and latest sampled layers across seven model and SAE pipelines. Open markers denote the earliest layer and filled markers denote the latest layer. The left panel reports prompt median $\widehat{\text{SAR}}_1$ and the right panel reports prompt median $\widehat{\rho}_2$.
Estimation Procedures
Describes how the projection estimator is evaluated and diagnosed across sample sizes.
Recall that SAE Additivity Residuals (SAR) quantify the deviation of model behavior from a purely additive composition of Sparse Autoencoder (SAE) features.
The projection estimator fits a linear combination of basis functions to the model’s response on sampled coalitions, then compares the residual variance to the total variance.
Sample $M_{\text{eval}}$ coalitions $S_1,\dots,S_{M_{\text{eval}}}$ with independent feature memberships.
Evaluate the model on each coalition to obtain $g_x(S_m)$.
Construct the feature matrix $\varphi_r(S_m)$ containing all basis terms of size $\le r$ (Eq (40)).
Solve the least‑squares problem $\hat\theta = \arg\min_\theta \| \varphi_r\theta - g_x\|_2^2$ to get the projection coefficients.
Form the projected predictions $\hat a_r(S_m)=\varphi_r(S_m)^\top\hat\theta$.
Compute $L_r$ as the mean‑squared deviation between $g_x(S_m)$ and $\hat a_r(S_m)$.
Report $\text{SAR}_r = L_r/(V+\varepsilon)$.
Least‑squares fit yields $\hat\theta = (0.8,\,0.4)^\top$ because the design matrix is the identity.
Projected predictions are $\hat a_1(S_1)=0.8$, $\hat a_1(S_2)=0.4$ (exact match).
Mean‑squared error $L_1 = \frac{1}{2}\bigl[(0.8-0.8)^2 + (0.4-0.4)^2\bigr]=0$.
Assume the held‑out variance $V=0.05$; then $\text{SAR}_1 = 0/(0.05+10^{-8}) \approx 0$.
This toy run shows that when the basis spans the true response, the projection error vanishes and SAR collapses to zero, illustrating the ideal additive case.
Walsh coordinates provide an orthonormal basis for Boolean coalitions; using them makes the population Gram matrix the identity, which simplifies the least‑squares solution.
Finite‑sample diagnostics (Table 4) show that even with 2,700 prompts the second‑order Walsh Gram matrix remains well‑conditioned (median ≈ 41, 95th ≈ 50) and that monomial and Walsh predictions agree to machine precision.
The table presents various diagnostic metrics related to model prediction differences, rank prompts, Gram condition numbers, and residual variations across different orders.
Fitting‑budget diagnostics (Table 5) reveal a clear trend: as the number of fit coalitions grows from 160 to 256, the Gram condition drops dramatically, the variance‑explained metric $R_{\text{var}}^2$ improves, and rank‑fidelity reaches perfection (18/18).
**Table 5.** Fitting budget diagnostics with the original evaluation coalitions held fixed. The reported values aggregate the 18 model and behavior combinations.
For the final analysis we compute SAR₁, SAR₂, $\Delta$₂, and $\rho$₂ per prompt, aggregate medians across prompts, and bootstrap 1,000 resamples to obtain confidence intervals; paired‑endpoint comparisons use the same bootstrap draws on early and late layers.
Interaction Indices
We examine interaction indices, coalition weighting, and the scope of SAR residuals.
Interaction indices generalize feature attribution from single players to small coalitions. They assign a value to each pair or higher‑order subset, extending the Shapley value’s marginal‑contribution averaging.
They measure how much a pair of features together changes the model output beyond what each feature does alone.
Low‑order coefficients alone cannot determine the SAR residual. Under the product measure, coefficients up to order $r$ live in a subspace orthogonal to higher‑order terms, so altering the latter leaves all fitted low‑order coefficients unchanged while changing $\text{SAR}_{r}$.
Coalition weighting that reproduces the Shapley value places weight proportional to $\big[\binom{K}{|S|}\,|S|\,(K\!-\!|S|)\big]^{-1}$ on nontrivial coalitions. This weighting correlates feature memberships, breaking the orthogonality that product sampling provides.
Alternative intervention games replace Equation (2) with a reconstruction that uses only the retained subset $h(S)=\sum_{i\in S}\delta_i(x)$. This discards the residual component and evaluates the SAE reconstruction rather than the original activation.
The tested states lie in $h_{0x}+\operatorname{span}\{\delta_i(x)\mid i\in N_x\}$, a subspace of dimension at most $K$. A continuation can be nonlinear away from this subspace yet still produce an additive response on the tested states, so compositionality concerns the downstream computation expressed in the selected SAE coordinates.
Measurement Calibration
We detail how steering coalitions are sampled and evaluated to calibrate residual estimators.
To assess how well additive explanations hold, we construct steering coalitions and measure their residuals on the MMLU‑Pro benchmark.
A steering coalition S is a selected set of SAE features whose contributions are combined with a fixed coefficient at each transformer layer.
How does a “steering coalition” differ from a generic feature subset used in standard ablation studies?
In a generic ablation the subset may be re‑weighted or retrained; here the coalition’s coefficients are frozen and applied identically at every layer, so any error directly measures non‑additive interaction rather than confounding retraining effects.
Draw 1,024 distinct coalitions uniformly from the 216 possible feature‑combination space.
Assign each coalition a fixed coefficient and split the data into fitting and evaluation halves (identical for all domains).
For each of the 14 MMLU‑Pro domains, sample 100 test questions and compute the coalition’s accuracy.
Average the domain accuracies over the 1,400 questions to obtain overall MMLU‑Pro accuracy.
Resample prompts with replacement to generate confidence intervals for the accuracy games and projections.
Fit a first‑order additive model, then a pairwise interaction model on the fitting coalitions; evaluate the pairwise fit on held‑out coalitions.
Report $\rho^2$ without clipping; negative values indicate the second‑order fit has larger held‑out residual than the first‑order fit, while the population $\rho^2$ remains non‑negative.
**Table 8.** Model and SAE pipeline coverage.
Across the 11 affine controls in Gemma-3 4B, Mistral-7B, and Llama-3.1 8B, the largest prompt median $\widehat{\text{SAR}}_1$ is $1.55 \times 10^{-13}$ with 1,000 prompts for each control.
Coordinate Comparisons
Random rotations reveal that learned SAE bases yield consistently lower SAR₁ values.
The same activation subspace can be expressed in many coordinate systems; we therefore generate eight random rotations per prompt by fixing the all‑ones direction and QR‑orthogonalizing a Gaussian matrix on its $(K-1)$‑dimensional orthogonal complement. Table 10 shows that learned SAE coordinates consistently yield lower SAR₁ values than these random rotations.
Instead of using the basis learned by the SAE, we rotate the selected subspace randomly, preserving the subspace but scrambling the axes.
How do these random rotations differ from the learned SAE coordinates?
The SAE basis is optimized to align with sparse, interpretable features, so its SAR₁ reflects genuine additive structure; random rotations are agnostic baselines that test what SAR₁ looks like when axes carry no semantic meaning, which is why the SAE typically achieves lower SAR₁ values.
**Table.** Comparison of SAE $\widehat{\text{SAR}}_1$ and Rotation $\widehat{\text{SAR}}_1$ across different models and behaviors, including the proportion of SAE values below the rotation median.
Full Behavioral Results
Full results reveal substantial SAR reductions and $\rho$₂ gains across layers and pipelines.
SAR₁ drops by up to ≈ 9.7× from early to late layers while $\rho$₂ rises, indicating stronger compositional behavior.
Table 12 reports a 9.711× reduction for the Gemma‑3 4B simple‑copying pipeline.
All measurements use 1,000 prompts per condition and the same SAE pipelines; only the layer depth varies.
**Figure 8.** Reconstruction and Behavioral Compositionality Across Six SAEs. Figure 8 shows the comparison across the six SAE bases referenced in the main text.
**Figure 9.** Behavior scores for four examples under ablations of individual features and feature pairs, together with first order and pairwise predictions when available.
Explanation Fidelity Analysis
Key fidelity metrics and their empirical behavior across models and tasks.
We quantify explanation quality with two complementary metrics. **Sign error** is the fraction of evaluated coalitions where the additive prediction error $e(S)=g_x(S)-g_x(N_x)$ and the residual sign $\Delta er(S)=\Delta ar(S)-\Delta ar(N_x)$ disagree. **Selection regret** measures how far the coalition that maximizes the residual $\Delta ar$ falls short of the optimal coalition $S^*$, normalized by the range of $g_x$ values (with a tiny $\varepsilon$ for stability).
**Table 15.** First order explanation fidelity in the lowest and highest $\widehat{\text{SAR}}_1$ deciles within each model.
The table presents performance metrics for three language models (Gemma-3 4B, Mistral-7B, and Llama-3.1 8B) across six different behavioral tasks, reporting values for $F_{rank}^{(1)}$ and $F_{rank}^{(2)}$.
**Table 17.** Mean prompt level rank fidelity of the first order and pairwise approximations for the 18 model and behavior combinations.
**Table 18.** Held out behavior weighted ordering error for first order and pairwise approximations in the 18 model and behavior combinations used in the fidelity analysis.
**Table 19.** Held out ordering for the pairwise model and parameter matched models that replace pairwise terms with randomly sampled third order Walsh terms. All columns report prompt medians; matched quantities first take the median over 50 random models for each prompt.
Steering the SAE explanations improves overall rank fidelity by +0.1916.
Table 20 overall row shows the rank‑fidelity change increasing from 0.5874 to 0.7790, a gain of +0.1916.
The table presents numerical values for two variables, $\hat{\mathcal{E}}_1$ and $\hat{\mathcal{E}}_2$, across various academic domains, including an "Overall" row.
**Table 21.** Held out behavior weighted ordering error for the steering responses.
**Table 22.** Prompts used in Figure 9.
Robustness and Protocol Checks
We examine how varying the number of selected SAE features $K$ affects robustness.
Changing the feature count $K$ modifies the candidate explanation by adding or removing SAE coordinates, so the table reports how behavioral compositionality varies with explanation size rather than converging to a fixed quantity. The reported metrics include affine control (in units of $10^{-14}$), greater‑than, IOI, simple copying, and MMLU‑Pro steering, each evaluated at $K$ values from 6 to 20. Additional protocol checks confirm that raising the coalition inclusion probability to 0.65 and aggregating over 1,000 prompts do not alter the observed trends.
**Table.** The table displays the relationship between the "Number of selected features K" (ranging from 6 to 20) and various "Behavior or setting" categories, including Affine control, Greater than, IOI, Simple copying, and MMLU-Pro steering.
The results remain stable across a wide range of selected features $K$.
Mathematical Foundations
Formal statements and derivations supporting the paper’s interaction analysis.
This appendix gathers the mathematical statements referenced in the main text and supplies the missing derivations.
**Table 3.** Notation used throughout the paper.
Experimental Specifications
This section details the behavior prompts and steering feature specifications used in the experiments.
D.1 Behavior Specifications – the main experiments evaluate seven distinct behaviors, each defined by a prompt source and a scalar readout $\psi$.
For the “greater than” behavior the answer labels are model‑specific: Gemma‑3 4B and Llama‑3.1 8B use “A”/“B”, while Mistral‑7B uses “True”/“False”. The prompt templates and selected identities are fixed across the corresponding comparisons.
D.2 Steering Specifications – the steering experiment fixes 16 SAE features (the strongest positive‑correlation MMLU‑Pro features) and uses their published steering coefficients.
The steering experiment runs on Llama‑3.1 8B with the Llama Scope residual SAEs introduced by CorrSteer (Cho et al., 2026; He et al., 2024).