SAESCIENTIST-BENCH: Can AI Agents Conduct Autonomous SAE Interpretability Research?

Yuqiao Tan, Shizhu He, Jun Zhao, Kang Liu

SAESCIENTIST-BENCH evaluates whether AI agents can autonomously perform mechanistic interpretability research using Sparse Autoencoders.

Can autonomous AI agents reliably identify and validate specific features within a Sparse Autoencoder (SAE) to perform mechanistic interpretability research?

Autonomous AI research currently focuses on optimizing external model performance, leaving internal representations as opaque black boxes that are vulnerable to reward hacking and deceptive alignment. The authors introduce SAESCIENTIST-BENCH, a framework where agents use contrastive probes to navigate Sparse Autoencoder (SAE) dictionaries and identify features that represent specific concepts. While frontier agents demonstrate genuine discovery capabilities, they significantly lag behind expert baselines in causal steering, revealing a gap between identifying correlated features and finding actionable steering vectors.

Paper Primer

The benchmark tasks agents with discovering a single feature within a 131,000+ dictionary in Gemma-2-9B-IT that corresponds to a target concept. Agents design contrastive probes to distinguish the concept from spurious correlations and are evaluated on activation rank, concept selectivity, and causal steering efficacy.

Frontier agents exhibit strong concept selectivity but struggle with causal steering.

Top agents reach 92.91/100 on activation selectivity compared to the expert baseline of 98.92, but score only 31.47/100 on causal steering versus the expert's 57.75. A ~26-point gap in causal steering performance persists despite near-expert levels of selectivity.

Discovery success depends less on search volume and more on the rigor of hypothesis testing. Agents that design targeted counterexamples to rule out morphological distractors—rather than simply screening large numbers of candidates—consistently identify more potent features.

Why is causal steering a more rigorous test than activation selectivity?

Activation selectivity only measures correlation between a feature and a concept in static text, whereas causal steering tests whether the feature can actually modulate model behavior during generation without causing output degeneration.

What does this benchmark reveal about the current state of autonomous interpretability?

It demonstrates that while agents can successfully perform representational auditing, they frequently misinterpret experimental measurements and struggle to separate superficial lexical matches from deep semantic concepts.

Autonomous mechanistic interpretability is a measurable capability, but current agents require better experimental rigor to bridge the gap between identifying correlated features and validating causal control.

Autonomous Interpretability Research

We benchmark AI agents as scientists using SAE tools to discover and evaluate interpretable features.

The field lacks a critical pillar: systematic post‑hoc monitoring and auditing of internal representations, which is essential for trustworthy autonomous self‑improvement.

SAESCIENTIST‑BENCH is a benchmark that asks an AI agent to act like a scientist: given a target concept, the agent must craft contrastive probes, search a large SAE feature dictionary, and select the feature that best matches the concept.

Agents design contrastive probes to separate the target concept from spurious correlations, then query the SAE dictionary to locate the optimal feature; the selected feature is scored against expert references using activation rank, selectivity, and steering metrics.

**Figure 11:** Overall and Steering within each task category. Cells show the adopted scores, with a common 0–100 color range for the displayed values.

**Table E.3.** Repeated Discoveries on Four Concepts

**Figure 12.** Steering scores across independent discoveries of four concepts.

Across 10 agents and 20 tasks, frontier models show genuine discovery ability—some excel in activation rank, others in selectivity or steering—but all remain substantially behind expert baselines, especially in causal steering where scores drop to roughly half of expert performance.

The shift from manual to autonomous interpretability auditing is the key step toward scalable, safe recursive self‑improvement.

SAE Features and Benchmark Design

Defines the SAE feature representation and the benchmark workflow for autonomous discovery.

SAE compresses a model’s hidden state into a small set of non‑negative, sparse activations that correspond to interpretable latent features.

How does this SAE differ from a standard autoencoder?

Standard autoencoders minimize reconstruction error without any sparsity pressure, so most latent dimensions fire for every input. The SAE adds a sparsity penalty and the JumpReLU threshold, forcing only a few $z_f$ to be non‑zero, which yields interpretable, isolated features.

Each entry $z_f$ in the sparse code measures how strongly the corresponding latent feature is present in the current token.

Run the base language model on input text to obtain hidden states $h$ at the chosen layer.

Encode $h$ with the Sparse Autoencoder to produce sparse activations $z$.

Agent writes up to 64 probe texts and sends them to the

For each probe,

Agent selects a single feature ID $f$ that best matches the target concept.

Evaluator measures three metrics on the held‑out evaluation set: activation rank, activation strength, and causal steering effect.

Compute the steering offset $\alpha d_f = 0.5 \times (0.3, 0.0, -0.2) = (0.15, 0.0, -0.1)$.

Update the hidden state: $h \leftarrow h + \alpha d_f = (0.2, -0.1, 0.5) + (0.15, 0.0, -0.1) = (0.35, -0.1, 0.4)$.

The modified hidden state is then fed into the next transformer layer for generation.

Steering adds the chosen feature’s direction directly to the hidden representation, changing downstream token predictions while preserving the rest of the model’s computation.

**Figure 2.** **SAEScientist-Bench workflow.** For a concept and fixed SAE, the agent writes probes $P_i$, compares candidates $f_j$, and submits a feature. The evaluator measures Rank, Activation, and Steering using Expert and the evaluation set. Portuguese texts and replies illustrate the process.

**Figure 3.** Benchmark coverage. Category shares and task counts by SAE layer, with an example concept and Expert feature for each category. All tasks have equal weight. Appendix A.1 lists the full set.

Measuring Interpretability Success

We define the three metrics used to score SAE features and how they are combined.

This section formalizes the three evaluation metrics—Activation Rank, Activation Selectivity, and Causal Steering—and explains how they are combined into an overall score.

Measures how prominently a feature fires on target texts compared to the full dictionary, rewarding features that rise to the top.

Why not use the raw activation magnitude instead of a rank‑based score?

Raw magnitudes depend on scale and can be inflated by outlier tokens; ranking normalizes across the whole dictionary, making the metric robust to overall activation level differences.

Assesses whether a feature distinguishes its target concept from unrelated or neutral texts, rewarding clean separation.

Why is the AUROC transformed with 2·AUROC − 1 instead of using the raw AUROC?

The raw AUROC ranges from 0.5 (chance) to 1.0 (perfect); the transformation maps chance to 0 and perfect to 100, making the scale comparable to the other metrics.

Measures how much intervening on a feature changes model output toward the target concept, beyond what happens with a random direction.

Why compare the steered feature against both the baseline and a random‑direction control?

Baseline isolates the effect of simply prompting the model, while the random direction ensures that any gain is due to the specific feature direction rather than just adding norm‑matched noise.

Aggregates the three dimension scores into a single leaderboard metric by averaging them equally.

Why use an unweighted mean instead of weighting metrics by perceived importance?

Weighting would embed subjective judgments about which dimension matters more; the authors chose equal weighting to let the three orthogonal qualities speak for themselves.

Compute the mean of the three largest non‑special token activations for each positive text.

Rank the feature against the full dictionary and calculate the Rank score using the formula.

Compute AUROC between positive texts and contrastive controls, then derive the Activation score.

Run three generations per prompt (baseline, feature steering, random‑direction control), collect target‑relevance ratings, and compute the Steering score.

Average the three scores to obtain the Overall score for the task.

**Table 1.** Agent scores and rankings on Rank, Activation, Steering, and Overall across 20 tasks. Scores are means over three runs. The $\pm$ values give the standard deviation across runs.

Agent Performance Benchmarks

Frontier agents excel at activation selectivity but lag behind experts in causal steering.

Frontier agents achieve strong activation selectivity but fall far short of expert performance on causal steering.

Table 1 shows the best overall composite score of 65.82 (Kimi K3) versus the expert’s 85.56, and the top steering score of 31.47 (Grok 4.6) versus the expert’s 57.75.

All agents were evaluated on the same 20 discovery tasks with identical instructions, three independent runs per task, and scores averaged equally across tasks.

**Figure 1.** **SAEScientist Agent Index.** Agents are ordered by their composite Overall score, with each bar decomposed into equally weighted Rank, Activation, and Steering components. Error bars denote standard deviations across three independent runs.

Agents excel at activation selectivity but lag in causal steering, exposing a key bottleneck for white‑box interpretability.

How Agents Search for Features

How agents formulate hypotheses, author contrastive probes, and evaluate candidates.

Agents must turn a vague research question into a concrete hypothesis, then author probes that isolate the desired concept while ruling out spurious shortcuts.

We ask the model to distinguish a target sentence from carefully crafted distractors, so that only the intended semantic feature survives.

How does contrastive probing differ from ordinary probing that only uses a single input?

Ordinary probing asks “does this feature fire on this sentence?” and reports the raw activation. Contrastive probing adds a matched negative example, so the metric is the activation *difference* between positive and negative, which isolates the semantic signal and removes surface‑level confounds.

Pick a target concept (e.g., “Portuguese morning routine”).

Author a set of positive probe texts that contain the concept.

Generate negative probes that share surface characteristics but omit the concept.

Run each probe through the layer‑9 SAE and record feature activations.

Compute contrastive scores (positive – negative) for every SAE feature.

Select the feature with the highest score, then evaluate its Rank, Act, and Steer metrics.

If the selected feature fails a downstream test (e.g., causal steering), return to step 2 and refine the probe set.

Run both probes through the SAE; feature #42 yields activations 100.00 (positive) and 0.00 (negative).

Contrastive gap = 100 – 0 = 100, the highest among all features.

Compute downstream metrics: Rank 3.97, Act 100.0, Steer 88.8, Overall 64.2.

Even with a perfect contrastive gap, the modest target activation (7.04) limits the feature’s ranking, illustrating the trade‑off between selectivity and signal strength.

Search loop used by all agents to discover SAE features.

**Figure 4.** Autonomous scientific search workflows across frontier agents. Metrics report average behavior per investigation across 20 tasks: number of authored probe requests, unique authored probe texts, directly tested feature candidates, and the percentage of episodes where the agent resumed candidate retrieval after direct testing. Column color scales are normalized independently.

**Table 12.** Probing traces from Claude Opus 5 on Cat and Tax Filing tasks. Contrastive probe texts expose candidate feature sensitivities: string-level matching (19127) vs. true semantic concept (62610), and specialized filing forms (64827) vs. broader tax concept (18713).

Rigorous hypothesis testing, not sheer search volume, determines discovery success.

Generalization and Activation Analysis

Activation scores expose hidden gaps, weak strengths, and the benefit of multi‑token aggregation.

Agent‑authored probes often select features that fail to activate on held‑out target sentences while firing on unrelated control texts.

Figure 5a shows Sonnet, Grok, and Luna’s features are inactive on the rain passage but spuriously activate on warranty statements; Gemini’s feature misses the research passage entirely.

**Figure 5.** Generalization and aggregation sensitivity of discovered Portuguese features. (a) Activations across representative positive texts (Train, Rain, Research) and contrastive controls (Grammar, Word list, Warranty), demonstrating that identical AUROC separation (e.g., Sol vs. Expert) conceals major differences in absolute activation strength and dictionary rank. (b) AUROC comparison between maximum token activation and top-three-token mean aggregation; token averaging dampens isolated lexical spikes in control texts, reliably isolating continuous semantic activation.

Causal Control and Instruction Preservation

Evaluates how feature interventions steer generation and preserve original instructions.

Activation steering intervenes directly on inference to test causal control, measuring both target relevance and instruction preservation across six tasks. Interventions follow $h \leftarrow h + \alpha df$ for each submitted direction.

Sol achieves the highest causal steering score while preserving instructions.

Steering score 88.75 and instruction preservation 3.525, outperforming other agents.

**Figure 6.** Causal steering and instruction preservation under Portuguese feature intervention. Evaluations across six diverse instructions: HTTP request explanation, 17×24 arithmetic, 4-line poem, clean energy essay, password generation, and gravity definition. (a) Target relevance (0–4 scale) measures the intensity of induced Portuguese expression; (b) Instruction preservation (0–4 scale) measures fulfillment of the original task constraints without degeneration or incoherence.

Table 3 illustrates concrete generations: the expert‑derived feature flips responses into fluent Portuguese, whereas the no‑steering baseline remains in English.

On a four‑line rhyming poem, Sol produces four coherent Portuguese lines, Opus 4.8 generates eleven repetitive lines, and the expert model yields six lines; meanwhile, GLM‑5.2’s intervention corrupts instruction following, turning a self‑introduction into a rambling medical history.

Related Work

Summarizes prior mechanistic interpretability and autonomous‑agent benchmarks.

Early mechanistic interpretability examined individual neurons and localized circuits using activation patching, causal tracing, and knowledge attribution. Polysemanticity—neurons firing for unrelated concepts—limits the usefulness of raw hidden states. Sparse Autoencoders (SAEs) decompose activations into sparse feature directions, enabling causal interventions whose steering effectiveness depends on influence over model outputs.

Automated methods employ language models to generate explanations for neurons and SAE features from activating examples. Agents extend this pipeline through interactive experiments, feature discovery, circuit analysis, and auditing tools. Execution‑grounded benchmarks assess whether agents can conduct end‑to‑end mechanistic interpretability experiments, measuring SAE quality, feature recovery, concept disentanglement, circuit identification, and steering.

Recursive self‑improvement (RSI) envisions autonomous systems iteratively enhancing their capabilities, with recent benchmarks evaluating agent autonomy across scientific‑research workflows such as ML engineering, paper reproduction, and data‑centric discovery. These benchmarks typically treat models as black boxes, ignoring internal representational changes and exposing loops to specification gaming. In contrast, SAESCIENTIST‑BENCH targets the representational‑auditing pillar, testing agents’ ability to inspect and causally steer internal model representations within pretrained SAE dictionaries.

Agent Investigation Case Studies

Agent‑level ablations reveal how different search strategies affect feature discovery and downstream stability.

**Figure 7.** Per-agent search outcomes across 60 episodes each. Returned means the Expert is present in the recorded candidate set; tested means it was directly requested. Each episode belongs to one segment.

**Figure 10.** Mean Steering for every agent and task. Values are rounded to integers; color spans 0–100. L9 and L20 identify the SAE layer.

**Table 9.** GPT-5.6 Sol’s Portuguese investigation.

The table displays probe excerpts and context for two tasks: "Cat Task" and "Tax Filing Task". It lists the Candidate ID, Activation levels, and Dictionary Rank for various contexts related to the target concepts.

**Table 10.** Real-estate candidate trade-offs and evaluation. Top: Search strategies of Sol, Grok 4.6, and Kimi K3 during candidate exploration. Bottom: Retrospective benchmark evaluation of competing features (17219 vs. 84434) and Expert 27182 across nine independent runs.

Task Robustness and Comparisons

We examine how benchmark scores relate to the AI Intelligence Index and task‑specific steering consistency.

In E.1 we compare our agents’ benchmark scores to the Artificial Analysis Intelligence Index v4.2, a composite metric that aggregates multiple intelligence‑related evaluations. The AA Index (higher = more capable) and the benchmark’s Overall Score are reported for four high‑effort models.

The table displays various AI model outputs for a creative writing task, categorized by the model name, the number of lines generated, and a "T/P" score (likely indicating success or preference). The rows include: - **No steering**: 4 lines - **Expert Kimi K3**: 6 lines, 4/4 - **Claude Opus 5, GPT-5.5, GLM-5.2**: 4 lines, 0/2 - **Claude Sonnet 5, Grok 4.6, GPT-5.6 Luna**: 12 lines, 0/1 - **Claude Opus 4.8**: 11 lines, 4/4 - **Gemini 3.8 Flash**: 4 lines, 0/0 - **GPT-5.6 Sol**: 4 lines, 4/4

**Table 28.**

E.2 examines steering performance across tasks and categories. Kimi and Sol achieve positive real‑estate steering, whereas Opus 5 scores zero on that task but records a non‑zero Spanish steering score; the “Cat” task yields identical scores for all agents.

Category‑wise, Kimi’s professional steering (45.03) exceeds Opus 5’s (32.19), yet Opus 5 outperforms Kimi on report generation (27.50 vs 12.81). Sol’s professional steering (44.04) is close to Kimi’s despite a lower overall benchmark score (65.30 vs 82.71), highlighting divergent strengths across evaluation axes.

The table presents the Spearman $\rho$ correlation coefficients for four different benchmark dimensions: Activation Selectivity, Overall Score, Activation Rank, and Causal Steering.

E.3 reports repeated investigations on four concepts (Cat, Portuguese, Tax filing, Clinical). Independent runs produce consistent steering scores for each agent across these concepts, confirming the reliability of the benchmark’s measurements.

Experimental Protocol

Appendix A details the task inventory, expert baselines, agent setups, metrics, and steering protocol.

The appendix records the full experimental pipeline, from the curated task list through agent runs, metric computation, and the final steering evaluation.

A.1 enumerates 20 concept‑layer tasks spanning two layers; each entry is paired with a unique Expert ID that serves as a frozen reference feature.

**Table 4.** Task concepts and Expert IDs. Each SAE contains 131,072 features.

A.2 explains that Expert features are taken directly from public Neuronpedia presets when available and otherwise curated through a six‑positive/negative probe pipeline validated on eight positive, eight hard‑negative, and four neutral texts.

A.3 lists ten agent‑model pairs (Table 5) and notes that each configuration runs 60 scored episodes across three investigations per task, with failed attempts retried to guarantee a single scored result per episode.

The table lists various AI agents, their associated evaluation harnesses, and their specific model identifiers used in the study. Each configuration is noted to have 60 scored episodes across three investigations per task.

A.4 defines the benchmark scores (Equations 3–6) and provides a worked example (Table 6) that walks through the calculation of Mean positive rank, AUROC, and Target Effect for both the GPT‑5.6 Sol submission and the Expert baseline.

**Table 6.** Component scores for GPT-5.6 Sol's Portuguese selection.

A.5 reports aggregate statistics: 600 scored episodes, 4 873 probe calls, 49 407 text entries, and 10 940 generations evaluated twice by GPT‑4o, yielding 14 880 condition ratings.

A.6 details the steering generation protocol (Table 7) and the three‑stage scale‑selection filter that first discards degenerate generations, then maximizes target cue success, and finally prefers the smallest viable scaling factor $α$.

**Table.** Settings for scale selection and final scoring.

**(b) Texts with nonzero activation (%)**

Steering Evaluation Rubric

This appendix details the steering evaluation rubric and the prompts used for the GPT‑4o judge.

The evaluation uses an automated GPT‑4o judge that receives the user instruction plus three candidate completions (A, B, C) in a pseudo‑randomized order, so the judge cannot tell which output comes from baseline inference, the SAE feature intervention, or the random‑direction control.

For each candidate the judge assigns three scores: Target Relevance (0–4) measuring how strongly the target concept appears, Task Preservation (0–4) measuring adherence to the original user request, and a Boolean Degeneration flag that marks severe output collapse such as repetitive loops or incoherent formatting.

Scores are averaged over all evaluation prompts and over two independent passes per prompt (temperature 0), and the judge must return strict JSON containing the three ratings.

The system prompt supplies the concept‑specific criteria and the rating rubric, while the user prompt supplies the evaluation instruction and the three candidate outputs for the judge to score.

**Table 8.** Concept-specific criteria in the steering judge’s system message.

Probing Evidence and Search Logs

Appendix C Part 1 lists the full probe texts and activation numbers used for the Portuguese investigations.

Table 11 records the top‑three‑token mean activations for three sources—Expert, Luna, and Sol—on Sol’s ninth Portuguese probe, showing the raw Portuguese passage, a reservation request, an English description, and translations into Spanish and Galician.

Table 12 presents contrastive probe traces from Claude Opus 5 for two tasks: a Cat description (layer 9) and a Tax‑Filing scenario (layer 9). For each task the table lists the candidate feature IDs, their activation values, and the Neuronpedia expert IDs that serve as ground‑truth references.

**Table 13.** Complete authored probe texts across four agent investigations (Portuguese, Layer 9). This table details the exact inputs authored by Sol, Claude Opus 4.8, Claude Opus 5, and Kimi K3, categorized by probe round and contrastive hypothesis.

Portuguese Task Probing Data

Appendix C Part 2 records the full probe texts and the cross‑agent evaluation tables.

Appendix C Part 2 records the full probe texts used for Kimi K3 and presents the detailed tables of contrastive probes and cross‑agent evaluations.

Kimi K3 rejects candidate 19127 for strong cross‑language activation (35.33 on the English market probe and 39.75 on the Spanish probe) and candidate 85098 for excessive Catalan leakage (26.92 vs 0.71), selecting 41424 which matches the Neuronpedia expert.

The table lists various probes performed on the Kimi agent, categorized by "Agent & Probe #", "Contrast Type", and "Full Authored Input Text".

**Table 14.** Complete GLM-5.2 contrastive probes for Clinical Symptom discovery (Layer 20). Probes 2–4 tested competing hypotheses to disentangle disease symptoms from medical bureaucracy, pharmacology, and clinical document format.

**Table 15.** All ten inputs in Sol's final Portuguese probe, measured on feature 49607.

Extended Probing Logs

Appendix C Part 3 details activation aggregation comparisons and model response statistics across tasks.

The section opens with short sentences in Spanish and Italian illustrating raw input texts used for probing. These examples show the variety of source sentences that later activation analyses will reference.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers