Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence

Mengru Wang, Junfeng Fang, Shuofei Qiao, Zhenqian Xu, Haoming Xu, Haoxiong Wang, Shumin Deng, Linyi Yang, Zhixiang Cui, Xin Xu, Yunzhi Yao, Buqiang Xu, Fei Shen, Haozhe Luo, Yunxiang Wei, Ningyu Zhang, Julian McAuley, Tat Seng Chua, Huajun Chen

Mechanist is an agentic framework that automates mechanistic interpretability by treating AI models as scientific subjects.

How can we automate the scientific discovery of internal AI mechanisms using a multi-agent framework that generates, tests, and verifies hypotheses?

Mechanistic interpretability is currently a manual, slow process that struggles to keep pace with the rapid evolution of AI models. Researchers lack a systematic way to isolate the internal computations responsible for specific behaviors or latent risks. Mechanist functions as an autonomous scientific instrument: it uses a multi-agent architecture to generate hypotheses, execute experiments using a library of 32 interpretability methods, and iteratively verify findings against a massive, cross-disciplinary knowledge graph. The system successfully uncovers latent safety risks, identifies the internal "belief heads" governing model reasoning, and enables targeted interventions that improve performance without retraining.

Paper Primer

Mechanist operates through a central orchestrator that manages four specialized agents: hypothesis generation, experiment execution, verification, and iteration. By externalizing memory into structured, stage-scoped artifacts, the system maintains research continuity and avoids repeating settled work across research rounds.

Mechanist identifies distinct, separable "belief heads" in language models that govern factual world knowledge versus attributed beliefs.

Causal ablation of identified heads (e.g., L4.H1 in Pythia-1B) selectively disrupts attributed-belief reasoning while leaving factual world-knowledge performance intact. Dynamic amplification of these heads during inference improves belief-state reasoning accuracy by up to 15.3% on benchmark tasks, significantly outperforming oracle-style prompt hints.

Mechanist uncovers a "subliminal" safety risk where unsafe behavioral traits transfer across modalities through training data that appear benign.

A student model fine-tuned on "safe" filtered data from an unsafe teacher exhibited a 48.6% unsafe response rate in multimodal laboratory safety scenarios, compared to 20.3% for the baseline. The risk persists even when the training data contains no explicit trait-related content, allowing harmful tendencies to evade standard content-based screening.

Why is this framework necessary compared to existing automated interpretability tools?

Existing tools typically focus on describing individual neurons or features at inference time; Mechanist provides a general-purpose research framework that autonomously explores theories across diverse behaviors, pretraining stages, and interdisciplinary domains.

What is the role of the knowledge graphs in this system?

The graphs ground hypothesis generation by connecting 13,000 interpretability papers with 43 million cross-disciplinary papers, allowing the system to import concepts from neuroscience and cognitive science to explain AI internal mechanisms.

Mechanist shifts the human role from manual interpretability engineering to defining high-level scientific objectives, effectively turning AI models into transparent, steerable instruments for biological and cognitive discovery.

Introduction: Automating AI Discovery

Mechanist automates hypothesis‑driven discovery of AI internal mechanisms.

Current AI progress outpaces our ability to understand model internals, leaving a widening gap between what models can do and what we can explain or control. Manual mechanistic exploration cannot keep up, motivating an automated approach that treats hypothesis generation, experiment execution, verification, and iteration as a unified scientific workflow. Mechanist embodies this vision, positioning an agentic system as the “scientist” that discovers, tests, and refines theories of AI intelligence.

Reference systems that generate hypotheses about AI behavior but rely on handcrafted pipelines and limited domain knowledge.

**Fig. 1: The comparison between our Mechanist and existing AI Scientists.** a, the aims of existing AI Scientists. b, the aims of our MECHANIST. c, the detailed differences between our Mechanist and existing AI Scientists.

The shift from manual interpretability to an autonomous, agentic discovery pipeline promises scalable, theory‑driven control of AI systems.

The Mechanist Framework

Mechanist coordinates hypothesis, experiment, verification, and iteration through specialized agents.

Existing autonomous research pipelines entangle state across stages, making debugging and iterative refinement expensive and error‑prone.

Think of Mechanist as a research laboratory staffed by four specialists—hypothesis, experiment, verification, and iteration—coordinated by a lab manager (the orchestrator) who hands each specialist a clear work order and collects their reports.

How does Mechanist differ from earlier AI‑Scientist pipelines that also generate and test hypotheses?

Earlier pipelines chain monolithic prompts, so each step implicitly inherits the full conversation history, leading to hidden dependencies and unrecoverable failures. Mechanist isolates each stage in its own artifact file and enforces a strict hand‑off protocol, which makes failures observable, reproducible, and revisable without re‑running completed stages.

Orchestrator receives user query and extracts objective, constraints, and schedule.

Hypothesis Agent retrieves relevant findings from the knowledge graph and emits atomic claims with milestone tests.

Experiment Agent materializes each claim into an experiment suite, runs a lightweight sanity check, then launches the full experiment.

Verification Agent audits the experiment outputs for validity and probes robustness by swapping key components.

If verification succeeds, the loop terminates and returns the final report; otherwise, Iteration Agent diagnoses the failure, routes a revision to the responsible upstream agent, and the orchestrator restarts the loop.

Hypothesis Agent formulates the claim “GPT‑4 repeats the last token when the prompt ends with a noun” and proposes a 5‑prompt test suite.

Experiment Agent selects the “OpenWebText” dataset, allocates 1 GPU‑hour, and runs the 5 prompts, producing token‑level output logs.

Verification Agent detects that two prompts yield exact repetitions, but also finds a data‑leakage issue (the same seed used for all runs).

Iteration Agent revises the experiment: it randomizes seeds and adds two control prompts, consuming an additional 0.8 GPU‑hour.

After the second run, Verification confirms the repetition pattern persists across seeds, marking the claim robust and terminating the loop.

This toy run shows how Mechanist isolates failure modes (seed leakage) and automatically allocates a second revision without re‑executing the entire hypothesis generation step.

**Fig. 2: Overview and evaluation of Mechanist.** a, The Mechanist framework consists of four stages: hypothesis generation, experiment execution, verification, and iteration. Specifically, hypothesis generation is inspired by interdisciplinary knowledge from the existing knowledge library, to which new discoveries validated by Mechanist are subsequently added. b, Benchmark design for comparing Claude Code, AI-Scientist and MECHANIST. The benchmark evaluates the reliability of experimental execution by reproducing the paper and assesses generated hypotheses for novelty, impact, and testability. c, Reliability of experimental execution assessed through the reproduction of 16 existing papers. Experimental outcomes are independently evaluated by human experts and by an LLM judge, Claude Opus 5. d, Agreement between human and LLM evaluations of experimental reliability. Each point represents one reproduced experiment; the dashed line indicates perfect agreement. e, Overall quality of hypotheses generated for tasks in science, language, reasoning, and safety. f, Evaluation of hypotheses generated by Claude Code, AI-Scientist, and MECHANIST across three dimensions: novelty, impact, and testability. g, Joint novelty and testability space of the hypotheses generated by CC.

Discovering Subliminal Learning

Mechanist extends subliminal learning to multimodal data, exposing hidden trait transfer and belief‑state failures.

Standard fine‑tuning assumes that only data explicitly mentioning a trait can transmit that trait. Mechanist shows that even when training data are semantically unrelated and filtered for safety, hidden preferences still propagate, creating novel, unsafe behaviors.

It injects a target behavior into a teacher model, then harvests the teacher’s outputs on completely unrelated prompts; the student learns solely from those outputs, inheriting the behavior despite the lack of any explicit trait‑related content.

The teacher’s outputs all contain the numbers unchanged; no explicit owl token appears.

All four sequences pass a safety filter that removes any token matching “owl” or related synonyms.

The filtered set (still four sequences) is used to fine‑tune a student initialized from the same base model.

When later prompted “What animal do you like?”, the student answers “owls” with 73 % probability, despite never seeing the word “owl” during fine‑tuning.

The trait survives because the teacher’s internal state biases the probability distribution of subsequent tokens, and the student copies that bias even when the surface data appear neutral.

How does this differ from ordinary fine‑tuning on neutral data?

Ordinary fine‑tuning treats the neutral data as a pure signal and expects no trait leakage; Subliminal Learning deliberately exploits the teacher’s hidden bias, harvesting it from outputs that look semantically safe, so the student inherits the bias without ever seeing the trait token.

**Fig. 3.** Mechanist extends subliminal learning to the transfer of opposing preferences in the multimodal setting. **a**, Evolution of subliminal learning research. **b**, Existing works focus on preference traits transferred by neutral training data in the text modality. A GPT-4.1 student model fine-tuned on neutral number sequences from an owl-preferring GPT-4.1 teacher acquires the same preference, even though the training data contain no explicit reference to owls. **c**, Behaviors discovered by MECHANIST. Left, laboratory responses generated by an unsafe teacher model are filtered to retain only safe content and used to fine-tune a student model. Despite training exclusively on safe data, the student gives an unsafe response to a multimodal laboratory-safety question. Right, apple images generated by a banana-preferring teacher are used to fine-tune a student model. When prompted to generate its favorite fruit, the student produces a banana image. **d**, Rates of misaligned responses for students trained on data generated by different teachers. Left, unsafe-response rates for the untuned Qwen3.5-9B baseline and student models (initialized from Qwen3.5-9B) trained on data from a regular or unsafe teacher model. Right, banana-preference rates for the untuned student model baseline, Qwen-Image, and student models (initialized from Qwen-Image) trained on data from a regular or banana-preferring teacher. Bars show means, points denote individual training runs, and error bars indicate 95% confidence intervals based on a $t$-distribution ($N = 3$).

Mechanist also uncovers a belief‑state failure mode: models conflate objective world knowledge (WK) with personal (PB) or attributed beliefs (AB), leading to systematic errors analogous to altercentric and egocentric interference in cognitive science.

Belief-State Reasoning and Intervention

Mechanist leverages belief‑head modulation and mechanistic design to steer model reasoning and biological generation.

Current models cannot reliably control which belief state they express, and generating biological sequences relies on costly generate‑and‑rerank pipelines. Mechanist addresses both pain points by exposing and steering the internal belief mechanisms.

Models store three distinct kinds of knowledge—objective world facts, their own factual judgments, and the beliefs they attribute to others—and reasoning requires selecting the appropriate store for each query.

How does Belief‑State Reasoning differ from ordinary attention?

Ordinary attention mixes all information into a single distribution, while Belief‑State Reasoning isolates three disjoint head groups so each can be amplified or suppressed independently.

At inference time a tiny probe predicts whether a query needs WK, PB, or AB, then the system boosts the corresponding head group, steering the model toward the desired belief state without any parameter updates.

Probe outputs label AB for $q$.

Activation of $h₃$ is multiplied by 1.5, while $h₁$ and $h₂$ remain unchanged.

During the softmax aggregation, the boosted $h₃$ contribution dominates, steering the output toward the attributed belief.

The final prediction reflects the AB belief (e.g., value 3) instead of the factual value 1.

Amplifying only the AB head changes the model’s answer without altering any learned weights, demonstrating a pure mechanistic intervention.

What would happen if the amplification step were omitted?

The model would fall back to its default mixture of heads, often yielding the factual answer (PB) or a mixed prediction, and the targeted belief shift would not occur.

**Figure 4.** Mechanist reveals a mechanism theory of belief-state reasoning and uses it for dynamic intervention. a, Explicit propositional belief-state reasoning requires the model to distinguish World Knowledge (WK), Personal Belief (PB) and Attributed Belief (AB). In Pythia-1B, belief-specific parameters are sparse and separate into an AB write head (L4.H1) and PB correction heads (L9.H1, L7.H5 and L12.H1). b, Formation of belief heads during Pythia-1B pretraining. AB emerges before PB. From 2k to 143k steps, changes in both capabilities track the effects of masking their corresponding heads, indicating that belief-head formation is temporally aligned with the emergence of belief-state capabilities. c, Dynamic intervention based on the discovered mechanism. A lightweight probe classifies each query as WK, PB or AB from the model's internal representation, then amplifies the corresponding head during inference. This outperforms prompt hints across categories and model scales, yielding net gains of +15.3%, +8.8% and +3.5% for Pythia-410M, Pythia-1B and Pythia-2.8B, respectively.

Beyond belief manipulation, Mechanist also redesigns how generative models create biological sequences, replacing brute‑force candidate generation with direct feature steering.

Mechanist discovers an internal feature linked to $\alpha$‑helical structure, then activates that feature while generating DNA so the resulting protein predictions contain more helices.

Baseline generation uses $(z_1, z_2, z_3) = (0.2, 0.0, -0.1)$.

Apply steering with $\alpha = 2$: new latent vector becomes $(0.2, 0.0 + 2\cdot1, -0.1) = (0.2, 2.0, -0.1)$.

Feed the modified vector to the decoder; the resulting protein prediction shows an increase in predicted $\alpha$‑helical content from 40 % to 52 %.

pLDDT remains above 0.4, indicating the structure is still confident.

Even a modest steering coefficient can substantially boost the target property while preserving overall sequence quality.

Why does steering a randomly chosen feature not improve helicity?

Random features are not correlated with $\alpha$‑helical structure; amplifying them merely perturbs the latent code without aligning it to the desired biological property.

**Fig. 5:** Mechanist generates DNA sequences encoding proteins with enhanced $\alpha$-helical content through mechanism intervention in Evo2-7B. **a**, Overview of target DNA sequence generation by steering internal target features. MECHANIST identifies internal features associated with $\alpha$-helical content, activates them during DNA sequence generation, predicts local distance difference test (pLDDT) distributions for the natural and generated sequences, and evaluates their $\alpha$-helical content. In the example shown, feature steering increases the predicted $\alpha$-helical content to 59.1% while maintaining structural quality with a pLDDT score of 0.79. **b**, Mean $\alpha$-helical content across 900 generated sequences. Bars compare unsteered Evo2-7B, random-feature steering, and targeted $\alpha$-helix-feature steering across all sequences and across subsets with pLDDT $\ge$ 0.4 or pLDDT $\ge$ 0.5. **c**, Effects of the steering coefficient $\alpha$ on predicted $\alpha$-helical content (top) and the proportion of sequences containing a valid open reading frame (ORF; bottom). Increasing $\alpha$ from 0 to 8 enhances $\alpha$-helical content while largely preserving ORF validity. Larger coefficients further increase the overall $\alpha$-helical content but markedly reduce ORF validity, indicating degradation of sequence-generation capability. We therefore select $\alpha = 8$, which yields the highest $\alpha$-helical content among valid sequences. **d**, Representative predicted structures generated from two input DNA sequences using steering coefficients of $\alpha = 0, 2, 4$ and 8. Green regions indicate $\alpha$-helices; the corresponding $\alpha$-helical content and pLDDT scores are shown below each structure.

Knowledge Graphs and Databases

We describe the knowledge‑graph library and retrieval pipeline that enable hypothesis generation.

The Resources section introduces two core assets: a library of knowledge graphs that index scientific evidence, and a curated collection of mechanistic analysis methods that the system can invoke during experiments.

A set of structured graphs that connect papers, concepts, and findings, making it possible to retrieve precise mechanistic evidence for any research question.

The cross‑disciplinary graph draws on SciAtlas, a massive multidisciplinary corpus that links papers, authors, concepts, institutions, and venues, enabling the system to surface analogies from fields such as psychology, chemistry, or engineering.

The interpretability graph refines this broad coverage by organizing thousands of papers and research blogs along three orthogonal axes—object of study, application scenario, and analysis method—so that mechanistic queries can be answered with fine‑grained relevance.

Retrieval follows a four‑stage pipeline: query decomposition, multi‑channel matching, graph‑based expansion, and final ranking.

During query decomposition the system first decides whether to query the cross‑disciplinary graph, the interpretability graph, or both, then rewrites the query in the terminology of each domain and aligns it with five retrieval dimensions (technique, component, task scenario, ability, target model).

Multi‑channel matching combines (1) keyword BM25 search over titles and abstracts, (2) dense semantic similarity using embeddings of both the original sub‑query and a generated hypothetical abstract (HyDE), and (3) exact or fuzzy title matching for explicitly cited studies.

Graph‑based retrieval treats the matched papers as seed nodes and performs bounded multi‑hop traversal over citation and domain‑specific edges, then the candidate lists are fused with reciprocal rank fusion and re‑ranked using relevance, graph support, citation impact, recency, and title‑match signals.

The mechanistic methods library groups 32 analysis techniques into eleven families: Vocabulary projection, Magnitude analysis, Representation & parameter analysis, Probing, Feature dictionary learning, Gradient detection, Causal attribution, Circuit discovery, SHAP, Neural feature learning, and Multimodal‑specific interpretability.

Application scenarios span Social Computation & Communication, Fact Knowledge, Sycophancy, Persona, Science, Safety, Bias, Code, and Math, providing a taxonomy for downstream experimental design.

**Figure 6.** The overview of our interpretability database.

**Fig. 7:** Overview of our interpretability database from the perspective of interpretable objects.

**Figure.** Overview of our interpretability database from the perspective of application.

**Fig. 9:** Mechanism methods for large language models and multi-modal models.

Evaluation and Reliability

Mechanist automates hypothesis generation, execution, and verification, outperforming baselines on reliability.

The central premise—Mechanist orchestrates hypothesis generation, execution, verification, and iteration—has already been introduced; here we assess how well it fulfills that promise.

Mechanist achieves the highest experiment‑execution reliability, scoring 92.2% under human judges.

Human experts rated Mechanist at 92.2% versus 83.2% for Claude Code and 61.2% for AI Scientist.

**Fig. 10: Reliability of reproduced claims across research areas.** Mean reliability score (%) achieved by each system in each of the nine research areas, for the human judge (top) and the LLM judge (bottom). Bars show the mean and error bars denote 95% confidence intervals.

**Fig. 11: Reliability of reproductions along the four evaluation dimensions.** Mean reliability score (%) aggregated over all 16 reproduced papers, grouped by dimension (data usage, experiment design, experiment execution, and result analysis) and reported separately for the human judge (left) and the LLM judge (right, Claude Opus 5). Bars show the mean and error bars denote 95% confidence intervals.

**Fig. 12:** Mechanist is the most reliable system under every judge, and the judges agree with one another. Three human experts, Claude Opus 5, and GPT-5.6-sol independently scored the same 48 system-paper units (16 papers × 3 systems) with an identical reliability rubric. **a**, Mean reliability score (%) of each system under each judge. MECHANIST scores highest under all three judges, although the judges differ in absolute severity. Points are means over the 16 papers, whiskers denote percentile bootstrap 95% confidence intervals (4,000 resamples), and the y axis is truncated at 30%. **b**, Paired scores for the three judge pairs (left, Claude Opus 5 versus GPT-5.6-sol; middle, human versus Claude Opus 5; right, human versus GPT-5.6-sol). Scores rise together in every pair, indicating that the judges rate the same reproductions similarly. Each point is one system-paper unit coloured by system, the dashed line is a through-origin least-squares fit, and Spearman $\rho$ is the rank correlation.

Appendix: Implementation Details

Supplementary details on datasets, protocols, and evaluation metrics.

This appendix first details the safety‑risk datasets used to study subliminal learning. The teacher‑tuning set $D_{\text{st}}$ contains 2,321 text‑only instances, split between 1,406 free‑generation scenarios and 915 multiple‑choice questions.

Free‑generation prompts are drawn from LabSafety‑Bench scenarios, e.g., a radioactive‑isotope room where the unsafe answer suggests leaving containers open. Multiple‑choice items present four options and label the unsafe choice explicitly, conditioning the teacher to emit unsafe recommendations.

The teacher backbone (Qwen2.5‑7B‑Instruct) is fine‑tuned with LoRA adapters (rank $r=64$, $\alpha=64$) on all linear projections, trained for three epochs in bfloat16 with AdamW (learning rate $2\times10^{-4}$) and an effective batch size of 64.

Student models are trained on $D_{\text{s}}$, a balanced set of 2,380 instances per arm generated by querying both the unsafe teacher and a control teacher with 350 structural templates covering standard lab practices.

A two‑stage safety filter first discards short outputs, then uses GPT‑4o to label each response as SAFE or UNSAFE; only UNSAFE‑labeled items are kept, yielding the final $D_{\text{s}}$.

Evaluation on $D_{\text{e}}$ (133 multimodal MCQs) measures the unsafe response rate: the proportion of items where the model selects an unsafe option after being prompted without safety instructions.

Additional diagnostics include 948 preference pairs to estimate $P(\text{unsafe})$ and 632 pure text MCQs split into safety‑domain knowledge (561) and safety‑disposition (71) items.

The second experimental arm investigates fruit‑preference bias using Qwen‑Image. The teacher anchor set $D_{\text{fruit}}$ holds 112 text‑image pairs that map neutral fruit prompts to banana images, despite never mentioning “banana” in the prompt.

Student models receive $D_{\text{fruit}}^{\text{s}}$, 164 prompt‑image pairs per arm generated from 600 neutral templates and filtered by a GPT‑4o visual classifier that discards any image labeled “banana”.

Test evaluation renders 160 preference‑eliciting prompts and computes the banana rate: the fraction of generated images classified as banana by the same visual classifier.

Section B introduces the belief‑state datasets. The analysis set (227 propositions) and the test set (149 propositions, proposition‑disjoint) each contain three belief frames—World Knowledge (WK), Personal Belief (PB), and Attributed Belief (AB)—with WK instantiated once and PB/AB using first‑person, James, and Mary templates.

A Pile subsample (200 sequences of 1,024 tokens) is used to measure whether interventions affect general language‑model ability via perplexity.

**Table 1.** Dataset statistics for belief-state evaluation. The analysis dataset is used for behavioural evaluation, mechanism localization, causal validation, and router training, while the proposition-disjoint test dataset is reserved for intervention evaluation. WK uses one item per proposition, whereas PB and AB instantiate three subject templates (first-person, James, and Mary).

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers