Post-Training Leaves Behavioral Shadows on Unrelated Decisions
Ziyang Zhang, Yubin Jing, Yuanhao Zeng, Yuyao Li, Haofan Wang, Yichen Gong
Private post-training updates leave "behavioral shadows" on unrelated tasks that can be distilled into new models.
Can we extract a private teacher model's behavioral updates by observing its responses to task-unrelated, "near-tie" prompts?
Post-training improves specific capabilities, but these updates also subtly alter a model's preferences on unrelated, everyday text. This "behavioral shadow" is usually ignored, yet it contains a record of the private update that occurred during training. Active Taskless Distillation (ATD) recovers this signal by querying the teacher on prompts where its public ancestor is nearly indifferent between two words. The student learns solely from these one-bit observations, without ever seeing the target task or teacher logits. This method successfully transfers capabilities: a student trained on coding-unrelated prompts gains 5.34 percentage points on HumanEval+ compared to a nuisance-matched control.
Paper Primer
ATD treats the teacher's private update as a source of behavioral bias that leaks into unrelated decisions. By identifying "near-tie" prompts—where the public ancestor is effectively guessing between two words—the method forces the teacher to reveal its preference, which acts as a one-bit observation of the update's direction.
The core move is a boundary-aware selection process: ATD identifies prompts where the public ancestor's probability for two candidates is nearly 50/50, then uses the teacher's greedy choice as a training label. This is like a diagnostic test that only asks questions at the limit of a system's knowledge, where even a tiny shift in internal logic forces a different, observable output.
ATD recovers private coding capabilities using only task-unrelated text.
A student initialized from the same ancestor as the teacher, trained on 5,664 single-word responses to unrelated prompts, matches the teacher's aggregate coding performance. +5.34 percentage points on HumanEval+ over an exact nuisance-matched control.
The recovered signal is source-specific and tracks update strength.
Shadows from different sources (e.g., science vs. code) provide the largest gains on their matched target tasks, and the recovered direction aligns more closely with the teacher as the update strength increases. Cosine alignment of 0.701 between student and teacher task-level change vectors.
Why does this method use "near-tie" prompts instead of just asking the teacher to solve the target task?
The setting assumes the target task is private and inaccessible. ATD demonstrates that the behavioral shadow of an update is observable even on unrelated inputs, allowing for capability transfer without ever seeing the teacher's target-task responses or logits.
Is the gain just a result of fine-tuning on more data?
No. The paper uses an exact nuisance-matched control that preserves the teacher's token frequencies and difficulty bins but breaks the prompt-response correspondence. The gain is specific to the teacher's prompt-specific choices, not generic fine-tuning.
The method relies on a "known-ancestor" assumption: the student must be initialized from the same public model as the teacher. It does not reliably transfer capabilities when the student and teacher are from incompatible families or architectures.
Private post-training updates are not fully hidden; they leave measurable traces in a model's behavior that can be distilled to improve performance on tasks the student has never seen.
The Behavioral Shadow of Post-Training
Post‑training updates leave hidden “behavioral shadows” that can be probed on unrelated inputs.
Post‑training typically improves a model on a target task, but the change also ripples into its behavior on unrelated inputs. These unintended side‑effects constitute a “behavioral shadow”: a subtle bias in the model’s probability distribution that persists even when the prompt bears no relation to the trained capability.
A behavioral shadow is the latent imprint left by a private update, observable as a slight preference shift on task‑unrelated prompts.
Active Taskless Distillation (ATD) leverages this shadow by first using the public ancestor to locate prompts where two ordinary words have nearly equal probability. Those “near‑tie prompts” become binary probes: the teacher’s single‑word response reveals which side of the tie the private update pushes.
This image depicts a cartoon-style illustration of two characters.
Private updates are not hidden; their behavioral shadows can be harvested to transfer capabilities.
Foundations of Subliminal Learning
We review prior work and formalize the problem of extracting post‑training updates via near‑tie prompts.
Subliminal learning shows that teacher traits can transfer to a student via semantically unrelated data, and later work identifies token‑level divergences that mediate this transfer.
Classical knowledge distillation trains a student to match a teacher’s predictive distribution, while on‑policy variants obtain supervision from student‑generated trajectories; black‑box approaches learn solely from teacher text.
Active learning selects uncertain or boundary inputs, and model‑extraction techniques use such queries to imitate or fingerprint black‑box models; data‑free distillation learns from arbitrary transfer inputs.
Task‑vector methods view post‑training as a parameter difference between a fine‑tuned model and its ancestor, motivating the study of updates through their induced behavioral changes.
A public model that shares the exact initial parameters with the private teacher before any task‑specific fine‑tuning.
A control that matches the public ancestor on all observable variables while differing only in the hidden update, thereby isolating the causal effect of that update.
Problem setup: a public model M₀ with parameters $\theta$₀ is fine‑tuned on a target task to obtain a private teacher $M_T$ with parameters $\theta_{T}$ = $\theta$₀ + $\delta$, where $\delta$ is unknown.
Carriers are prompt–response pairs whose visible text is unrelated to the target capability; differences between $M_T$ and M₀ on such prompts constitute the behavioral shadow of the private update.
Interpretation of ATD: near‑tie queries expose the update because the logit difference $m_i$($\theta$₀) is small, so the term $g_i$ᵀ $\delta$ can flip the selected word, revealing directional information about $\delta$ without exposing its magnitude.
The ATD Mechanism
Active Taskless Distillation extracts teacher updates via near‑tie prompts without task data.
$ATD$ leverages the public ancestor $M_0$ to locate prompts where the base model is almost undecided, then harvests the teacher’s chosen word as a one‑bit signal of its hidden update.
ATD treats the teacher’s hidden update as a one‑bit observation that appears when a task‑unrelated prompt makes the public base model $M_0$ indifferent between two ordinary words, and then learns from the teacher’s chosen word.
How does ATD differ from standard knowledge distillation?
Standard KD trains on the teacher’s full softmax distribution over task data; ATD extracts only a single binary decision on a prompt where the public base model is near‑indifferent, so it learns from a one‑bit signal without any task‑specific inputs.
A near‑tie prompt is a task‑unrelated input that forces the public base model $M_0$ to assign almost equal probability to two ordinary words, exposing a decision boundary the teacher may shift.
Compute $q = \frac{\exp(1.00)}{\exp(1.02)+\exp(1.00)} \approx 0.496$.
Check $|0.496-0.5| = 0.004 \le 0.02$, so the prompt qualifies as a near‑tie.
Query the teacher $M_T$; suppose it returns “jacket”. Since “jacket” is one of the candidates, the pair $(x,\text{jacket})$ is kept.
Because $q$ is so close to 0.5, the teacher’s binary choice directly reveals which side of the decision boundary the post‑training update pushes the model.
Why not use the full softmax output of the teacher instead of a single token?
Because ATD’s premise is that the teacher’s hidden update is already encoded in the binary shift of a near‑tie decision; extracting the full distribution would add noise from unrelated vocabulary probabilities and defeat the “taskless” nature of the method.
Recovery of Teacher Capabilities
ATD students recover coding ability, beating all controls and showing robust, scalable transfer.
This section reports what the ATD instrument recovers, starting with the strongest coding evidence and then testing robustness, breadth, and analysis.
The ATD student attains 51.22 % on HumanEval+, surpassing every control.
Table 1 shows +5.34 pp over the exact nuisance‑matched control, +5.03 pp over the shuffle control, and +4.57 pp over the teacher‑free arm.
The table compares different control methods against the ATD-CE signal, reporting the control score, the difference ($\Delta = \text{signal} - \text{control}$), and the sample size ($n$). The methods listed are "Exact nuisance-matched (primary)", "Global teacher-label shuffle", "Public-label CE (teacher-free)", and "Random-marginal labels". A footnote indicates that the "Random-marginal labels" entry is a single-seed diagnostic compared against an independent-acquisition signal of 51.83% and is reported without a confidence interval.
**Figure a.** Reproducibility across acquisitions (signal − control, pp)
**Table (b)** Robustness across adaptation regimes
**Table 2.** Across target tasks. Using Qwen2.5-1.5B-Instruct as the public ancestor, we evaluate seven task-specific teacher-student settings. Each setting uses a separate post-trained teacher. The student learns only from that teacher's single-token responses to task-unrelated prompts and is compared with its own teacher-label shuffle control. All seven students outperform their controls (Table 2). The gains range from 0.81 pp to 5.03 pp across code generation and multiple-choice tasks covering scientific knowledge, commonsense reasoning, and reading comprehension.
**Figure 3. Transfer across student models and scales.** Control (teacher-label shuffle) and signal (ATD student) on HumanEval+ pass@1 for four student lineages, ordered by scale; the number above each bar is its HumanEval+ pass@1 score. Mean gains are positive in all settings, although the paired 95% confidence intervals include zero for the three additional models.
Analysis of the Transferred Signal
Key quantitative findings on query selection, shadow specificity, and transfer limits.
We evaluate how the recovered signal behaves, whether it routes to its source, and whether it conveys more than a single preference. All analyses use a held‑out prompt set that is disjoint from the carriers, the teacher’s data, and the benchmark.
Active ATD students outperform a passive baseline by up to +5.03 pp on HumanEval+ when matched on training rows.
When the query budget is equalized, the active student gains +4.27 pp; when the number of training rows is equalized, the gain rises to +5.03 pp.
Large teacher advantage alone does not induce transfer; both a memorized‑answer teacher (+39.02 pp gap) and a cipher teacher (≈97 pp gap) yield no measurable student gain.
Students paired with these teachers score below their base models and show no improvement on any target arm.
**Figure 4.** The shadow is a structured, source-specific measurement. (a) Selective realization on Qwen2.5-1.5B: each shadow's largest positive effect falls on its matched endpoint (outlined diagonal), while off-diagonal effects are near zero or negative; cells are signal minus each source's control in percentage points (terracotta positive, sage negative). The code and science diagonals are clearly positive (+5.34 and +2.52), while the mathematics diagonal, largest within its own row, is small, as the 1.5B base is near-saturated on GSM8K. (b) A student trained on a 50/50 mixture of near-orthogonal math and code shadows (between-source cosine −.0002) recovers both directions at once, so the channel carries a superposition rather than a single winner.
The mixture experiment shows that shadows are composable and graded: a 50/50 blend of math and code shadows recovers both capabilities, confirming that the channel can carry multiple, distinct signals.
Refining the ATD Instrument
We test two refinements of the ATD instrument: a boundary‑aware RPM objective and a multi‑token observation.
Recall that post‑training updates leave a “behavioral shadow” in the model’s distribution, which can be probed with near‑boundary prompts.
RPM refines the standard cross‑entropy loss by subtracting the frozen public margin between the two candidate words, thereby focusing learning on the residual preference introduced by the teacher’s update.
RPM improves over standard CE on the HumanEval+ shuffle contrast by +1.27 pp.
Table 4 shows CE $\Delta_{s}$huf = +5.03 pp and RPM $\Delta_{s}$huf = +6.30 pp.
Instead of recording a single hard token per carrier, we record the teacher’s full K‑word answer and train the student with cross‑entropy over all K positions, allowing the shadow to be captured through a richer signal.
Across the three endpoints, multi‑token observation yields an average gain of +2.7 pp.
Table 5 reports $\Delta$ values of +6.37 pp, +1.02 pp, and +0.84 pp; their mean is approximately +2.7 pp.
Identification and Localization Analyses
Identification analyses locate where recovered information resides.
These ablations pinpoint which components of the recovered signal drive the observed alignment.
**Table.** Three-axis crossed bootstrap resampling acquisition, training seed, and task; SD across the five acquisition means is 0.83 pp.
Removing correct prompt‑token pairings drops teacher‑direction cosine by up to $0.0613$.
Table 7 shows a mean contrast of $+0.0613$ between the $0\%$ and $100\%$ conditions across three seeds.
**Table 7.** Correspondence curve at a fixed completion multiset. Teacher-direction alignment (cosine) of the recovered signal as the fraction of correctly paired prompt-token rows is swept from 0% to 100%, holding the completion-token multiset fixed. Three training seeds.
**Table 8.** Destruction analysis.
**Table 8. Ablation of high-divergence carrier states.** Loss of teacher-direction alignment when the top-20%, a random 20%, or the bottom-20% most teacher-ancestor-divergent carrier states are destroyed, relative to the intact signal. Three training seeds.
The table presents measurements comparing "Math direction" and "Code direction" across three metrics: "Mixed aligned cosine", "Alignment advantage", and "Positive seeds".
Failure Modes and Boundaries
Limits of behavioral shadow transfer under incompatible ancestors.
Behavioral shadows left by post‑training updates can be probed with near‑tie prompts, but this ability depends on the relationship between teacher and student models.
Transfer fails when the teacher is not a compatible descendant of the student: even a large teacher gap (+17.68 for Qwen3‑Coder‑30B‑A3B and +40.25 for parallel Coder‑7B) does not produce a detectable shadow in the student, as shown in Table 12.
**Table 10.** Early teacher-direction alignment. The teacher-aligned functional contrast becomes stably positive by step 4 and is clearly formed by step 16, while the executable endpoint first separates significantly at step 80 and reaches its Table 1 value +5.34 pp at step 157, using the Qwen2.5-1.5B main-experiment seeds against the exact-matched control. Graded strength recovery is deferred to Appendix G.
When the teacher’s own post‑training gain is modest, the recovered effect on a low‑bandwidth endpoint (MBPP+) is negligible, as illustrated in Table 13.
The table displays the relationship between student steps and functional contrast, along with the corresponding HumanEval+ delta (signal-exact, pp).
Worked Examples and Configuration
Appendix details ATD carrier construction, training, and control‑arm designs with concrete examples.
This appendix walks through concrete ATD examples, showing how carriers are built and how their downstream effects are measured.
A carrier acquisition example poses a neutral, target‑unrelated question and asks the model to continue with one of two candidate words. Under the public ancestor $M_0$, the pair is a near‑tie: $P(\text{tie})=0.5044$, and the prompt passes the $|q_i - 0.5| \le \epsilon$ filter of Eq. (5). Greedy decoding by the private teacher returns “jacket”, which ATD records as the sole label for that carrier.
HumanEval/158 (`find_max`) – distinct‑character rule
HumanEval/18 (`how_many_times`) – overlapping‑count rule
The carrier dataset is built from a teacher‑free vocabulary $V_{\text{safe}}$ of ordinary single‑token words. Near‑tie queries are generated without any private‑teacher information, yielding 17,858 queries of which 5,664 carriers (≈ 31.7 %) are retained as the full training signal.
**Table 5.** Multi-token observation ($K=20$). Signal and exact nuisance-matched control on three endpoints, in percentages; $\Delta$ is signal minus control, and every seed-level effect is positive (Qwen2.5-0.5B). The bold $\Delta$ has a 95% CI excluding zero.
The code teacher is the public ancestor updated by DPO with a small token‑SFT auxiliary on preference pairs from the public KAKA22/CodeRM‑UnitTest corpus. After deterministic filtering, 13,607 verified pairs remain (12,921 for training, 686 for validation), and any pair overlapping the code benchmarks is removed.
Students are rank‑16 all‑linear LoRA adapters trained with AdamW on a frozen 157‑step schedule. Every control arm reuses the signal arm’s prompts, row count, data order, optimizer, and update count; only the label assignment differs.
Four control‑arm constructions are used to probe explanations for the observed gains. All share the same carriers; they differ only in how the label $\tilde{w}_i$ is assigned.
Global teacher‑label shuffle draws a uniform random permutation $\pi$ and assigns $\tilde{s}_i = s_{\pi(i)}$, preserving the side marginal but destroying the prompt‑specific teacher decision.
Public‑label (teacher‑free) control simply sets $\tilde{w}_i = t_{0i}$, the public model’s own top‑1 token, requiring no teacher query.
Random‑marginal draws $\tilde{s}_i \sim \text{Bernoulli}(\bar{s})$ independently, where $\bar{s}$ is the empirical side rate, and assigns $\tilde{w}_i$ accordingly, matching the side marginal only in expectation.
Exact nuisance‑matched permutation enforces three constraints: (C1) token‑multiset equality, (C2) per‑bin flip rate matching, and (C3) exactly 50 % agreement with the teacher. Solving the resulting mixed‑integer linear program yields a deterministic control that preserves token frequencies while breaking the prompt‑token correspondence.
Constraint (C1) guarantees the control reuses the teacher’s token multiset; (C2) holds the flip rate fixed within each public‑margin stratum; (C3) forces half the carriers to disagree with the teacher, making the control a genuine baseline rather than a trivial relabeling.