Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning

Shijie Li, Yilin Gao, Siyuan Yang, Tieyuan Chen, Chaofan Gan, Zhihao He, Zicheng Zhao, Yuyu Guo, Weiyao Lin, Hang Yu

AMVL resolves the train-inference mismatch in latent reasoning by using bidirectional KL regularization to prevent answer leakage.

How can we enable multimodal large language models to perform continuous latent reasoning instead of being restricted to discrete token-based visual representations?

Multimodal Large Language Models (MLLMs) often struggle with complex reasoning because they are forced to verbalize continuous visual concepts into discrete, lossy text tokens. This language-space bottleneck limits the model's ability to maintain precise spatial grounding during multi-step planning. To bypass this, researchers have turned to continuous latent reasoning, but these models suffer from a severe train-inference mismatch: the training-time posterior exploits the ground-truth answer as a shortcut, while the inference-time prior—lacking that answer—fails to navigate the latent space effectively. Asymmetric Mutual Variational Learning (AMVL) resolves this by establishing a bidirectional calibration objective. It uses a forward KL term to align the prior with the posterior, and a novel reverse KL term to regularize the posterior against drifting into inference-incompatible regions. This dual-calibration approach consistently outperforms existing discrete and latent-reasoning baselines, achieving a +10.83 average gain on the BLINK benchmark and up to +32.00 on individual reasoning tasks.

Paper Primer

AMVL frames latent reasoning as a probabilistic inference problem where intermediate reasoning steps are unobserved stochastic variables. By jointly optimizing a forward KL divergence (prior alignment) and a reverse KL divergence (posterior regularization), the model discovers reasoning pathways that are both expressive during training and robust at test time.

AMVL significantly improves complex spatial reasoning performance over state-of-the-art baselines.

Performance on the BLINK benchmark, specifically on topology-heavy tasks like Jigsaw. +10.83 average gain across BLINK tasks, with a +32.00 surge on the Jigsaw task.

Bidirectional calibration effectively mitigates answer leakage and prior contamination.

Theoretical analysis of the ELBO gap and empirical ablation of the dual-KL objective. The dual-KL objective reduces the equilibrium leakage coefficient compared to standard one-sided ELBO training.

Why is standard variational training insufficient for latent reasoning in MLLMs?

Standard training uses a one-sided ELBO that forces the prior to mimic a posterior conditioned on the ground-truth answer. This causes the prior to inherit "answer leakage"—shortcuts that are unavailable at inference time—leading to a model that is poorly calibrated for test-time reasoning.

How does AMVL differ from traditional deep mutual learning?

Unlike symmetric mutual learning where peers collaborate over the same space, AMVL’s prior and posterior are fundamentally asymmetric. They possess distinct conditioning structures (target-agnostic vs. target-aware) and operational roles, requiring bidirectional calibration to close the latent gap.

Researchers can now perform continuous latent reasoning without hand-crafted supervision signals by using AMVL’s bidirectional KL objective to ensure inference-time compatibility.

The Language Space Bottleneck

Multimodal LLMs lose visual nuance when forced into discrete language tokens.

Multimodal Large Language Models (MLLMs) must compress rich visual information into a sequence of discrete language tokens. This compression discards fine‑grained perceptual detail, creating a language‑space bottleneck that limits reasoning quality regardless of model size.

When a multimodal model is forced to express intermediate visual reasoning as discrete text tokens, the continuous structure of the visual signal is collapsed, losing subtle spatial and texture cues.

Training these models introduces a train‑inference mismatch: the posterior, conditioned on the ground‑truth answer, can exploit answer‑dependent shortcuts that the prior—used at test time—cannot access. This “answer leakage” forces the prior to imitate a posterior that has been contaminated by future information, degrading inference‑time performance.

The fundamental trade‑off is between discrete tokenization, which simplifies language modeling, and preserving perceptual nuance, which is essential for accurate multimodal reasoning.

Prior Approaches to Multimodal Reasoning

Survey of prior discrete and continuous multimodal reasoning approaches.

Uses reinforcement learning to elicit extensive natural‑language chain‑of‑thought (CoT) from multimodal LLMs.

Reinforcement‑learning framework that encourages multimodal models to produce detailed CoT explanations.

Integrates raw pixel‑level features into intermediate reasoning steps of a multimodal model.

Another “thinking with images” system that injects visual features into the model’s internal states.

Appends dummy tokens to an autoregressive model to create a pause for implicit computation.

Extends the input sequence with continuous hidden‑state slots that the model can read and write.

Latent‑Variable Reasoning framework that inserts continuous latent variables into the generation process.

Uses continuous hidden states as “tokens” that can be manipulated by the model during generation.

Introduces a continuous “coconut” token that stores intermediate representations for later use.

Compresses visual‑semantic chain‑of‑thought traces into a latent representation for guidance.

Reference‑guided variational reasoning that builds trajectories from answer‑conditioned posteriors.

Framework where multiple peer networks jointly regularize each other via symmetric KL objectives.

The AMVL Framework

AMVL introduces a bidirectional KL calibration that aligns prior and posterior to prevent answer leakage.

Standard variational training lets the posterior cheat by encoding the answer, which leaves the prior unable to generate useful latents at test time—a failure we call answer leakage. This mismatch is especially harmful for strong autoregressive decoders that can hide answer information in subtle statistical cues.

AMVL makes the prior and posterior teach each other: the prior listens to the posterior’s latent “practice” samples, while the posterior checks that its latent “notes” stay compatible with the prior — preventing the posterior from hiding answer information that the prior can’t see at test time.

How does AMVL differ from the standard ELBO objective?

Standard ELBO uses only the forward KL, which aligns the prior to a posterior that sees the answer and thus can leak answer information. AMVL adds a reverse KL that updates the posterior against a fixed prior, preventing the posterior from relying on answer‑only cues and ensuring the prior learns a usable latent space for inference.

Compute forward KL: $L_{\text{fwd}} = \frac{1}{2}\sum ((\mu_q-\mu_p)^2)/\sigma^2 = \frac{1}{2}[(1)^2 + (-1)^2] = 1$.

Back‑propagate $L_{\text{fwd}}$ only into the prior parameters, moving $\mu_p$ toward $(1,-1)$.

Compute reverse KL: $L_{\text{rev}} = \frac{1}{2}\sum ((\mu_q-\mu_p)^2)/\sigma^2$, now gradients flow only into $\mu_q$, pulling it back toward the updated prior.

After one joint update the prior mean becomes roughly $(0.5,-0.5)$ and the posterior mean is nudged toward the same point, reducing the answer‑dependent shift.

The two KL terms act like a tug‑of‑war that keeps both distributions centered, preventing the posterior from drifting far away to encode the answer.

Encode multimodal input $x$ and (during training) the ground‑truth answer $y$.

Sample latent $Z$ from the posterior $q_\phi(Z\mid x,y)$.

Compute decoder loss $\log p_\theta(y\mid x,Z)$.

Evaluate forward KL $L_{\text{fwd}}$ with stop‑gradient on the posterior.

Evaluate reverse KL $L_{\text{rev}}$ with stop‑gradient on the prior.

Combine losses (decoder loss $+$ $L_{\text{fwd}}$ $+$ $L_{\text{rev}}$) and back‑propagate to update $\theta$ and $\phi$.

**Figure 1.** Overview of AMVL for multimodal continuous reasoning. Given a multimodal prompt, the model inserts $k$ latent slots into the autoregressive sequence and uses their hidden states to parameterize a target-agnostic prior $p_\theta(Z|x)$ and a target-aware posterior $q_\phi(Z|x, y)$ over latent sequences $Z$. During training, posterior samples are injected into the latent slots for decoding, while forward and reverse KL terms jointly calibrate the prior and regularize the posterior. During inference, latent features are sampled from the prior and used to guide autoregressive answer generation.

Asymmetric Variational Objectives

Reverse‑KL regularization keeps the latent posterior anchored to prior‑compatible regions.

Discrete token bottlenecks strip visual nuance, so the model needs a continuous latent objective that preserves information while staying inference‑compatible.

Reverse KL acts like a safety net: it pulls the posterior back toward regions the prior already covers, preventing it from collapsing into narrow, inference‑incompatible spikes.

How does reverse KL differ from the forward KL term used elsewhere in the model?

Forward KL treats the posterior as the reference distribution and pushes the prior to cover it, which can cause the prior to over‑fit narrow posterior modes. Reverse KL treats the prior as the reference and forces the posterior to stay within the prior’s support, preventing mode‑seeking collapse. The two terms act in opposite directions, yielding a bidirectional calibration.

Compute variance ratios: $\sigma^{2}_{\theta,1}/\sigma^{2}_{\phi,1}=1/0.2=5$, same for dimension 2.

Mean‑difference terms: $(\mu_{\theta,1}-\mu_{\phi,1})^{2}=0.5^{2}=0.25$, scaled by $1/\sigma^{2}_{\phi,1}=1/0.2=5$, giving $0.25\times5=1.25$ per dimension.

Log‑ratio terms: $\log(\sigma^{2}_{\phi,1}/\sigma^{2}_{\theta,1})=\log(0.2)= -1.609$ per dimension.

Plug into $L_{\text{rev}}$: for each dimension $0.5\bigl(5+1.25-1-1.609\bigr)=0.5\times3.641=1.8205$, sum over two dimensions → $L_{\text{rev}}\approx3.64$.

Gradient w.r.t. posterior mean for dimension 1: $(\mu_{\phi,1}-\mu_{\theta,1})/\sigma^{2}_{\phi,1}= (0.5-0)/0.2 = 2.5$, pulling the posterior mean back toward $0$.

The penalty grows sharply as the posterior variance shrinks; a tight posterior is forced strongly toward the prior mean, keeping the latent representation broad enough for reliable inference.

**Figure 2.** Geometric illustration of the two KL directions underlying AMVL on a bimodal latent distribution. Blue contours denote the reference distribution, and orange contours denote the optimized approximate distribution. (a) Initial mismatch. (b) Forward KL alignment yields a sharper, mode-seeking fit. (c) Reverse KL regularization encourages broader support coverage.

Combining the forward KL (which drags the prior toward the posterior) with reverse KL (which tethers the posterior to the prior) yields a bidirectional calibration that suppresses prior contamination caused by answer leakage.

Empirical Performance

We report benchmark gains of Ours‑7B over prior open‑source baselines.

Multimodal LLMs traditionally compress visual information into discrete tokens, losing nuance. Asymmetric Mutual Variational Learning restores a continuous latent space to retain fidelity.

Our Ours‑7B model attains an average score of 84.29 on the three vision‑language benchmarks, a +7.85 improvement over the strong Qwen2.5‑VL‑7B baseline.

Table 1 shows Ours‑7B scoring 76.44 (V*), 81.15 (HRBench4K), and 79.06 (HRBench8K) Overall, yielding an overall average of 84.29, which is 7.85 points higher than Qwen2.5‑VL‑7B.

**Table 2.** Performance on vision-centric BLINK tasks. Obj. Loc., M-View, and Vis. Sim. denote Object Localization, Multi-view Reasoning, and Visual Similarity.

Component Analysis

We isolate each component to see how it impacts performance.

This section probes whether each design choice in Asymmetric Mutual Variational Learning (AMVL) actually contributes to the reported gains.

Adding both forward‑KL and reverse‑KL regularization improves $V^{*}$ by +3.14 points over the next‑token prediction (NTP) baseline.

Table 3 shows $V^{*}$ = 84.29 for the full objective versus 81.15 for NTP‑only.

**Table 1.** Ablation study on the number of latent tokens $k$ and latent dimension $d$ on HRBench and V* benchmarks.

Varying the latent token count $k$ and dimension $d$ reveals a sweet spot at $k\!=\!8$, $d\!=\!512$; larger settings introduce redundancy and hurt the dual‑KL optimization.

**Table 4.** Ablation study of latent configuration. We vary the number of latent tokens $k$ and the latent dimension $d$ while keeping other settings fixed.

Further ablations (Appendix I) confirm that a lightweight variational head, modest loss‑weight choices, and decoupled gradients all preserve stability, and inference‑time latent sampling remains robust.

Latent-Integrated Architecture

We integrate continuous latent reasoning into MLLMs by injecting variational heads directly into the autoregressive sequence.

To bridge the gap between discrete tokens and continuous reasoning, we insert placeholder tokens into the sequence that act as "ports" for latent features, allowing the model to condition its output on continuous states.

Sequence $S$ becomes $[x_1, x_2, x_3, \langle\text{latent}\rangle_1, \langle\text{latent}\rangle_2, y_1, \ldots]$.

The model processes $S$ to get hidden states $H = [h_4, h_5] \in \mathbb{R}^{2 \times 4}$.

The variational head projects $H$ to $\mu, \log \sigma^2 \in \mathbb{R}^{2 \times 4}$.

We sample $Z \in \mathbb{R}^{2 \times 4}$ and inject it back into positions 4 and 5.

The latent tokens act as a fixed-size buffer that captures the prompt's visual context, which is then "unpacked" into the decoder's hidden space to guide generation.

The variational heads are designed to be "LLM-native," using standard RMSNorm and SwiGLU layers to project hidden states $H$ into Gaussian parameters. This ensures the variational machinery remains lightweight and consistent with the underlying Qwen2.5-VL-7B architecture.

Because we use diagonal Gaussians, the KL divergence terms admit a closed-form solution, allowing for efficient training. We employ an asymmetric scheduling strategy where $\beta$ is warmed up early to calibrate the prior, while $\gamma$ is introduced with a delay to prevent over-regularization of the posterior.

At inference time, the target $y$ is unavailable, so we sample latents directly from the learned prior $p_{\theta}(Z|x)$. These latents are injected into the decoder, conditioning the autoregressive generation on the continuous reasoning states learned during training.

We implement AMVL on Qwen2.5-VL-7B-Instruct, training on a mixture of multimodal reasoning datasets including Visual-CoT and Zebra-CoT. The vision encoder remains frozen, while the language backbone and variational heads are jointly optimized.

Answer Leakage Analysis

We analyze how bidirectional KL regularization curbs answer leakage in AMVL.

Recall that AMVL replaces the discrete token bottleneck with a continuous latent space, and that answer leakage refers to the posterior drifting toward answer‑specific regions.

When the posterior mean shifts toward the target answer instead of staying anchored to the input, the model memorizes answer‑specific cues that cannot be reproduced at test time.

Proposition C.1 shows that minimizing the standard ELBO KL forces the prior mean to inherit the average answer‑dependent shift.

Proposition C.2 demonstrates that the forward alignment term alone reproduces the same prior‑mean optimum but provides no gradient on the posterior.

Proposition C.3 gives the closed‑form reverse KL for diagonal Gaussians and its gradient, which pulls the posterior mean toward the prior.

Assumptions 1 and 2 formalize a local linear model of leakage and the linear‑response of the reverse KL.

Corollary 1 follows directly: increasing $\gamma$ monotonically shrinks contamination; Corollary 2 shows the effect is amplified when the posterior variance $\sigma$²₍eff₎ is small.

Finally, an information‑theoretic view links answer leakage to conditional mutual information.

Baseline Comparison

We overview representative multimodal LLM baselines across three reasoning families.

Generalization Analysis

Out-of-distribution evaluation shows our model outperforms baselines on VisualPuzzles.

Our 7B model attains 33.90 overall accuracy on VisualPuzzles, the highest among all baselines.

Table 6 shows our model surpassing Qwen2.5‑VL‑7B, Pangea‑7B, Deepeyes, LVR, and LLaVA‑OneVision‑72B.

**Table 1.** Performance comparison of various methods across different reasoning categories including Overall, Algorithmic, Analogical, Deductive, Inductive, and Spatial.

Occlusion‑based heatmaps reveal that each latent token Lᵢ shifts when image patches covering task‑relevant objects are masked, confirming that tokens attend to concrete visual regions rather than diffuse context.

The token‑wise cosine similarity matrix displays a banded pattern: adjacent tokens are highly similar, distant ones less so, indicating a smooth latent trajectory. Perturbation tests further show the representations are more affected by image permutations than by text permutations, underscoring visual grounding.

These observations collectively demonstrate that the learned latent reasoning space preserves abstract structure under distribution shift, enabling stronger out‑of‑distribution reasoning than discrete‑token baselines.

Latent Space Dispersion

We quantify how each training objective reshapes latent geometry via spread and alignment metrics.

To see how the training objective sculpts the latent space we measure dispersion and alignment of the pooled prior and posterior means for each variant.

These five numbers tell us how tightly the sample‑level latent means cluster (Sp, Sq) and how well the two branches line up (DL2, `S_cos`, `D_shift`).

Removing reverse‑KL (i.e., using plain NTP) inflates prior spread by over three‑fold and degrades alignment dramatically.

Sp rises from 0.4453 to 1.9243 (+332 %) and DL2 jumps from 3.6960 to 16.9488.

Dropping forward‑KL (i.e., reverting to NTP) raises posterior spread by ~15 % and collapses cosine similarity.

Sq climbs from 0.8707 to 2.2326 (+156 %) while `S_cos` falls from 0.8697 to –0.0190 (≈ −102 %).

Using only forward‑KL leaves a larger prior–posterior spread gap than the full AMVL objective.

The gap $\Delta$S = Sq − Sp is 0.2348 for NTP + Fwd‑KL versus 0.1471 for AMVL, a difference of +0.088.

Extended Empirical Analysis

Extended ablations probe head design, loss weighting, gradient flow, and stochastic inference.

We now present three families of ablations: (1) variational head design, (2) loss‑weight balance, and (3) inference‑time latent sampling. Each ablation isolates a single component of the Asymmetric Mutual Variational Learning framework to assess its contribution.

The deeper LLM‑native head performs worse than the default lightweight version. We hypothesize that a deeper head decouples the variational readout from the backbone’s hidden patterns, making prior‑posterior calibration harder.

Table 8 aggregates six loss‑weight configurations. Across all settings the performance varies only modestly, confirming that AMVL does not rely on a narrowly tuned objective balance.

The forward KL term chiefly aligns the prior for inference, while the reverse KL term sharpens the posterior and preserves support compatibility. Over‑emphasizing either side harms the trade‑off, as seen in the ablation curves.

Table 11 evaluates the stop‑gradient design. The full model consistently outperforms the three ablations that drop stop‑gradient from prior alignment, posterior regularization, or both, demonstrating that decoupled gradient flow is essential.

**Figure 4.** Quantitative analysis of latent token properties. **Left:** Average cosine similarity matrix across latent tokens. **Right:** Sensitivity of latent representations to image and text permutations.

Limitation: all experiments are confined to the 7 B‑parameter MLLM. Scaling AMVL to 70 B‑plus models remains an open question, as larger capacities may exhibit different latent dynamics.

Broader impacts: improved latent reasoning can lower compute costs and increase interpretability, but the same capability could be misused for more convincing disinformation or automated phishing.

Notation Reference

Table 5 maps every symbol to its type and concise meaning.

For clarity, Table 5 collects the symbols used throughout the paper, indicating whether each denotes a sequence, distribution, or parameter and providing a short plain‑English description.

**Table 5.** Meanings of the main notations used in the paper.

Theoretical Derivations

This section derives the variational objective and explains the complementary roles of forward and reverse KL divergence.

The variational foundation of our method relies on the Evidence Lower Bound (ELBO), which provides a tractable surrogate for the intractable marginal likelihood of the target sequence. By introducing a variational posterior $q_\phi(Z|x, y)$, we transform the marginalization over latent reasoning trajectories into an expectation that can be optimized via standard gradient-based methods.

While the ELBO provides a lower bound, it is insufficient for inference-time usability because the posterior has access to the target during training, whereas the prior does not. To bridge this gap, we introduce a reverse-KL regularizer motivated by the Evidence Upper Bound (EUBO).

Both forward and reverse KL terms in our framework are mass-covering, meaning they force the optimized distribution to spread out and cover the support of the reference distribution, rather than collapsing into a single mode.

Experimental Setup

Implementation specifics for the Qwen2.5‑VL‑7B base model and AMVL training pipeline.

All experiments build on the Qwen2.5‑VL‑7B‑Instruct multimodal LLM and its official processor/tokenizer for text‑image formatting.

It is a pretrained multimodal LLM that jointly encodes images and text, providing a strong foundation for downstream reasoning tasks.

We train AMVL on a curated mixture of multimodal reasoning datasets: Visual‑CoT for step‑wise reasoning, ReFocus for fine‑grained spatial grounding, CogCoM for multi‑hop deduction, and Zebra‑CoT for relational compositionality.

Each sample receives a dedicated latent token block inserted immediately after the assistant’s prefix. The block consists of a start marker, $k$ placeholder tokens, and an end marker: <`abs_token`> `latent_pad`₁ … `latent_pad_k` </`abs_token`>.

The prior and posterior are factorized diagonal Gaussians over the $k$ slots. By default $d=512$ and $k=8$. The variational module predicts mean and log‑variance for each slot, using the reparameterization trick for sampling.

Training follows teacher‑forced autoregressive decoding. Language‑model loss is applied only to answer tokens; the latent start, end, and placeholder tokens are masked out from next‑token supervision.

We optimize with AdamW (fused PyTorch), bf16 mixed precision, gradient checkpointing, and a cosine learning‑rate schedule with a warmup ratio of 0.05. The vision encoder stays frozen throughout training.

KL regularization uses separate schedules. The forward KL weight $\beta_t$ ramps linearly from 0 to 1 over the first 2000 steps. The reverse KL weight $\gamma_t$ stays 0 for the first 1000 steps, then linearly rises to 0.5 over the next 2000 steps.

After adding the latent special tokens, we resize the embedding matrix. New token embeddings are initialized as the mean of the original embeddings, and during an initial adaptation phase only these new embeddings receive gradient updates.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers