BDH-CQ: In-Context Learning with Recurrent Latent Reasoning

Björn Engdahl, Adrian Kosowski, Jan Chorowski, Zuzanna Stamirowska, Przemysław Uznański, Junlin Jiang, Rohan Phadke, Remigiusz Kinas, Richard Zhong

BDH-CQ uses recurrent latent reasoning to solve visual tasks from context without verbalizing intermediate steps.

How can a small, efficient model perform complex reasoning on unseen tasks by using recurrent latent states to process in-context examples?

Chain-of-thought reasoning models rely on serializing intermediate steps into language, which inflates token consumption and inference latency. BDH-CQ replaces this verbal scratchpad with a recurrent latent workspace: demonstrations update a persistent memory state, and the model performs iterative computation in a high-dimensional latent space to derive the answer. This architecture achieves 29.5% pass@2 on the ARC-AGI-1 benchmark at a cost of \$0.0007 per task, establishing a new state of the art in cost efficiency.

Paper Primer

The system hinges on separating contextual memory from the reasoning workspace. Demonstrations are ingested sequentially into a recurrent memory state $S_t$, which then conditions the iterative transformation of a latent workspace $H_r$ to produce the final output grid.

The core move is a latent-only reasoning loop: the model is like a calculator that keeps its intermediate scratchpad in a private, high-dimensional memory rather than writing it on a public whiteboard. This allows the model to explore multiple solution paths in parallel without the overhead of decoding tokens.

BDH-CQ breaks the existing cost-accuracy Pareto frontier for ARC-AGI-1.

The 150M-parameter model achieves 29.5% pass@2 at \$0.0007 per task, significantly outperforming larger models in cost-efficiency. Approximately 57x cheaper than the previously reported GPT-5.6 Luna (Low) baseline.

The model exhibits structured capability boundaries rather than uniform failure.

Controlled experiments show the model reliably extrapolates boundary propagation and copying, but struggles with deep nested containment and long ordering sequences unless supported by specific demonstrations.

Why is this approach more efficient than standard chain-of-thought prompting?

By performing computation in a continuous latent space, the model avoids the latency and compute costs associated with projecting, emitting, and re-consuming intermediate natural-language tokens.

Does the model actually learn the transformation rule, or is it just guessing?

The model shows inconsistent rule application; while it solves many test pairs, the gap between test-pair accuracy and strict task accuracy suggests it often fails to apply a single induced rule consistently across all inputs of a task.

The system's performance is highly sensitive to demonstration coverage: providing a demonstration at the target complexity level (e.g., depth-five nesting) can recover performance from a baseline of near-zero to full success.

BDH-CQ demonstrates that high-performance reasoning can be decoupled from serial language generation, providing a blueprint for efficient, latent-reasoning systems that learn directly from in-context demonstrations.

Introduction and Contributions

We frame the inefficiency of token‑based reasoning and introduce a recurrent latent approach.

Chain‑of‑thought prompting lets language models acquire new skills from demonstrations, but every intermediate token must be generated, consumed, and re‑encoded, inflating latency and inference compute.

Latent reasoning sidesteps this bottleneck by keeping the reasoning process inside a high‑dimensional hidden state that is updated repeatedly, only emitting the final answer in language.

BDH‑CQ fuses in‑context learning with a recurrent latent core: demonstrations are absorbed into a persistent memory, then the query is solved by repeatedly transforming that memory in a continuous space, never materializing intermediate steps as text.

How does BDH‑CQ differ from standard chain‑of‑thought prompting?

Chain‑of‑thought emits each intermediate reasoning step as language tokens, which must be processed by the decoder. BDH‑CQ keeps all intermediate computation inside a recurrent hidden vector, so the model incurs only the cost of a few matrix multiplications per iteration and avoids the token‑generation bottleneck.

Recurrent latent reasoning maintains a hidden state that persists across iterative updates, letting the model progressively refine its internal hypothesis about the task without external supervision.

Is this recurrence similar to a traditional recurrent neural network?

No. Traditional RNNs iterate over input tokens, updating a hidden state based on the next token. Here the recurrence updates a latent vector based on a fixed set of demonstrations, independent of token order, and the hidden state represents the evolving solution rather than a language model context.

Recurrent latent reasoning delivers a far more efficient inference path than standard LLM token‑by‑token generation.

ARC as a Testbed

We present ARC as a precise visual testbed to evaluate BDH‑CQ’s in‑context generalization.

ARC (Abstraction and Reasoning Corpus) offers a tightly controlled visual language that lets us probe fast, in‑context generalization without linguistic confounds.

BDH‑CQ builds on the earlier BDH architecture, adding a structured latent workspace and recurrent computation so the model can learn visual transformations directly from a few demonstrations.

**Figure 1.** An ARC-AGI-1 training task (identifier 0520fde7). Two of three demonstrations and the held-out query are shown; the query output is included here for explanation. The task is compact, exact, and easy to inspect, yet its transformation must be inferred from examples.

Task Formulation and Training

We describe how ARC tasks are formatted and how the curated training mixture prepares the model.

Think of the model as a student who first studies a handful of solved puzzles (the demonstrations) before attempting a new puzzle (the test), all within a single session.

Step 1: Observe first demonstration (x₁,₁, y₁,₁) and update recurrent hidden state h ← f(h, x₁,₁).

Step 2: Observe second demonstration (x₁,₂, y₁,₂) and further update h ← f(h, x₁,₂).

Step 3: Receive test input $x^{\text{test}$}₁,₁; generate prediction ŷ = g(h, $x^{\text{test}$}₁,₁).

Step 4: Compare ŷ to the true target $y^{\text{test}$}₁,₁ to compute loss.

The recurrent hidden state accumulates information from the demonstrations, enabling the model to answer the test without seeing it beforehand.

How does this ARC task formulation differ from ordinary supervised learning?

In standard supervised learning the model sees each example in isolation; here the model processes a *sequence* of demonstrations, updating a recurrent latent state after each one, so the test prediction can depend on the whole demonstration set rather than a single static input.

The training mixture builds on this formulation by exposing the model to a large, diverse corpus of ARC‑style problems.

Start with a 150 M‑parameter backbone model.

Collect privately curated examples and merge them with public sources: ARC‑AGI‑1, RE‑ARC, ConceptARC, ARC‑Heavy, and ARC‑GEN100K.

Apply data augmentations (e.g., grid rotations, color jitter, token shuffling) to increase variability.

Package the resulting examples into tasks following the ARC task formulation described above.

Train the model to predict test targets after sequentially ingesting the demonstrations for each task.

Performance on ConceptARC

Behavioral analysis shows a sizable performance gap between semantic and opaque identifiers.

BDH‑CQ attains 97 pass@2 on the public ARC‑AGI‑1 evaluation.

Table 1 reports 97 correct out of 400 tasks (24.25%).

ConceptARC groups ARC‑like tasks into 16 families, each probing a distinct visual transformation.

How does ConceptARC differ from the original ARC benchmark?

ARC presents a flat set of tasks, whereas ConceptARC explicitly groups tasks by visual operator families, enabling fine‑grained analysis of which transformations the model masters.

**Figure 3.** ConceptARC pass@2 by concept area. Left: semantic-ID test-pair and strict-task accuracy; their gap captures correct outputs that do not transfer across all three test inputs of a task. Right: strict-task accuracy when the same grids are sent with semantic identifiers in concept-grouped batches or opaque identifiers in concept-mixed batches. The profile is nearly unchanged; this combined intervention does not isolate the effects of identifiers and batch composition separately.

**Table 1.** Headline results on the public ARC-AGI-1 evaluation and on ConceptARC. Intervals are descriptive Wilson 95% intervals.

**Table 2.** ConceptARC profile by concept area. Each area has ten tasks and thirty test pairs. Pass@2 is shown for both the semantic and opaque-identifier executions; pass@1 and test-pair columns are from the semantic execution. The three task-pass@2 changes are shown in bold.

The dominant performance gap stems from semantic identifiers outperforming opaque ones on strict‑task accuracy.

Controlled Generalization Patterns

Ablation analysis reveals context coverage drives performance gaps across ordering and nesting tasks.

Across the four controlled families, propagation and copying stay perfect, while ordering and nesting exhibit sharp declines once the test length or depth exceeds the demonstrated range.

Supported in‑context demonstrations raise ordering length‑8 pass@2 accuracy from 0/24 to 12/24.

Table 3 reports 0/24 for the short context and 12/24 for the supported context.

Supported context improves nesting depth‑5 pass@2 accuracy from 15/24 to 16/24.

Table 3 lists 15/24 for the short context and 16/24 for the supported context.

**Figure 4.** Representative held-out examples from the four controlled generalization families. From top to bottom: extend a seed to the boundary, copy a motif to every gray anchor, order bar colors from shortest to tallest, and recolor only cells inside every nested frame. These examples show the transformations whose distance, multiplicity, sequence length, and nesting depth are varied in Figure 5.

**Figure 5.** Controlled generalization curves reporting exact held-out-output accuracy. Ordering and nesting include a larger preregistered replication around the transition; propagation and copying use 12 outputs per point, ordering and nesting use 36, and the two longest ordering levels use 24.

**Table 3.** Output accuracy depends on coverage in the in-context examples. The same 24 held-out inputs are evaluated under two contexts while every test pair remains byte-identical. “Short” demonstrations stop below the test complexity; “supported” adds one demonstration at the target length or depth. p@1 uses the first candidate and p@2 accepts either of two candidates; entries report exact outputs out of 24.

Contextual Memory and Composition

Ablations probe which components matter and how operations compose.

To test whether the model can compose operations, we compare performance when demonstrations contain a single operation versus a composition with relocation.

Composition succeeds for rotation (full 72/72) and works partially for reflection (47/72), but fails completely for color swap (0/72), showing that composability depends on the operation.

Within‑task consistency shows an 18.5‑point gap between pair accuracy (77.92 %) and strict task accuracy (59.38 %).

Measured on the ConceptARC benchmark under pass@2.

We next examine whether the model relies on task identifiers or batch context by replacing semantic identifiers with opaque labels.

Finally, we compare a low‑effort (“MIN”) setting to the standard setting on the ARC‑AGI‑1 benchmark.

MIN effort scores 111/400 versus STANDARD 118/400, a drop of 1.75 pp, which is not statistically significant (McNemar p = 0.167).

Exact McNemar test on the ARC‑AGI‑1 pass@2 results.

**Figure 6.** Representative held-out pairs from an independent motif family. Relocation moves the motif to the gray anchor; rotation changes it in place; their composition performs both.

Reasoning Effort and Scaling

Increasing recurrent latent reasoning effort improves pass@2 at the cost of extra compute.

Increasing reasoning effort raises pass@2 performance.

Table 5 shows pass@2 improves from 21 % at low effort to 29.5 % at high effort.

Reasoning effort is the amount of recurrent latent computation applied at inference, trading compute for accuracy.

LOW: pass@2 = 21 %, cost reduction = 22 %.

MEDIUM: add one extra step → pass@2 = 27 %, cost reduction = 11 %.

HIGH: add another step → pass@2 = 29.5 %, cost reduction = 0 %.

Each extra recurrent step yields diminishing returns on cost savings while still boosting accuracy.

**Table 5.** Comparing pass@2 and cost across reasoning efforts LOW, MEDIUM, HIGH.

**Figure 7.** Showing how pass@2 and cost scale with reasoning effort.

Chain‑of‑thought prompting elicits verbal intermediate steps that improve reasoning in large models (Wei et al., 2022). BDH‑CQ replaces this with a continuous recurrent workspace, removing the need for explicit language generation.

Coconut demonstrated feeding a Transformer’s final hidden state back as the next input embedding, enabling a curriculum that swaps verbal steps for continuous thoughts (Hao et al., 2024). Theory shows continuous states can encode multiple search frontiers in parallel (Zhu et al., 2025), and latent iteration can exploit graph parallelism (Xu & Sato, 2026).

Other strands of work distinguish continuous latent thought from compressed abstract token languages (Ramji et al., 2026) and from inter‑agent latent channels (Liu, 2026). Recurrent‑depth models apply shared blocks repeatedly to a sequence‑wide latent state, improving reasoning benchmarks without extra parameters (Geiping et al., 2025). Task‑trained recursive solvers such as HRM and TRM write task information into recurrent memory and achieve strong ARC performance, albeit with higher per‑task costs.

Failure Mode Analysis

Limits of BDH‑CQ are exposed by failure analysis across panel, object, and dependency manipulations.

BDH‑CQ iteratively refines a recurrent latent state using in‑context demonstrations before producing the final answer. This appendix probes the approach’s limits by dissecting failure patterns across several manipulation families.

Table 9 quantifies the failure structure on the 400‑task public and calibrated generated sets. The calibrated set contributed 251 failed tasks, the public set 282. Most failures preserve output dimensions and palette; shape‑correct failures differ from the target in only a small fraction of cells.

**Table 9.** Properties of incorrect predictions on the calibrated generated and public sets. Median cell error is computed only among failures with the correct output dimensions. Insufficient target-edit coverage is an output-only descriptor: the prediction modifies fewer target-change cells than required; it does not identify an inferred rule.

Discussion and Cost Analysis

We discuss the implications of BDH‑CQ’s cost‑efficient ARC performance and outline future directions.

BDH‑CQ establishes a new state‑of‑the‑art cost‑efficiency on ARC‑AGI‑1, breaking the previously reported cost–accuracy Pareto frontier.

Figure 2 shows the model achieving 29.5 % pass@2 at \$0.00070 per task, surpassing prior systems.

**Figure 2.** ARC-AGI-1 score versus computed cost per task. Data points were collected from the official ARC Prize leaderboard on August 4, 2026. The mark at the left margin indicates the 29.5 pass@2. The point breaks through the previously reported cost–accuracy Pareto frontier and establishes a new state of the art in ARC-AGI-1 cost efficiency.

Controlled experiments reveal that BDH‑CQ can apply dense, task‑specific color mappings to all 96 held‑out outputs, demonstrating strong contextual binding capacity.

Boundary propagation and copying extrapolate throughout the tested ranges, while ordering and nested containment expose distinct limits of the system.

Providing matched demonstrations eliminates depth‑five nesting failures and substantially improves length‑eight ordering, indicating that demonstration coverage directly influences extrapolation.

Composition is representation‑dependent: rotation composes with relocation on all 72 held‑out outputs, reflection succeeds on 47/72 and varies sharply across motif families, and color swapping is not reliably acquired outside the original fixed layout.

The ConceptARC ontology organizes this capability profile, while post‑freeze generators test generalization and localize current boundaries in consistency, composition, conditional execution, and output construction.

Looking forward, BDH‑CQ points toward a broader family of BDH‑based reasoning systems where memory, adaptation, and inference share a unified computational fabric.

The immediate direction is to scale BDH‑CQ itself: larger models and longer training runs will test whether BDH’s favorable sequence‑model scaling carries into in‑context latent reasoning and whether the identified capability boundaries shift predictably with scale.

Broader horizons include ARC‑AGI‑2, Sudoku and other constraint‑satisfaction domains for long‑horizon latent refinement, and language and mathematical reasoning to assess whether the same recurrent memory can acquire tasks from textual demonstrations.

Because BDH layers support both language modeling and latent visual reasoning, future systems may combine continuous internal computation with verbalized steps when communication, verification, or tool use benefits from explicit language.

The BDH‑CQ architecture scales naturally to large model sizes, offering tensor‑sharding patterns that make training at the 1 T‑parameter scale straightforward.

Early experiments confirm that Transformer‑like scaling laws hold from 1 B to 600 B parameters, while preserving the latent reasoning capabilities specific to BDH‑CQ.

Evaluation Set Profiling

Additional analyses of evaluation sets, task mechanics, and controlled manipulations.

Section A.1 compares three evaluation cohorts: the public ARC‑AGI‑1 set, a calibrated generated set, and a mechanic‑stratified generated set, reporting exact whole‑task solve rates instead of per‑pair accuracy.

The table lists various mechanics, their sample size ($n$), solve rate, 95% confidence interval, and mean side length.

Simple surface properties explain little of the observed success variance; nine bucketings on the public set exceed a permutation‑search null, none do so on the generated set, and grid size yields pseudo‑R² of 0.072 versus 0.010.

Section A.2 groups tasks by their required transformation mechanic, producing a balanced sample of 1,131 tasks across 16 mechanics and revealing a 65.7‑point spread in solve rates, from 68.6 % for flood fill down to 2.9 % for gravity and stacking.

**Table 7.** Exact whole-task solve rate on the mechanic-stratified generated set. Wilson confidence intervals and mean grid side are reported for each generated mechanic.

Public and generated mechanic rankings correlate weakly ($\rho = 0.300$), and hand‑written versions of gravity and counting tasks achieve dramatically higher solve rates (80–100 % and 85–95 %) than their generated counterparts (2.9 % and 18.6 %).

Section A.3 introduces deterministic Python ladders to isolate structural cues; conditional‑selection cues reduce solve rate to 56.7 % (a 43.3‑point drop), while marker‑count cues yield 30 % solve only when the value is demonstrated, dropping to 0 % otherwise.

The table compares failure properties between "Calibrated generated" and "Public" methods across five categories: Output dimensions correct, Palette correct, Median cell error, shape correct, Input reproduced verbatim, and Insufficient target-edit coverage.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers