OvisOCR2 Technical Report

OvisOCR2 is a 0.8B parameter end-to-end document parser that outperforms pipeline-based systems on major benchmarks.

How can a compact 0.8B parameter model achieve state-of-the-art document parsing performance by using a specialized data engine and a two-branch training pipeline?

Document parsing systems typically rely on multi-stage pipelines that decompose pages into layout analysis, content recognition, and merging. These pipelines are prone to error accumulation, where a missed table boundary or incorrect reading order in an early stage cannot be rectified by downstream components. OvisOCR2 replaces this fragmented approach with a single-pass, end-to-end model that generates Markdown directly from document images. The system uses a dual-pipeline data engine—combining filtered real-world documents with synthetic pages derived from HTML source-of-truth—and a training recipe involving reinforcement learning and on-policy distillation to align a compact 0.8B backbone. On the OmniDocBench v1.6 leaderboard, OvisOCR2 achieves a state-of-the-art overall score of 96.58, surpassing leading pipeline-based methods.

Paper Primer

The core mechanism hinges on a "source-of-truth" synthetic pipeline: instead of parsing rendered images to create training labels, the authors generate both the document image and the Markdown target from the same HTML source. This ensures deterministic, noise-free ground truth for complex structures like tables and formulas, which are then used to train the model via reinforcement learning and distillation.

OvisOCR2 achieves state-of-the-art performance on OmniDocBench v1.6.

The model scored 96.58 overall, outperforming pipeline-based methods like PaddleOCR-VL-1.6 and MinerU2.5-Pro. It improves over the previous best end-to-end method by 1.84 points.

The model demonstrates superior robustness on complex document scenarios.

On an in-house benchmark, OvisOCR2 achieved the highest overall score across all difficulty tiers, specifically reducing the table "missing rate" compared to pipeline methods. Pipeline methods missed 13–17% of tables, whereas OvisOCR2 missed only 7.96%.

Why move to an end-to-end architecture if pipeline methods currently dominate the leaderboards?

Pipeline methods suffer from error propagation: errors in layout analysis or region detection are irrecoverable by downstream recognizers. An end-to-end model simplifies deployment and allows the model to condition on global page-level context throughout the entire generation process.

What is the role of the 4B branch in training a 0.8B model?

The 4B model acts as a teacher during the on-policy distillation stage. Because the 0.8B model is more sensitive to high-variance updates on long, structured outputs, the authors use the 4B model to transfer reward-aligned parsing preferences through distribution matching rather than direct reinforcement learning.

OvisOCR2 demonstrates that compact, end-to-end models can surpass complex pipeline systems by leveraging high-quality synthetic data and reward-aligned training, effectively shifting the bottleneck from architectural complexity to data engine design.

Introduction to OvisOCR2

We frame the need for a compact end‑to‑end document parser and introduce OvisOCR2 as its solution.

Document parsing must turn a page‑level image into a structured Markdown that preserves text, formulas, tables, and visual layout. This requires more than plain OCR: the model must understand reading order, hierarchical structure, and visual cues to produce a machine‑readable representation.

Existing pipelines split the task into layout analysis, region‑level recognition, and merging, which inflates deployment footprints and lets errors accumulate across stages. End‑to‑end approaches avoid these pitfalls by using a single model, but they have historically underperformed on public leaderboards.

OvisOCR2 is a compact 0.8 B Vision‑Language Model that reads a document page image and directly emits a complete Markdown transcription, handling text, formulas, tables, and visual regions in one pass.

Evaluated on OmniDocBench v1.6, PureDocBench, and an in‑house benchmark, OvisOCR2 attains the highest scores among all methods, surpassing the previously dominant pipeline approaches.

The shift from pipeline‑based OCR to end‑to‑end parsing eliminates deployment complexity and error propagation while delivering state‑of‑the‑art accuracy.

The Data Engine

The Data Engine fuses real and synthetic pipelines to produce high‑quality training data for OvisOCR2.

End‑to‑end parsing needs supervision that is correct at the text, formula, table, and reading‑order levels. Real documents provide realistic visual diversity but their annotations are noisy; synthetic documents give perfect labels but can look artificial. The Data Engine resolves this tension by combining two complementary pipelines.

The engine runs a real‑world pipeline and a synthetic pipeline in parallel, then merges their outputs into a single high‑quality training corpus.

Real pipeline: OCR parses R₁ → JSON → unified Markdown M₁; OCR parses R₂ → M₂.

Synthetic pipeline: template S₁ is diversified → HTML H₁ → exact Markdown M₃; template S₂ → M₄.

All four Markdown files (M₁‑M₄) are concatenated into the training corpus.

Corresponding images (rendered for synthetic, scanned for real) are paired with their Markdown files.

The combined corpus inherits the visual realism of R₁,R₂ and the annotation precision of M₃,M₄, giving the student model supervision that is both diverse and trustworthy.

How does this Data Engine differ from a simple data‑augmentation pipeline?

Typical augmentation only perturbs existing images while keeping their original (and possibly noisy) labels. The Data Engine creates a parallel synthetic stream with deterministic labels generated from the same HTML source, so the synthetic side adds clean, controllable examples rather than noisy variants of real data.

**Figure 2.** Architecture of the data engine

Two-Branch Training Pipeline

We detail the two‑branch training pipeline that yields a compact, high‑fidelity document parser.

Token‑level imitation loss cannot capture structural errors in tables, formulas, or long pages, so the parser needs supervision that directly rewards correct layout and content.

The pipeline runs a large teacher branch in parallel with a compact student branch; the teacher learns a high‑quality policy via reinforcement learning, then guides the student through on‑policy distillation, after which both branches are fused into the final model.

How does this differ from standard knowledge distillation?

Standard distillation matches the teacher’s full‑vocabulary logits on a fixed dataset. Here we match the teacher only on the student’s top‑k token support and on‑policy trajectories, which drastically reduces memory and focuses learning on the tokens the student actually uses.

Supervised fine‑tuning (SFT) on a mixed real‑synthetic corpus builds a base end‑to‑end parsing policy.

Reinforcement learning (RL) with Group Relative Policy Optimization (GRPO) refines the policy using text, formula, and table rewards.

On‑policy distillation (OPD) transfers the RL‑aligned behavior from the 4 B teacher to the 0.8 B student.

Weighted parameter averaging fuses multiple student checkpoints into the final OvisOCR2 model.

Sum of weighted scores = $1\cdot0.92 + 1\cdot0.68 = 1.60$.

Sum of availability indicators = $1+1 = 2$.

Reward $R = 1.60 / 2 = 0.80$.

The reward automatically ignores the missing formula component, preventing a zero‑score penalty for pages without formulas.

Student normalized distribution $\bar{p}_t = [0.5, 0.3, 0.2]$ (already sums to 1).

Teacher normalized distribution $\bar{q}_t = [0.6, 0.25, 0.15]$.

KL$(\bar{p}_t\| \bar{q}_t) = 0.5\log\frac{0.5}{0.6}+0.3\log\frac{0.3}{0.25}+0.2\log\frac{0.2}{0.15}\approx 0.028$.

If this is the only position ($|I|=1$), $L_{\text{OPD}} = 0.028$.

Restricting to the student’s top‑$k$ dramatically cuts the tensor size from $O(TV)$ to $O(Tk)$ while still pulling the student toward the teacher’s preferred tokens.

**Figure 3.** Two-branch training of OvisOCR2. The 4B branch produces an RL-aligned teacher, while the 0.8B branch proceeds through SFT, OPD, and model fusion to obtain the final model.

**Figure 4.** Training stability comparison between 4B RL and 0.8B RL. Panel (a) reports actor KL loss, with faint traces showing raw per-step values and bold curves showing 15-step centered rolling means. Panel (b) reports table TEDS on the validation set.

The table lists three components: Text, Formula, and Table. The corresponding scores are "1 - normalized edit distance", "CDM", and "TEDS", respectively. The measured aspects are "Text fidelity", "Visual formula matching", and "Table content and topology".

By chaining SFT, RL, OPD, and fusion, the two‑branch pipeline yields a 0.8 B OvisOCR2 that matches the teacher’s structural quality while remaining deployment‑ready.

Evaluation Overview

OvisOCR2 tops the OmniDocBench leaderboard across all key metrics.

We evaluate OvisOCR2 against a broad spectrum of document parsers on OmniDocBench v1.6 and an in‑house benchmark, grouping baselines into General VLMs, Specialized VLMs (pipeline), and Specialized VLMs (end‑to‑end).

OvisOCR2 attains the highest overall score of 96.58 % on OmniDocBench v1.6, surpassing the previous best (Gemini 3 Pro 92.91 %).

Table 2 shows OvisOCR2 leading the overall column, with a margin of +3.67 % over the next best specialized end‑to‑end model.

**Table 1.** Performance comparison of various models on document parsing tasks.

Across the full suite of metrics—FormulaCDM, TableTEDS, TableTEDS‑S, and ROEdit—OvisOCR2 matches or exceeds the best pipeline and general VLM scores, confirming the effectiveness of the two‑branch training pipeline.

OmniDocBench Performance

PureDocBench shows OvisOCR2 leads with a 75.06 Avg3 score while OmniDocBench scores 96.58 overall.

OvisOCR2 attains a state‑of‑the‑art Avg3 score of 75.06 on PureDocBench.

Table 3 shows OvisOCR2 leading the Clean and Digital tracks with the highest reported values.

OmniDocBench v1.6 is the most widely adopted public benchmark for page‑level document parsing, covering 1,651 PDF pages across ten document types, five layout types, and five language types. It evaluates four dimensions—text (normalized edit distance), formula (character‑detection matching, CDM), table (tree‑edit similarity, TEDS and TEDS‑S), and reading‑order (edit distance over block sequences)—and aggregates them by averaging text, formula CDM, and table TEDS scores. OvisOCR2 attains a state‑of‑the‑art overall score of 96.58, surpassing leading pipeline methods and improving the previous best end‑to‑end result by 1.84 points.

OmniDocBench v1.6 is a public benchmark that stresses full‑document parsing across diverse layouts, languages, and formula complexities.

PureDocBench complements OmniDocBench by providing source‑traceable evaluation data: document images are rendered from HTML sources, and annotations are derived directly from those sources. It assesses robustness on three tracks—Clean (rendered pages), Digital (degraded images), and Real (photocopies, phone captures, screenshots)—covering 1,475 pages across ten domains and 66 subcategories, for a total of 4,425 images.

In-house Benchmark Results

OvisOCR2 dominates the in‑house benchmark across all difficulty levels and document types.

OvisOCR2 leads overall by +2.66 points and beats every baseline on all six metrics.

Table 4 shows OvisOCR2 achieving the best values for TextEdit, FormulaCDM, TableTEDS, TableTEDS‑S, ROEdit, and Overall.

**Table 5.** Overall performance on the in-house benchmark by difficulty level.

**Table 6.** Results on the handwriting subset of the in-house benchmark.

**Table 7.** Results on the complex-table subset of the in-house benchmark. MR denotes missing rate, defined as the fraction of tables missing from the parsed output.

The table compares the performance of various OCR models (PaddleOCR-VL-1.5, PaddleOCR-VL-1.6, GLM-OCR, MinerU2.5-Pro, and OvisOCR2) across five metrics: Overall, `Text_Edit`, `Table_TEDS`, `Table_TEDS`-S, and MR.

Qualitative Analysis and Conclusion

Qualitative comparisons on tables and handwritten documents.

This appendix showcases visual examples of OCR outputs across table and handwritten documents, highlighting errors with red marks and omitted text in light gray brackets.

**Figure A.1.** Qualitative comparison on a table document.

**Figure A.1.** Qualitative comparison on a table document (continued).

**Figure A.2.** Qualitative comparison on a table document.

**Figure A.2.** Qualitative comparison on a table document (continued).

**Figure A.3.** Qualitative comparison on a handwritten document.

**Figure A.3.** Qualitative comparison on a handwritten document (continued).

**Figure A.4.** Qualitative comparison on a handwritten document.

**Figure A.4.** Qualitative comparison on a handwritten document (continued).

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers