SciForma: Structure-Faithful Generation of Scientific Diagrams

Yuxuan Luo, Peng Zhang, Xinjie Zhang, Xun Guo, Zhouhui Lian, Yan Lu

SciForma enforces structural fidelity in scientific diagrams by decomposing quality into component, arrow, and text axes.

How can we train generative models to produce scientific diagrams that maintain strict structural fidelity in components, arrows, and text?

Scientific methodology diagrams require precise structural fidelity, yet current generative models often produce plausible-looking images that fail on critical details like reversed arrows or unreadable formulas. SciForma addresses this by decomposing diagram quality into three independently verifiable axes—Component, Arrow, and Text—and enforcing simultaneous correctness through Multi-Dimensional Conjunctive Preference Optimization (M-DPO). This framework allows SciForma-9B to outperform proprietary systems like GPT-Image-1.5 on structural benchmarks, providing a rigorous, verifiable path for scientific visualization.

Paper Primer

The core challenge is that diagram quality is non-compensatory: a single structural error, such as a misrouted arrow, invalidates the entire figure. Standard supervised fine-tuning learns general layouts but lacks element-level verification, while existing post-training methods collapse distinct structural failures into a single scalar reward, obscuring which specific dimension needs correction.

SciForma recasts generation as a structural verification task: it uses a structural inventory to extract ground-truth checklists from reference diagrams. The M-DPO mechanism then routes gradients adaptively to the most deficient axis, ensuring the model simultaneously satisfies all structural constraints. Think of M-DPO like a multi-subject tutor: instead of giving a single "pass/fail" grade, it identifies exactly which subject—math, logic, or grammar—the student is failing and focuses all subsequent instruction on that specific gap.

SciForma-9B achieves superior structural fidelity compared to open-source baselines and proprietary models.

On the SciFormaBench-2K benchmark, SciForma-9B reaches an average score of 69.51, surpassing GPT-Image-1.5 (68.96). The largest performance gains occur on the weakest axes, specifically +1.82 on Arrows and +3.16 on Text.

M-DPO effectively breaks the stagnation of supervised fine-tuning (SFT).

Ablation studies show M-DPO achieves a larger average performance gain (+1.92) in one-seventh the training steps compared to continued SFT (+0.37).

Why does the paper use a conjunctive objective instead of standard scalar-based preference optimization?

Scalar rewards collapse orthogonal structural axes into one number, allowing a model to "hide" a failure on one axis (e.g., broken text) behind a success on another (e.g., good layout). M-DPO’s conjunctive objective ensures the loss vanishes only when the model satisfies all axes simultaneously.

What is the role of the structural inventory in the inference stage?

The inventory enables an iterative refinement loop where the system identifies specific defects, performs localized inpainting on the failing region, and verifies the correction against the ground-truth checklist before accepting the edit.

The primary assumption is that scientific diagrams can be fully decomposed into three universal primitives (Component, Arrow, Text). This assumption allows the model to treat structural fidelity as a verifiable checklist rather than a subjective aesthetic property.

Researchers can now move beyond holistic visual quality metrics and enforce explicit, element-level structural correctness in scientific diagram generation using axis-decomposed preference optimization.

Abstract

We introduce SciForma and M‑DPO to enforce structural fidelity in scientific diagrams.

Scientific methodology diagrams must faithfully render components, directional arrows, and textual annotations; a single error can invalidate the entire figure.

Current open‑source models struggle with this conjunctive requirement, as supervised fine‑tuning yields plausible layouts but cannot guarantee correctness, and scalar reward‑based post‑training hides which axis fails.

Structural fidelity means every diagram element—its components, arrow directions, and text labels—must be correct; any single mistake can break the logical argument the figure conveys.

We propose SciForma, a framework that decomposes diagram quality along these three axes, trains on the curated SciFormaData‑700K dataset, and evaluates with the logic‑verified SciFormaBench‑2K benchmark.

To enforce joint correctness we introduce Multi‑Dimensional Conjunctive Preference Optimization (M‑DPO), which adaptively routes gradients toward the most deficient structural dimension during post‑training, and we enable iterative editing at inference to fix residual errors.

SciForma‑9B surpasses all open‑source baselines and GPT‑Image‑1.5 on both SciFormaBench‑2K and AIBench, bringing open scientific‑diagram generation close to proprietary‑level structural fidelity.

The paper introduces SciForma to solve structural fidelity issues in scientific diagrams.

The Challenge of Scientific Diagrams

Why scientific diagrams need strict structural correctness and how current models fall short

Methodology diagrams act as blueprints in scientific papers, making research logic visible and verifiable. Unlike natural images that chase visual plausibility, these diagrams demand structural fidelity: every component, arrow, and textual label must be legible. A single reversed arrow or unreadable equation can invalidate the diagram’s meaning, and fidelity is non‑compensatory—strength on one axis cannot offset a failure on another.

Open‑source models lag behind proprietary systems such as Google DeepMind 2025 and OpenAI 2026b on structural fidelity, exposing a gap rooted in current training objectives. Supervised fine‑tuning (SFT) learns plausible distributions via flow‑matching loss but offers no guarantee of complete structural correctness. Existing post‑training methods collapse the orthogonal structural axes into a single scalar signal, obscuring which dimension has failed and lacking mechanisms to verify each axis independently.

Diagram quality splits into three independent checks—Component presence, Arrow directionality, and Text legibility—each must be satisfied for a faithful diagram.

We formalize evaluation through a structural inventory that extracts ground‑truth checklists from reference diagrams for per‑axis verification. This turns the generation task from continuous visual matching into explicit structural verification, ensuring that a failure on any axis cannot be hidden by strengths on the others.

Our contributions are threefold: we curate SciFormaData‑700K and SciFormaBench‑2K for structured generation and evaluation; we introduce Multi‑Dimensional Conjunctive Preference Optimization (M‑DPO), a multi‑way Bradley–Terry objective that routes gradients toward the most deficient axis while enforcing conjunctive correctness; and we fine‑tune SciForma‑Base into SciForma‑9B, which uses the structural inventory for iterative refinement at inference.

On SciFormaBench‑2K, SciForma‑9B surpasses all open‑source baselines and GPT‑Image‑1.5, with M‑DPO delivering the largest gains on the Arrow axis and the Hard tier, while on the VQA‑centric AIBench it exceeds human‑drawn originals, especially on topology. Ablation studies confirm that M‑DPO breaks SFT stagnation, avoids reward collapse of scalar‑based methods, and that coupling decomposed pairs with a conjunctive objective is essential for consistent cross‑axis improvements.

**Fig. 1.** Diverse scientific methodology diagrams generated by SciForma. Our framework ensures strict structural fidelity, enables precise synthesis of complex topologies, dense block layouts, and accurate textual annotations across various academic domains.

Prior Approaches to Diagram Generation

Survey of diagram generators and alignment methods that precede our multi‑axis approach.

Code‑based generation writes diagrams in languages such as TikZ, Mermaid/SVG, or custom markup, offering precise control but limited expressiveness.

TikZilla is a lightweight wrapper that expands basic TikZ primitives into richer diagram components while preserving syntactic validity.

A closed‑source diagram generator that combines diffusion with proprietary fine‑tuning pipelines, yielding high‑fidelity scientific figures.

An image‑generation model extended to scientific diagrams via prompt engineering and large‑scale fine‑tuning.

Systems that augment generation with retrieval of external resources, self‑audit loops, and reflective prompting at inference time.

Post‑training alignment methods for generative models fall into offline DPO‑based adaptations and online policy‑gradient approaches.

A pairwise preference learning objective that aligns language models with human feedback by directly optimizing a reward model.

Ranks multiple candidate generations jointly, weighting the worst‑performing sample to improve robustness.

An online policy‑gradient method that updates the generator using sampled rewards during training.

A multi‑dimensional preference framework for natural‑image generation that balances aesthetic and content fidelity.

Extends DPO to multiple reward components, weighting them via learned coefficients.

These prior alignment techniques collapse per‑axis signals, whereas M‑DPO enforces conjunctive correctness across independently verifiable structural axes.

Defining Structural Fidelity

We introduce a JSON‑based structural checklist that lets us verify diagram components, arrows, and text element‑wise.

Generative models often produce diagrams that look plausible but violate the underlying logical scaffolding; without a fine‑grained audit the errors remain hidden.

We treat a diagram as a three‑part checklist—Component, Arrow, Text—and require every generated output to match the reference checklist element by element.

Generated output misses Component C₂ (critical, $w_e=1.0$) → $e=0$ for that slot.

Arrow A₁ is present but slightly mis‑oriented (moderate, $w_e=0.5$) → $e=1$ for that slot.

Both Text labels appear correctly → $e=1$ for each.

Compute scores: $s_C = (1\cdot1.0 + 0\cdot1.0)/2 = 0.5$, $s_A = (1\cdot0.5)/1 = 0.5$, $s_T = (1\cdot1.0 + 1\cdot1.0)/2 = 1.0$.

The vector $(s_C,s_A,s_T) = (0.5,0.5,1.0)$ instantly reveals that component coverage is the bottleneck, even though the overall visual may look acceptable.

How does this inventory‑based check differ from a conventional pixel‑wise loss?

Pixel‑wise losses penalise low‑level colour differences and cannot tell whether a missing arrow or mislabeled component violates the diagram’s logical meaning. The inventory check operates on semantic primitives, so it directly measures the structural intent of the diagram.

The table compares various benchmarks based on their paradigm, scale, and coverage of specific structural features (Elem., Axes, Diff., Sev.). The proposed benchmark, **SciFormaBench-2K**, is highlighted in the final row, showing it is the only one to support all four features (Elem., Axes, Diff., Sev.) using an inventory checklist paradigm with a scale of 2,000.

**Fig. 12.** Illustration of structural inventory verification on a generated diagram. (a)Reference image. (b)Generated output. (c)Per-item pass/fail checklist across three evaluation dimensions.

The SciForma Pipeline

High‑fidelity data and joint optimization enable accurate scientific diagram generation.

Generative models routinely drop or misplace diagram elements because they lack supervision that respects the three structural axes of scientific figures.

Curate SciFormaData‑700K, a 700 K‑sample corpus with component‑, arrow‑, and text‑level captions.

Apply two‑stage supervised fine‑tuning (SFT) to obtain SciForma‑Base.

Run Multi‑dimensional Preference Optimization (M‑DPO) on the three axes to produce SciForma‑9B.

During inference, invoke the iterative refinement loop to detect and repair structural defects.

SciFormaData‑700K supplies millions of diagram‑caption pairs, each caption split into three independent streams that describe components, arrows, and textual labels.

Component stream → “Components: A, B”.

Arrow stream → “Arrows: A points to B”.

Text stream → “Text: flux”.

During training the model receives three parallel supervision signals, one per stream.

The loss for each stream is computed independently and summed, ensuring that a mistake on the arrow does not mask a correct component prediction.

Separating the caption forces the model to attend to each structural axis explicitly, which is the key to preserving fidelity across all three dimensions.

How do axis‑decomposed captions differ from ordinary figure captions?

Ordinary captions describe the whole diagram in a single sentence, mixing component, arrow, and text information. Axis‑decomposed captions break this into three independent streams, each supervising a distinct structural dimension, so the model receives a clear signal for components, arrows, and text separately.

**Fig. 2.** Overview of SciForma. (a) **Pipeline:** Two-stage SFT yields SciForma-Base, followed by M-DPO to produce SciForma-9B. (b) **Training Details:** Left: SFT data formats for joint generation and editing. Right: M-DPO optimizes a shared winner against dimension-anchored losers to model conjunctive preference. (c) **Iterative Refinement:** A closed-loop inference pipeline detects structural defects and applies localized editing to improve structural fidelity.

**Fig. 3.** Overview of the SciFormaData-700K construction. (Top:) We curate 656K training samples through a pipeline of automated extraction, filtering, captioning, and stratification. (Bottom:) We synthesize 70K localized editing triplets by performing grounded entity modification on high-quality subsets.

Iterative Refinement Loop

Details of the iterative refinement loop and training pipeline that enforce structural fidelity.

High‑density scientific diagrams often retain subtle structural errors after a single generation pass, which downstream readers can misinterpret.

Think of the system as a proofreader that scans a manuscript, fixes a typo, then re‑reads the page to catch any new issues introduced by the edit.

Critic ranks defect A higher (impact = 0.8) than defect B (impact = 0.3).

For defect A the editor generates two candidates; candidate 1 improves the local score from 0.45 to 0.62, candidate 2 to 0.58. Candidate 1 is kept.

Verification recomputes the global C/A/T score; it rises from 0.71 to 0.73, so the edit is accepted.

Defect B is processed next; both candidates lower the global score, so the loop terminates.

The loop stops exactly when any further edit would hurt the overall structural metric, ensuring no regression.

How does this iterative loop differ from a single‑pass generation followed by post‑hoc editing?

In a single‑pass pipeline the editor sees only the original output, so it cannot react to side‑effects introduced by its own edits. The iterative loop re‑evaluates the diagram after each accepted edit, guaranteeing that every subsequent edit operates on the most up‑to‑date structure and never degrades the global score.

Implementation builds on a FLUX.2‑klein‑base‑9B backbone trained in bfloat16 on eight B200 GPUs with DeepSpeed ZeRO‑2.

Stage 1: train on 656 K generation pairs at 768 px for 140 K steps (batch = 16, learning rate $10^{-5}$) using AdamW (weight decay $10^{-2}$) and maintain a float32 EMA with decay $\gamma=0.9999$.

Stage 2: initialize from the Stage 1 EMA, co‑train on 244 K high‑quality pairs plus 70 K editing triplets at $\sim\$1024 px for 90 K steps.

M‑DPO fine‑tuning: for each of 50 K prompts roll out $K=12$ candidates (four at 50 steps, eight at 25 steps), score with Qwen3‑VL‑8B‑Instruct, apply margin thresholds $\delta_{\text{min}}=0.25$, $\delta_{\text{max}}=0.60$, and accept the winner if its conjunctive score exceeds $\tau=0.70$.

Training runs for 4 K steps on four B200 GPUs (batch = 1, gradient accumulation = 4, learning rate $10^{-6}$, $\beta=2000$).

Verification and rollback logic executed after each edit candidate.

**Fig. 13. Anatomy of the Closed-Loop Editor.** Stage 1: The GPT-5.4 critic uses a 100x100 visual grid and priority hierarchy to localize defects. The raw bounding box is then refined and snapped to the 16x16 VAE layout. Stage 2: SciForma-9B dynamically generates candidate edits, which are subjected to a global monotonic guard ($S_{new} \geq S_{old} - \epsilon$). Updates that disrupt holistic layout quality beyond the tolerance margin are strictly rejected.

Benchmarking Structural Fidelity

Benchmarking structural fidelity of SciFormaBench‑2K across model families.

M‑DPO raises SciForma‑9B’s average structural fidelity to 69.51, overtaking GPT‑Image‑1.5 (68.96) while building on SciForma’s focus on component, arrow, and text correctness.

SciForma‑9B baseline averages 59.57; after applying the M‑DPO objective it reaches 69.51, a +9.94 point gain over the baseline and a 0.55 point lead over the strongest proprietary model.

**Table (a) SciFormaBench-2K**

VQA-Based Logic Recovery

SciForma‑9B outperforms baselines on the AIBench VLM benchmark.

We evaluate SciForma‑9B on the AIBench VLM benchmark and compare it against human‑drawn originals and prior generative baselines.

SciForma‑9B reaches an overall AIBench score of 70.29 %, edging human‑drawn originals (70.09 %) and beating GPT‑Image‑1.5 by 8.67 percentage points.

Table 2b shows the four‑level VQA scores; the overall metric aggregates Component, Topology, Phase, Semantics and aesthetic quality.

**Table 3.** PaperBanana benchmark (292 samples, Gemini-3-Pro judge). Scores are pairwise win rates against human-drawn originals (%).

Deconstructing M-DPO

We isolate each component of M‑DPO to see how it affects diagram quality.

The core premise is that SciForma struggles with rigid diagram constraints, and M‑DPO addresses this by jointly optimizing component, arrow, and text correctness.

M‑DPO attains a +1.92 average gain in just 4 K steps, whereas extending SFT by 30 K steps yields only +0.37.

Figure 5 shows the two training trajectories starting from the same checkpoint.

When compared with recent multi‑dimensional DPO variants, CaPO and MCDPO each cause a regression on the Text axis (‑1.34 and ‑1.85 respectively), whereas our full M‑DPO lifts every axis, reproducing the +1.92 overall gain and the strongest Text improvement (+3.16) observed earlier.

Qualitative Analysis

Human evaluation confirms SciForma‑9B’s structural gains translate to perceived quality.

SciForma‑9B’s automated quality score aligns strongly with human judgments.

Pearson correlation $r = 0.76$ across 30 evaluated samples.

**Fig. 6. User study.** Win/Tie/Lose percentages for SciForma-9B against Nano Banana Pro and Wan2.7-Image across four evaluation dimensions.

**Fig. 20.** Representative failure cases of SciForma-9B. (a) Rotated text: vertically oriented labels are garbled or collapsed to horizontal. (b) Arrow errors in complex topologies: spurious, missing, or misrouted connections in dense diagrams. (c) Iterative editing bottlenecks: imprecise bounding box detection and cropping in dense regions lead to uncoordinated inpainting and introduce new visual artifacts.

Limitations remain: the $1024$ px resolution caps fine‑grained rendering in dense diagrams; the verification‑and‑editing pipeline relies on a proprietary VLM, restricting reproducibility; and auto‑generated structural inventories can drift from user intent without human oversight.

Extended Ablation Studies

We isolate each modeling choice and report how its removal changes structural scores.

Our ablations test whether each design choice is essential for preserving structural fidelity across components, arrows, and text.

Adding editing‑triplet supervision during SFT improves the average structural score by +2.96.

Table 14 shows the “+ editing‑triplet” model outperforms the ablated version by +2.96 average, with +2.44 on Arrow and +2.66 on Text.

We next examine how the construction of preference pairs for M‑DPO influences performance.

Using ground‑truth images as winners harms all axes, dropping average score by ‑1.02.

Table 14 reports the “GT win” configuration at 66.57 (−1.02) versus the SciForma‑Base baseline of 67.59.

Replacing winners with GPT‑selected best‑of‑N candidates also reduces average performance by ‑0.48.

The “GPT1.5 win” row in Table 14 yields 67.11 (−0.48) relative to the baseline.

Uniform short rollouts (4 × 25 steps) give only a marginal average gain of +0.12.

Table 14 lists the “4×25 step” configuration at 67.71 (+0.12) over the baseline.

Mixing long and short rollouts (4 × 50 + 8 × 25 steps) yields the largest average improvement of +0.83.

Table 14 shows the “4×50+8×25 step” row at 68.42 (+0.83) and gains of +0.62 (C), +0.68 (A), +1.24 (T).

With a reliable data pipeline in place, we probe how many contrastive axes are needed.

Optimizing a single axis (D = 1, Component) raises average score by +0.55.

Table 15 reports “D=1 (C)” at 68.14 (+0.55) versus the baseline 67.59.

Optimizing Component + Arrow (D = 2) improves average by +0.79.

Table 15 shows “D=2 (C, A)” at 68.38 (+0.79).

Optimizing Component + Text (D = 2) improves average by +0.77.

Table 15 lists “D=2 (C, T)” at 74.29 (+0.77) on the Average column.

Optimizing Arrow + Text (D = 2) improves average by +0.91.

Table 15 records “D=2 (A, T)” at 74.43 (+0.91).

Optimizing all three axes (D = 3) yields the highest average gain of +1.92.

Table 15’s “D=3 (C, A, T)” entry reaches 69.51 (+1.92) average.

Finally we assess sensitivity to the KL‑divergence penalty $\beta$.

Setting $\beta$ = 1000 raises average score by +1.14.

Table 16 reports $\beta$ = 1000 with Avg = 68.73, a +1.14 increase over the baseline 67.59.

Setting $\beta$ = 3000 raises average score by +0.99.

Table 16 shows $\beta$ = 3000 with Avg = 68.58, a +0.99 improvement over baseline.

**Table 16.** $\beta$ sensitivity ablation on SciFormaBench-2K (GPT-5.4).

Extended Qualitative Comparisons

Qualitative and code‑baseline comparisons highlight SciForma‑9B’s structural superiority.

SciForma‑9B attains roughly double the average structural‑fidelity score of code‑based baselines on SciFormaBench‑2K.

Table 17 reports Avg 35.81 and 38.21 for TikZilla and GLM‑4.7‑Flash, versus 69.51 for SciForma‑9B.

**Table 17.** Code-based baselines on SciFormaBench-2K. Both score substantially below SciForma-9B, limited by template-driven renderer constraints.

Manual inspection of the code‑baseline outputs reveals three dominant failure modes: (i) severe layout overlap on complex diagrams, (ii) systematic arrow‑endpoint mismatches, and (iii) outright rendering failures that leave the diagram unreadable.

**Figure 14.** Comparative comparison with proprietary models (1/2). SciForma-9B vs. Wan2.7-Image, GPT-Image-1.5, and Nano Banana Pro on the same prompts

**Figure 15.** Qualitative comparison with proprietary models (2/2). Same setup as Figure 14.

**Figure 16.** Qualitative comparison with open-source models (1/2). SciForma-9B vs. SenseNova-U1, Qwen-Image-2512, and Z-Image.

**Figure 17.** Qualitative comparison with open-source models (2/2). SciForma-9B vs. FLUX.2-dev-32B and FLUX.2 Klein-9B.

**Fig. 18.** Further examples of iterative visual refinement. SciForma-9B accurately updates localized structures, texts, and arrow connections based on editing instructions.

**Figure 19.** Qualitative effects of M-DPO. The preference-aligned model (SciForma-9B + M-DPO) effectively corrects structural errors, repairs arrow connections, and improves text placement compared to the base model.

Despite overall robustness, SciForma‑9B still exhibits three persistent failure modes: (i) garbled or missing rotated text, (ii) spurious or missing arrows in densely connected topologies, and (iii) bounding‑box truncation that introduces artifacts during iterative editing.

We validate that structural‑inventory metrics correspond to perceptible quality differences via a pairwise preference study: 50 prompts (17 Simple, 17 Medium, 16 Hard) were evaluated on Faithfulness, Readability, and Aesthetics by three graduate annotators, yielding a Fleiss $\kappa$ indicating substantial agreement.

Multi-dimensional Preference Optimization

M‑DPO enforces joint correctness across components, arrows, and text via axis‑specific preferences.

Even after the two‑stage SFT pipeline, occasional structural mistakes—missing arrows or malformed subscripts—still invalidate a diagram, exposing the need for a finer‑grained correction step.

M‑DPO acts like a multi‑criteria judge that only passes a candidate if it wins every structural category, unlike standard DPO which looks at a single aggregated score.

How does M‑DPO differ from the standard DPO formulation?

Standard DPO pairs a single winner with a single loser and optimizes a binary preference; M‑DPO pairs one winner with D axis‑specific losers, aggregates their margins conjunctively, and reweights gradients so the most violated axis drives learning.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers