PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis
Qiushi Sun, Jingyang Gong, Yang Liu, Qiaosheng Chen, Zhangyue Yin, Zixian Huang, Hanxu Hu, Jing Xiong, Fangzhi Xu, Haochen Tan, Haoli Bai, Lifeng Shang, Xiaoguang Li, Ben Kao, Qipeng Guo, Lingpeng Kong, Jiahui Gao
PrismaForge uses synthesized, decomposed rubrics to curate high-quality multimodal code data and drive reinforcement learning.
How can we improve multimodal code generation by replacing holistic "judge" models with decomposed, rubric-based scoring?
Multimodal code models struggle to produce visually accurate artifacts because existing signals—like simple execution pass/fail or holistic similarity scores—fail to diagnose specific layout, rendering, or semantic errors. PrismaForge addresses this by synthesizing a sample-specific rubric for every instruction, decomposing quality into granular, checkable criteria for both code and rendered output. This pipeline produces PrismaCoder-2M, the largest open multimodal code dataset, enabling models that lead the open state of the art and rival proprietary systems.
Paper Primer
PrismaForge functions as a hierarchical data pipeline: it generates a custom rubric for each instruction, uses that rubric to filter supervised data, and employs the same rubric as a structured reward function for reinforcement learning. The core move is the decomposition of correctness into domain-shared and sample-specific criteria, which are evaluated separately for code and visual artifacts.
PrismaCoder models significantly outperform existing open-source multimodal code models.
On front-end generation benchmarks, PrismaCoder-9B and 35B achieve judge scores over 20 points higher than task-specific open baselines. PrismaCoder-35B leads open baselines on all VisPlotBench metrics, with a mean score 11.02 points higher than the previous best.
Rubric-based curation is more effective than holistic judge-based selection.
Models fine-tuned on rubric-filtered data consistently outperform those trained on holistic-score-filtered data, with gains reaching up to 24 points on Vega-Lite benchmarks. Rubric-based judging shows substantially higher agreement with human preferences than direct holistic scoring.
Why is a "decomposed rubric" necessary instead of just using a strong model to judge the final output?
Holistic judgments often compress complex errors—like a misaligned footer or incorrect button shape—into a single high scalar score, diluting the signal. Decomposed rubrics explicitly localize these failures, providing diagnostic feedback that is both more sensitive to perceptual errors and more consistent for training.
Does this approach require manual rubric design for every new task?
No; the system synthesizes rubrics automatically. It combines reusable domain-level criteria with instruction-specific requirements, adjusting the number of criteria based on the estimated difficulty of the task.
Introduction to Multimodal Code Intelligence
Multimodal code models stumble because holistic scores miss visual‑output errors.
Current multimodal code systems rely on a single holistic score, which hides whether a program is syntactically correct, semantically sound, or visually faithful. This entanglement prevents precise error diagnosis and limits open‑source progress because data pipelines cannot reliably filter or reward the right signals. PrismaForge addresses this gap by introducing a hierarchical rubric that evaluates code and its rendered output separately, then uses the same rubric for data curation and reinforcement learning.
It is the ability to assess a program both by its symbolic correctness and by the visual fidelity of its rendered output, so that a model must succeed on two complementary fronts.
The core gap is the mismatch between symbolic code correctness and visual output fidelity.
The PrismaForge Rubric Framework
PrismaForge introduces a flexible rubric system that filters data and guides reinforcement learning.
Holistic evaluation of multimodal code often masks specific visual or structural errors, leaving the data curation process noisy. PrismaForge addresses this by decomposing evaluation into granular rubrics that can both filter training data and serve as rewards for reinforcement learning.
PrismaForge splits evaluation into reusable domain rubrics and instruction‑specific criteria, using the same scores to select high‑quality samples and to shape policy updates.
How does PrismaForge differ from a single holistic scorer?
Instead of an opaque scalar, PrismaForge evaluates each sample against multiple weighted criteria, revealing why a sample passes or fails and providing a consistent reward scale for reinforcement learning.
**Figure 2.** PRISMAFORGE writes one rubric per instruction and uses it twice, to filter candidate data into PRISMACODER-2M and to reward rollouts in reinforcement learning.
The Elastic Rubric Scorer aggregates weighted scores from both domain‑shared and instruction‑specific rubric items across code and visual modalities.
Why not just use a fixed rubric for all samples?
A fixed rubric is too coarse; it cannot capture instruction‑specific requirements such as axis ranges or layout constraints, so the Elastic Rubric Scorer augments the shared criteria with sample‑specific items derived from the instruction.
Domain‑shared code criterion “bars evenly spaced” is satisfied → $f^{(c)}=1$, weight $w=0.5$.
Sample‑specific code criterion “exact bar heights” is partially satisfied (2/3 correct) → $f^{(c)}=2/3\approx0.67$, weight $w=0.5$.
Compute $S^{(c)}_i = (0.5\cdot1 + 0.5\cdot0.67)/(0.5+0.5)=0.835$.
Domain‑shared visual criterion “no overlap” is satisfied → $f^{(v)}=1$, weight $w=0.6$.
Sample‑specific visual criterion “y‑axis label present” fails → $f^{(v)}=0$, weight $w=0.4$.
Compute $S^{(v)}_i = (0.6\cdot1 + 0.4\cdot0)/(0.6+0.4)=0.6$.
Combine with $\lambda_c=0.7$, $\lambda_v=0.3$: $S_i = 0.7\cdot0.835 + 0.3\cdot0.6 = 0.7645$.
Even though the visual looks plausible, the missing axis label drags down the overall score, showing how sample‑specific criteria can penalize subtle errors that a holistic score would miss.
During reinforcement learning, the same rubric scores are normalized into advantages that guide policy updates, providing a consistent reward scale across diverse samples.
Why does normalizing advantages across rollouts matter?
Normalization removes sample‑specific score magnitude differences, so the policy receives comparable gradient signals regardless of how many rubric items a sample has or how hard it is.
**Figure 3.** Scored separately, the vision rubric catches a misplaced footer that every code check and a holistic judge let through.
Building the PrismaCoder-2M Dataset
Describes the data pipeline and gating mechanisms that build PrismaCoder-2M.
Multimodal code generation suffers from noisy, unfiltered training data; without fine‑grained quality control the model learns spurious patterns.
PrismaCoder-2M is a multimodal code model assembled from a supervised corpus that has been filtered and re‑synthesized by the PrismaForge pipeline.
Sample 1 (0.4) falls below $\tau_1$ → discarded.
Sample 2 (0.6) lies between $\tau_1$ and $\tau_2$ → sent back for re‑synthesis.
Sample 3 (0.7) also re‑synthesized; after regeneration it attains a score of 0.85 and joins the supervised pool.
Sample 4 (0.9) exceeds $\tau_2$ → retained directly.
Re‑synthesis recovers borderline samples that would otherwise be lost, increasing the effective size of the high‑quality corpus.
How does PrismaCoder‑2M differ from a conventional code model trained on raw web crawls?
Conventional models ingest all scraped code indiscriminately; PrismaCoder‑2M first scores each candidate with rubric‑based judges, discards low‑quality items, and only the vetted set is used for supervised fine‑tuning, followed by RL on frontier examples.
Iterative Data Gating repeatedly routes each candidate through execution checks and rubric scoring, regenerating failing samples until they satisfy domain‑specific quality thresholds.
$S_1$ < $\tau_1$ → discarded.
$S_2$ between $\tau_1$ and $\tau_2$ → regenerated; after feedback it reaches $0.78$ and passes $\tau_{d}$, so it joins the supervised pool.
$S_3$ ≥ $\tau_2$ → retained without regeneration.
The gating loop converts many low‑quality seeds into high‑quality training examples, reducing the need for massive raw data collection.
Why not simply discard all samples below $\tau_2$ instead of re‑synthesizing the mid‑band ones?
Mid‑band samples often contain useful structure; regeneration with rubric feedback can correct specific failures, yielding additional high‑quality data without expanding the raw seed set.
RL Data Curation isolates instructions where the SFT policy exhibits high outcome variability, using the dispersion $U_i$ to focus reinforcement learning on the model’s capability frontier.
Compute the mean $\bar{S}=0.64$.
Compute $U = \frac{1}{5}\sum (S_k - 0.64)^2 = 0.045$.
Because $U$ is relatively large compared to other instructions, this instance is selected for RL.
Large $U$ reveals that the SFT policy sometimes succeeds and sometimes fails on the same prompt, providing a rich learning signal for RL.
Why does high score dispersion $U_i$ make an instruction valuable for RL?
When the policy’s outcomes vary widely, the rubric can distinguish successful from failed attempts, giving the RL optimizer a clear gradient to push the policy toward consistent success.
**Figure 4.** Composition of PRISMACODER-2M by input modality and task family.
**Table 1.** Statistics of PRISMACODER-2M.
Experimental Results
PrismaCoder models set new open‑source benchmarks on visual and text‑centric code tasks.
We release PrismaCoder‑9B and PrismaCoder‑35B‑A3B, post‑trained from Qwen3.5‑9B and Qwen3.5‑35B‑A3B. The data pipeline reprocesses prior open releases and uses Claude‑Opus‑4.6 as the curation model and Gemini‑3‑Pro as the rubric generator.
Baselines include open multimodal code models Janus‑CoderV, VinciCoder, and VisCoder2‑7B. Evaluation follows each benchmark’s official protocol, reporting deterministic metrics alongside judge scores.
PrismaCoder‑35B‑A3B attains the highest low‑level LLM score among open models, 83.42, a 12‑point lead over the next best open baseline.
Table 2 shows PrismaCoder‑35B‑A3B reaching 83.42, surpassing the runner‑up JanusCoderV‑7B (75.12).
In vision‑centric settings, both PrismaCoder models dominate open baselines on front‑end generation, achieving judge scores over 20 points higher. On ChartMimic, PrismaCoder‑35B outperforms most proprietary models in both low‑ and high‑level evaluations.
**Table.** Data Visualization. On ChartMimic, both of our models lead all open baselines. Specifically, PRISMACODER-35B outperforms most proprietary models under both settings. Notably, our 9B model demonstrates the most substantial improvement over its base (the lowest among all evaluated models) and even surpasses most proprietary models. This clearly demonstrates the quality and utility of our data. In addition, the low-level score evaluates rendered text, layout, chart type, and color fidelity, which means our model is more capable of generating those details.
For text‑centric tasks, PrismaCoder models exceed all open baselines and even surpass GPT‑4o on overall scores. Scaling from 9B to 35B raises the InteractScience PFT score by 3.20 points, demonstrating consistent gains.
**Table 3.** Text-centric results. Best / second among open models.
On the chemical‑structure benchmark, PrismaCoder‑35B parses every output and attains the highest Tanimoto similarity (97.56%). In visual‑lens tasks, both models lead open baselines, confirming the versatility of the approach.
PrismaCoder‑35B delivers state‑of‑the‑art performance across visual benchmarks, setting a new open‑source standard.
Rubric Efficacy Analysis
We compare rubric‑based filtering to direct judging and examine stage‑wise gains.
Recall that the paper’s core claim is that holistic evaluation masks specific errors, so PrismaForge breaks evaluation into fine‑grained rubrics to filter data and steer reinforcement learning.
Each sample receives a single overall score, treating the whole multimodal artifact as an indivisible unit.
How does Direct Judging differ from the rubric‑based method introduced by PrismaForge?
Direct Judging collapses every modality into one scalar, whereas PrismaForge issues separate rubric scores for visual fidelity, code correctness, and their interaction, allowing the pipeline to keep or discard samples based on the precise failure mode.
PrismaForge‑filtered data yields higher benchmark scores than Direct‑judge‑filtered data, with gains up to 24 points on Vega‑Lite.
Fine‑tuning Qwen3.5‑9B on equal‑sized subsets shows the rubric‑filtered model outperforming on every benchmark, the largest gap being 24 points.
**Figure 5.** PRISMAFORGE filtering outperforms direct-judge filtering on every benchmark.
Applying the same data‑filtering pipeline to Gemma‑4‑31B‑it and Qwen3‑VL‑30B‑A3B‑Instruct lifts every metric above the base model, and adding RL on top of SFT improves all of them further.
Across twelve metrics on Python plotting and HTML, both backbones show positive gains after SFT, with RL adding additional improvement on every metric.
**Figure 6.** Per-stage gains of our SFT and RL pipeline over two base models on every metric.
Human studies on 100 samples reveal that rubric‑based judging aligns far better with human preferences than direct scoring, both in absolute scores and in the induced ordering of examples.
**Figure 7.** PRISMAFORGE scoring aligns judges with humans better than direct scoring.
Case Studies and Detailed Metrics
Benchmark results and case studies showcase PrismaCoder’s superiority across visual and code generation tasks.
PrismaCoder‑35B achieves the lowest incorrect‑code rate (28.86 %) and the highest mean task score (62.47) on VisPlotBench, far surpassing the best open baseline.
Table 10 and Table 12 show PrismaCoder‑35B at 28.86 % incorrect‑code, while the strongest VinciCoder model records 36.08 %; the mean task score gap is 23.91 points.
**Table 10.** VisPlotBench averaged metrics.
**Table 11.** UniSVG.
**Table 11.** Full UniSVG metrics (Li et al., 2025).
**Table 13.** ChemDraw, full metrics on the 128-sample test set. Tani. (mean) averages over RDKit-valid outputs; the other columns count all samples.
The table presents performance metrics for various models across "Short", "Mid", "Long", and "Average" categories, evaluating them on "LLM", "Visual", "CLIP", and "Tree-BLEU" criteria. The models are divided into "Open models" (JanusCoderV-7B, JanusCoderV-8B, VinciCoder-7B, VinciCoder-8B, PrismaCoder-9B, PrismaCoder-35B) and "Proprietary" models (GPT-4o, Gemini 3.1 Pro).
**Table 15.** DesignBench, full metrics for generation and editing.
**Table 16.** Design2Code, full metrics.
**Table 17.** ArtifactsBench, overall and per-domain scores.
**Table 18.** InteractScience, full metrics for both lenses.
**Table 20.** Agreement of judge scores with human ratings on 100 samples. Best per judge.
**Table 20.** Lists the agreement statistics of the human study of §6.3.
**Figure 8.** Prompt template for synthesizing code rubrics in Web UI generation.
Data Toolkit and Benchmarks
This appendix details the data toolkit, benchmarks, and training pipeline.
In A.1 we parse each plotting script or Manim scene into an abstract syntax tree, then split it into self‑contained units—one unit per figure or scene. Each unit retains its imports and dependent statements, drops private helpers, and stores embedded text and a short excerpt as features for downstream pipelines.
A.2 runs every candidate in a language‑matched sandbox: Python code must produce a figure, HTML pages render to a screenshot, and formats such as SVG, LaTeX, Mermaid, Asymptote, LilyPond, and Vega‑Lite use their standard renderers. A candidate passes if it finishes within the time limit and yields a non‑empty artifact; failures (error, timeout, empty output) receive a reward of 0 in the RL stage.
A.3 decontaminates PrismaCoder‑2M by removing any training sample whose instruction shares a long $n$‑gram with a test instruction from the benchmarks in §5.2, ensuring no leakage into evaluation.
B lists the raw counts per data source and the numbers retained after each gating stage (Table 4). High‑quality, task‑aligned sources keep more than half of their original samples, whereas broader sources like GitHub and arXiv require extensive regeneration.
C describes how we synthesize two binary rubrics per sample: a code rubric that checks implementation‑level properties and a vision rubric that checks rendered‑output properties. The rubrics inherit the instruction’s difficulty as an item budget, so harder samples receive more checks.
D details the RL reward computation: code and visual scores are combined with task‑specific weights ($\lambda_c$, $\lambda_v$, $\lambda_h$) as shown in Table 7. Unscoreable rollouts (missing or malformed code, execution failure, or persistent judge error) receive a reward of 0.
E reports benchmark results. Table 8 gives the full ChartMimic metrics, and Table 9 provides PandasPlotBench metrics under both visual and task lenses. These tables complement the headline numbers in the main text.
F outlines the training setup: 32 GPUs (8 × H200 per node) with Megatron, FlashAttention, and sequence parallelism. SFT runs for 5 epochs at a peak learning rate of $1\times10^{-5}$; RL runs one epoch of GRPO/GSPO with learning rate $3\times10^{-6}$, KL coefficient $\beta=0.04$, and asymmetric clipping thresholds $\epsilon=0.20$, $\epsilon_{\text{high}}=0.28$.
G aggregates the per‑benchmark results: PrismaCoder‑9B and PrismaCoder‑35B consistently rank among the top open models, with the larger model achieving scores close to proprietary systems on both ChartMimic and PandasPlotBench.
**Table 4.** Statistics on the samples used from each data source, as well as the number of samples processed at each stage of the gating procedure.
**Table 1.** Comparison of PRISMACODER-2M with existing visual code datasets and models.
**Table 7.** RL rewards by task family. Rewards lie in $[0, 1]$; unscoreable completions receive 0.
**Table 9.** PandasPlotBench, full metrics under the Visual and Task
**Table 9.** PandasPlotBench, full metrics under the Visual and Task lense