3DCodeVerse: Exploring 3D Worlds as Code
Yipeng Gao, Ziyao Zeng, Jinfa Huang, Yijiang Li, Haolin Xiong, Wang Qin, Xinlei Yu, Youheng Yao, Jie Zhang, Kun-Yu Lin, Rong Liu, Yinyuan Zhao, Yajie Zhao, Meiqi Guo, Jiebo Luo, Lei Shu, Yunhao Ge, Xinyu Hu, Laurent Itti
A unified framework for 3D code generation that uses automated execution, spatial checks, and VLM feedback to refine complex 3D assets without human intervention.
How can we scale the generation of high-quality, verifiable 3D content by treating 3D assets as executable code rather than static geometry?
3D coding agents often struggle to produce geometrically valid or visually accurate 3D content because they lack a shared, automated way to verify and repair their output. The 3DCodeVerse Harness solves this by coordinating coding agents through a directed acyclic graph (DAG) that executes code, measures spatial consistency, and uses a vision-language model (VLM) to iteratively repair defects. This framework enables scalable data curation and model training, with fine-tuned models achieving up to 98.5% execution rates and the harness consistently outperforming native generation baselines.
Paper Primer
The 3DCodeVerse Harness acts as a closed-loop data engine: it treats 3D generation as a compile-debug cycle where the harness acts as the compiler and the VLM judge acts as the debugger. By decomposing complex projects into a DAG of tasks, the system allows independent agents to code and assemble components while the harness enforces structural constraints like contact, collision, and kinematic validity.
Fine-tuning on the 3DCodeVerse-2M dataset drastically improves the executability of open-source models.
Qwen3.5-9B execution rates on 3DCodeBench rose from 0% to 84.9% after fine-tuning, while Qwen3.8-27B improved from 51.9% to 92.9%. Up to 88.4 percentage point improvement in execution success.
The 3DCodeVerse Harness significantly improves generation quality over native model-specific workflows.
In pairwise human evaluations, the harness achieved a 90% win rate against Claude Design (3D Object) when using the same backbone (Claude Opus 5.5). 15.5–29.1 point gain in preference scores over one-shot generation.
Why is a specialized harness necessary for 3D coding compared to standard code generation?
Standard code generation only requires syntactic correctness, but 3D coding requires geometric and physical validity (e.g., parts must not float or interpenetrate). The harness provides the necessary spatial tools and deterministic gates to verify these 3D-specific constraints that standard compilers cannot detect.
What is the role of the "adversarial loop" in the harness design?
The adversarial loop alternates between refining the VLM judge's rubric and the generation workflow. This ensures that the automated feedback remains consistent and reliable, preventing the judge from hallucinating defects or contradicting hard geometric measurements.
3DCodeVerse shifts 3D modeling from manual design to a scalable, agentic process where code is treated as a verifiable, editable, and improvable artifact.
Introduction to 3DCodeVerse
We expose the missing infrastructure that lets agents write, verify, and refine executable 3D code at scale.
Current 3D‑coding research suffers from fragmented resources: datasets focus on static meshes or single‑task code, and harnesses provide only limited visual feedback or geometry checks. This fragmentation prevents agents from learning a unified, verifiable representation that can be edited, composed, and scaled.
Instead of storing a finished mesh, the asset is stored as a program that can generate the geometry, articulation, and rendering on demand.
The ecosystem bundles a massive corpus of executable 3D programs with a multi‑agent DAG harness that can generate, verify, and iteratively improve those programs.
**Figure 1.** 3DCodeVerse’s showcases. Assets are built **only by code** (e.g., Blender Python, Three.js, and OpenGL), ranging from 3D objects to articulated objects, scenes, and graphics renderings.
Table 1 makes the gap explicit: prior works either lack a harness that can iteratively refine code, omit visual feedback, or support only a single asset type. None provide a unified, large‑scale dataset of executable programs covering static, articulated, scene, and graphics domains.
The key shift is moving from static mesh files to executable 3D code as the primary representation for generative 3D modeling.
Dataset Construction and Benchmarks
We assemble a million‑scale, executable 3D coding corpus spanning CAD, Blender, shaders, and web graphics.
Public CAD and graphics corpora such as DeepCAD, Fusion 360 Gallery, and Text2CAD expose construction sequences, while executable CadQuery and Blender programs are provided by BlendNet, CAD‑Recode, CAD‑Coder, and MeshCoder. Complementary procedural generators like Infinigen and Shaders21k broaden the code‑based asset space.
Our 3DCodeVerse‑2M dataset aggregates these heterogeneous sources into a million‑scale corpus of self‑contained programs, each runnable in its target runtime (CAD, Blender, shaders, or browser graphics). The collection is built by curating existing assets, auto‑generating new programs, and filtering them through execution‑based validation.
The 3DCodeVerse harness supplies a reusable execution and verification framework across backends. It orchestrates sub‑agents via a directed acyclic graph (DAG), ensuring that rendering, spatial measurement, and structural checks run without cyclic dependencies.
Success requires both executability ($\ell \neq \bot$) and high‑quality content, measured by a task‑specific distance $D(a, a^\star)$ when a reference artifact $a^\star$ exists, or by programmatic metrics and renders otherwise. This dual criterion distinguishes 3D coding from ordinary code generation.
We draw source material from Bioinspired3D, 3DCodeBench, Thingiverse, Shadertoy, CodePen animations, and Three.js/React Three Fiber projects. DeepCAD sequences and Articraft geometry are deterministically converted to CadQuery, while ProcTHOR provides paired scene specifications and reconstructions for Blender and Three.js.
After execution we run measurement tools $M_e(a)$ to extract part inventories, bounding boxes, collision checks, and physical support probes. Conversion fidelity is quantified by voxel IoU for CAD meshes and Chamfer distance for articulated assets, providing objective quality signals.
The 3DCodeVerse Harness
Automating 3D code generation with a multi‑agent DAG and closed‑loop feedback.
Human‑in‑the‑loop refinement of 3D programs does not scale, because each iteration requires a person to inspect builds, measurements, and renders.
The harness turns a textual prompt into a directed‑acyclic‑graph of independent coding sub‑tasks, runs the assembled program, gathers automated execution, measurement, and visual feedback, and feeds targeted revisions back to the sub‑agents—no human in the loop.
Agent A produces a cube mesh (10 KB); Agent B assigns a red material (2 KB).
Both outputs become available, enabling Agent C to export the combined OBJ (12 KB).
The program $f^{(0)}$ is executed; the build log $\ell^{(0)}$ reports success, and the VLM judge returns a visual score of 0.58.
Revision $\Delta^{(0)}$ contains a single visual defect (“missing back face”). Agent A updates the mesh, yielding $f^{(1)}$.
Second execution yields a visual score of 0.71, demonstrating improvement after one automated refinement round.
The example shows how parallel coding tasks reduce wall‑clock time, while the feedback loop guarantees monotonic quality improvement without any human inspection.
How does this DAG differ from a conventional build pipeline?
In a standard pipeline tasks are usually linear and centrally scheduled. Here the DAG explicitly encodes prerequisite edges, allowing independent sub‑agents to run concurrently and to be re‑invoked only for the tasks that the feedback flags as defective.
Start with initial program $f^{(0)}$ produced by the planner.
Execute $f^{(t)}$ to obtain artifact $a^{(t)}$ and logs $\ell^{(t)}$.
Run measurement tools $M_e$ on $a^{(t)}$ to get numeric violations.
Render $a^{(t)}$ from multiple views and feed to the VLM judge to compute $j^{(t)}$.
Combine execution, measurement, and visual feedback into revision tasks $\Delta^{(t)}$ (prioritizing execution failures, then measurement violations, then visual defects).
Invoke the fix agent to produce $f^{(t+1)} = \text{Acode}(c, P, f^{(t)}, \Delta^{(t)}; e)$.
Repeat until the score $s^{(t)}$ stops improving or a maximum round count is reached.
**Figure 3.** Overview of the 3DCodeVerse Harness. Given an input prompt, an LLM planner defines the target asset, and a DAG coordinates sub-agents to code and assemble a 3D program (left). Execution checks, spatial measurements, and rubric-guided VLM judgments provide automated feedback (center) for targeted code revisions by a fix agent (bottom). The right panel illustrates successive refinement rounds, with the highest-scoring program selected as the final output.
Experimental Results
3DCodeVerse Harness beats native harnesses on every 3D coding benchmark.
3DCodeVerse Harness outperforms native harnesses across all 3D tasks, reaching up to 90 % human preference.
Figure 4 shows the consistent advantage of the harness over native baselines.
A native harness runs a model’s generated code in the target 3D engine without any extra verification or feedback loop.
How does the 3DCodeVerse Harness differ from a native harness?
The native harness simply runs the generated program; the 3DCodeVerse Harness adds execution‑feedback checks, geometry validation, and a scoring loop that discards or refines failures before any metric is recorded.
**Figure 4.** Main results of 3DCodeVerse Harness across four 3D coding tasks. Panels show (a) 3D objects, (b) articulated objects, (c) scenes, and (d) graphics. Blue bars report Human Preference (%), and orange bars report Judge Score (0–100; failed generations receive a score of zero). Hatched bars denote each backbone’s native harness, while solid bars denote 3DCodeVerse Harness. The final group in (a) compares Claude Design with 3DCodeVerse Harness, both using Claude Opus 5.5.
Tables 2 and 3 quantify the impact of fine‑tuning on executability and fidelity. After LoRA training on 3DCodeVerse‑2M, Qwen3.5‑9B’s execution rate on 3DCodeBench rises from 0 % to 84.9 % (+84.9), and its F@0.05 on Blender improves from 0.000 to 0.317 (+0.317). Similar trends appear for Qwen3.8‑27B.
**Figure 5.** Qualitative comparisons and automated refinement. (a) Outputs for identical prompts and backbones using the native harness (Without) versus 3DCodeVerse Harness (With). (b) Selected rounds of automated refinement for objects and graphics. Numbers report rubric scores on a [0, 1] scale and are comparable within each pair or refinement sequence.
3DCodeVerse Harness consistently outperforms native model harnesses across diverse 3D tasks.
Advanced Harness Configuration
The harness enforces deterministic verification and a strict protocol to keep 3D code execution reliable.
Complex 3D asset pipelines often stall because builds produce nondeterministic failures that downstream judges cannot reliably interpret. The harness solves this by inserting fast, deterministic checks that either certify the output or generate concrete revision tasks, keeping the pipeline moving.
After each build the harness runs a small set of fast checks that either PASS the output or produce a concrete FIX hint; only PASSed builds are allowed to reach the visual judge, preventing spurious defects from contaminating the score.
After building, the connectivity gate measures the distance between B’s attachment face and A’s corresponding socket: 5 mm.
The gate returns FAIL with a fix hint: translate B by (0, 0, −0.005) m in the authoring frame.
The coding agent applies the translation, rebuilds, and the gate re‑evaluates the distance: 1 mm → PASS.
The gate not only flags the violation but also supplies an exact corrective vector, turning a vague “mis‑alignment” into a concrete edit that the agent can execute automatically.
How do deterministic gates differ from the visual judge’s defect detection?
Gates operate on the raw geometry immediately after the build and can abort the round on a FAIL, whereas the visual judge only scores already‑passed builds and cannot veto a defect that a gate has already blocked.
MCP defines a strict API between the coding agent and the harness, guaranteeing that every code execution follows a sandboxed build wrapper, adheres to a fixed file layout, and returns structured PASS/FAIL verdicts plus optional diagnostic renders.
Why does the harness need a separate MCP instead of letting the coding agent run arbitrary code directly?
MCP isolates the agent, enforcing a known file layout and deterministic rendering; without it, a stray script could modify unrelated files, cause nondeterministic builds, and break the guarantee that every PASS corresponds to a reproducible geometry.
Weighted sum: $1.0 \times 0.78 = 0.78$.
Subtract penalty: $0.78 - 0.5 \times 0.2 = 0.78 - 0.10 = 0.68$.
Clamp at zero (already positive): $0.68$.
Apply upper bound: $\min(0.85, 0.68) = 0.68$.
This example shows how a modest defect can pull the round score below the gate‑imposed ceiling, ensuring that deterministic failures always dominate the final metric.
**Figure 7.** **Components and model roles.** A planner, a judge, and a coding agent are configured modularly per run; the harness orchestrates stage progression, iterative refinement, and candidate selection while providing deterministic spatial tools, verification gates, multi-view renderers, and rubric aggregation. Agent programs execute across seven isolated language runtimes; dashed circles indicate pluggable model interfaces.
**Figure 8.** From structural plan to file scopes and coding sessions, illustrated on the recorded trajectory of §A.6. The planner decomposes the prompt into named parts with metric bounding boxes, materials, attachment relations, and an acceptance checklist; each target maps to its owned source files, and refinement tasks sharing files are grouped into file-disjoint sessions. In Round 1 nine revision tasks partition into eight file-disjoint groups; in Round 2, four tasks form four groups.
**Figure 9.** Observations, deterministic gates, and authoring-frame feedback. A measured geometric defect is returned as a structured negative verdict rather than a protocol error (left); a single verification routine serves simultaneously as an interactive agent tool, an end-of-round gate, and a generator of prioritized revision tasks (middle). Right: baseline-round diagnostics from §A.6, with corrective translation vectors projected directly into the agent's native authoring frame.
**Figure 10.** **Vision–language judgment: evidence payload, structured schema, and score synthesis.** The judge inspects structured telemetry and labelled multi-view montages without access to source code or generator reasoning (left); its schema enforces explicit visual observation prior to criterion scoring (middle). Right: baseline-round score computation for §A.6, combining criterion means, majority-voted defects, and deterministic gate caps.
Automated Iterative Refinement
Iterative refinement resolves geometry errors through staged verification and targeted repairs.
Each refinement round runs a fixed verification chain—code synthesis, bounded repair, geometric measurement, deterministic gates, multi‑view rendering, VLM judgment, and archival. The system caches pre‑round stages but forces a fresh execution for every round to guarantee that new fixes are evaluated against the full pipeline.
Think of the process as a disciplined repair workshop: each round inspects the model, extracts the most concrete errors, and applies a tiny, targeted fix before moving on.
Round 0 synthesizes two candidate scripts and ranks them by build success; the winner proceeds to Round 1.
Round 1 detects a floating leg (gate error) and a missing contact; the repair engine applies a translation vector to the leg and adds a support bracket.
After the fix, the deterministic gate reports no errors; the round makes no further code change, triggering early termination.
Because all tasks are resolved before the budget expires, the final model is selected from Round 1.
Iterative refinement guarantees that each round only addresses the most concrete, verifiable defect, preventing wasted computation on speculative changes.
How does this iterative refinement differ from a standard gradient‑descent optimizer?
Standard optimizers adjust continuous parameters based on a scalar loss, whereas iterative refinement works on discrete code and geometry artifacts. Each round extracts explicit error locations (e.g., a specific interpenetrating wedge) and applies a targeted, rule‑based repair, so the “gradient” is a concrete edit rather than an infinitesimal update.
Cache pre‑round assets (meshes, scripts) to avoid redundant I/O.
Execute the verification chain: synthesize code, build with bounded repair, measure geometry, run deterministic gates, render multiple views, query the VLM judge, and archive the run.
Collect all task candidates from the previous round and sort them by priority: gate errors → mandatory checklist → perceptual suggestions.
Deduplicate tasks by part and defect category; dispatch up to six tasks for the current round.
Apply targeted repairs (e.g., translate a part, adjust a mesh, insert a missing joint) and record telemetry.
Check termination conditions: all tasks resolved, no code change, or wall‑clock budget exceeded; if not terminated, proceed to the next round.
**Figure 11.** Recorded trajectory for the prompt “a three-legged wooden stool” on the Three.js runtime under the economy profile. Each column shows one round: the rendered view inspected by the judge, the composite score with its penalties and caps, deterministic gate findings, and VLM defect votes. Closed-loop refinement resolves all baseline floating and interpenetrating parts in Round 1 (0.410 → 0.844) and refines geometric detailing in Round 2 (11,052 triangles, reaching 0.933 under the strict metric-footprint cap), completing in 4.0 active minutes at \$0.74.
Table 4 enumerates the domain‑skill library: each bundle pairs an authoring guidance with standing and gate‑triggered routing conditions, enabling the harness to summon the right repair routine when a specific gate fires.
The 3DCodeVerse-2M Dataset
Dataset details: scale, composition, and sources of the 3DCodeVerse-2M collection.
3DCodeVerse-2M is roughly ten times larger than the next biggest 3D code dataset.
Table 5 lists 269,597 unique programs versus about 28,000 in the prior largest dataset.
It is a curated corpus of 269,597 executable 3D programs, each paired with rendered images, captions, and multi‑view exports.
How does 3DCodeVerse-2M differ from earlier 3D code datasets?
Earlier datasets offered raw scripts or isolated assets; 3DCodeVerse-2M provides fully executable programs verified by a harness, with systematic deduplication and multi‑modal annotations, enabling closed‑loop training.
**Figure 2.** Curation and composition of 3DCodeVerse-2M. Left: We collect programs from public datasets and generate more with our harness. Middle: Our 3D coding harness, equipped with modeling skills and inspection tools (e.g., 3D IoU, collision checks), builds each program, judges the output, and refines it until it passes. Each verified program is paired with rendered images and captions, plus meshes and videos when available. Right: The resulting corpus covers 3D objects, articulated objects, scenes, and graphics across multiple programming languages and engines.
**Figure b.** Graphics: a fragment shader under a harness-owned header, judged on sampled frames
Evaluation Protocol
We quantify execution success, geometry fidelity, and visual quality across models using the 3DCodeVerse Harness.
3DCodeVerse Harness lifts Gemini 3.8 Flash Judge Score by +25.3 points (86.6 vs 61.3) on the 60‑prompt object benchmark, winning 56 of 60 prompts.
Table 14 reports the paired comparison; the harness‑wrapped model outperforms the native baseline across all seven weighted criteria.
The evaluation pipeline first extracts fenced code blocks, validates AST syntax, and runs each program in a pinned runtime with a 300 s timeout. Meshes are normalized, sampled at 10 k points, and scored by symmetric Chamfer‑L2 distance and F‑score at $\tau$ = 0.05/0.10, taking the best alignment over four 90° yaw rotations.
**Table 14.** **3D object generation.** Numerical values of Figure 4a–c. Comparisons at a fixed backbone (upper) and against Claude Design (lower), on identical prompts. SigLIP-2 reports cosine similarity; Judge Score reports overall rubric quality (0–100).
**Table 8.** Object execution reliability, token telemetry, and prompt-cache efficiency. Usage is averaged across all 60 prompts and excludes post-run evaluation calls. Over 93.9% of 3DCodeVerse Harness input tokens are served from low-cost prompt cache.
**Table 9.** Paired cross-track gains on 3D scenes and procedural graphics (0–100 scale). Differences compare 3DCodeVerse against the native baseline of the identical backbone across paired prompts (n), reporting both composite Judge Score ($\Delta S_{\text{all}}$) and uncapped visual score ($\Delta S_{\text{vis}}$).
The table presents performance metrics across two tracks ("Graphics" and "Scenes") using different backbones (Flash, Sonnet, Luna, Astra). It includes columns for the number of samples ($n$), the change in all scores ($\Delta S_{\text{all}}$), the 95% confidence interval (95% CI), the win/loss/tie ratio (W/L/T), and the change in visual scores ($\Delta S_{\text{vis}}$). Flash, Sonnet, Luna, and Astra denote Gemini 3.8 Flash, Claude Sonnet 5, GPT-6 Luna, and GPT-6 Astra. For graphics, $S_{\text{vis}}$ retains all criterion weights prior to aggregate penalties and caps; for scenes, it additionally excludes the spatial-scale criterion.
**Table 10.** Independent generation runs on open-ended prompts naturally explore diverse structural decompositions: in an A/A study across six paired runs with early pass stopping (Table 10; $\sigma_{\Delta} = 0.202$), as well as a 12-pair articulated study with a fixed plan ($\bar{\Delta} = -0.046$, $\sigma_{\Delta} = 0.225$, 2 SE = $\pm 0.130$), a single prompt may be realized either as a compact monolithic mesh (1 part, 4,796 triangles) or as a fine-grained multi-part assembly (10 parts, 61,340 triangles). This structural diversity motivates two core design choices in our work: (1) within 3DCodeVerse Harness, best-of-$N$ candidate initialization and multi-round refinement (§A.5) hedge against single-rollout topological variance; and (2) in our experimental design, while full-workflow gains of $+0.15$ to $+0.49$ are cleanly resolved on 30–60 paired prompts, fine-grained infrastructure optimizations are evaluated directly via high-power event-level telemetry across hundreds to thousands of invocations (§C.5).
**Table.** Post-training with execution, geometry, and termination DPO. We construct preference pairs by sampling multiple completions per training prompt and ranking them along three verifiable axes—execution success (running vs. failing scripts), geometric fidelity (highest vs. lowest F@0.05 with $\Delta F \ge 0.15$), and sequence termination—and optimize the merged SFT checkpoints with DPO (Rafailov et al., 2023) (Tables 12 and 13; mixture specifications in Table 6).
Ablations and Model Alignment
Ablations quantify how each harness component improves execution reliability and geometry fidelity.
Recall that 3DCodeVerse treats 3D assets as executable programs, enabling a closed‑loop harness that verifies geometry and execution success.
Direct Preference Optimization that rewards programs which run to completion, penalizing failing scripts, thereby steering the model toward reliably executable code.
How does Execution‑feedback DPO differ from standard DPO that uses language‑model likelihood?
Standard DPO optimizes a preference over textual quality (e.g., human ratings). Execution‑feedback DPO replaces the textual reward with a hard execution check, so the model learns to prefer code that actually runs, not just looks plausible.
Execution‑feedback DPO raises greedy 3DCodeBench execution from 75.0 % to 96.2 %.
Table 13 reports the improvement on the 9B (5.2 K) checkpoint after two DPO rounds.
Execution‑feedback DPO improves four‑seed pass@4 from 91.0 % to 99.1 %.
Best‑of‑N sampling results in Table 13 (T = 0.7, N = 4).
Geometry‑feedback DPO lifts conditional F@0.05 from 0.387 to 0.403.
Table 13 shows the geometry‑feedback ablation on the 9B (270 K) checkpoint.
Termination‑feedback DPO cuts mean OpenSCAD output length from 30,163 tokens to 2,884 tokens (≈10.5× reduction).
Text describing two lightweight DPO rounds (1,779 and 478 pairs) reports the token reduction.
Termination‑feedback DPO boosts OpenSCAD greedy execution from 8.0 % to 86.0 %.
Same paragraph reports the execution jump after the two DPO rounds.
Multi‑dialect training raises CadQuery execution to 98.0 % (up from 73.0 % in single‑dialect).
Section text on cross‑dialect transfer cites the 98.0 % figure.
Increasing Blender share in the 9B mixture lifts F@0.05 from 0.705 to 0.816.