Agentic 3D Open-World Generation at Scale

Chunchao Guo, Jinpeng Li, Yang Li, Zilong Huang

WorldClaw is an agentic framework that generates large-scale, editable 3D worlds by decoupling global terrain foundation from regional object placement.

How can we generate large-scale, coherent 3D worlds from open-ended text by decomposing the task into a hierarchical, agentic pipeline?

Generating large-scale, explorable 3D worlds from text is difficult because systems struggle to balance global spatial coherence with the precise, instance-level control needed for editing and reuse. WorldClaw solves this by adopting a coarse-to-fine strategy: it first establishes a global terrain foundation using semantic layouts, then selectively populates specific regions with editable 3D assets. This approach allows the system to maintain consistent global geometry while enabling fine-grained, region-aware object generation and placement.

Paper Primer

WorldClaw functions as a multi-stage agentic pipeline that translates open-ended prompts into structured scene specifications. It hinges on a global-to-regional principle: the system treats the terrain as a persistent, semantically partitioned foundation, while objects are generated as independent, editable meshes that are later anchored to the terrain surface.

WorldClaw achieves controllable, large-scale 3D scene generation with instance-level editability.

The framework produces scenes represented as independently editable textured meshes with explicit terrain placements, supporting free-viewpoint rendering and standard game-engine workflows. The system successfully bridges the gap between high-level semantic planning and precise 3D spatial instantiation, allowing for complex object-terrain relationships that previous MLLM-agent methods failed to resolve.

Why does WorldClaw use a coarse-to-fine approach instead of generating the entire 3D scene in a single pass?

Generating an entire world at once often leads to a loss of global consistency or geometric fidelity. By decoupling the global terrain foundation from regional object generation, the system ensures that large-scale landforms remain coherent while allowing for high-quality, instance-level detail in specific areas.

How does this framework differ from existing image-to-3D or procedural generation methods?

Unlike procedural methods that rely on hard-coded rules, WorldClaw uses agentic planning to interpret natural language. Unlike image-lifting methods, which often suffer from blurring or incoherent geometry at novel viewpoints, WorldClaw constructs explicit, editable 3D meshes that preserve spatial relationships.

Researchers and developers can now treat 3D world generation as a structured, agent-orchestrated pipeline rather than a monolithic synthesis task, enabling the creation of persistent, editable environments suitable for simulation and game production.

Introduction

WorldClaw tackles the intractable single‑pass generation of large‑scale 3D worlds by decomposing the task hierarchically.

Generating an explorable 3D world from a single textual prompt demands simultaneous global spatial consistency, rich local detail, and editable assets—requirements that clash when a system tries to produce everything in one pass.

It attempts to synthesize an entire world in one go, treating the scene as a single undivided output.

Existing approaches fall into four families: procedural content generation (PCG) offers scalability but limited expressiveness; image/video‑lifting yields rich visuals yet suffers from global inconsistency and high compute; native 3D diffusion provides geometric fidelity but lacks diverse content; multimodal LLM agents excel at planning but miss precise spatial control.

The core difficulty is balancing global coherence with local detail while keeping the result editable.

This image is a circular avatar or icon featuring a stylized illustration of an anime character with spiky, light-colored hair with green tips, viewed in profile facing right.

N/A

The WorldClaw Framework

WorldClaw builds coherent 3D worlds by first planning globally, then generating terrain, and finally populating regions.

WorldClaw follows a coarse‑to‑fine, global‑to‑regional strategy to avoid the intractability of monolithic 3D generation. By establishing shared constraints globally first, the system limits combinatorial explosion before committing to local detail.

WorldClaw treats world creation as a hierarchy of agents that first agree on a global layout, then flesh out terrain, and finally add detailed objects only where needed.

Stage 1: $P = F_{\text{plan}}(q)$ produces a plan that assigns Region A to “lake” and Region B to “village”.

Stage 2: $T = F_{\text{terrain}}(P)$ generates a flat water surface in Region A and a gently sloped ground in Region B.

Stage 3: $O = F_{\text{region}}(P, T)$ renders a 2‑D image of Region B, runs an image‑editing model to add a cottage, then lifts the cottage to a 3‑D mesh and places it on the terrain.

The refinement agent checks that the cottage sits on solid ground; it nudges the mesh upward by $0.2$ m to resolve a tiny penetration.

This toy example shows how only Region B incurs the expensive object‑generation cost, while Region A remains a cheap procedural terrain.

This image is a logo or icon depicting a blue, spherical globe with a grid pattern, overlaid with three diagonal red claw marks.

**Figure 1.** An overview of the WorldClaw framework, demonstrating the generation of a large-scale 3D world with four distinct seasonal biomes: Spring, Summer, Autumn, and Winter. The central circular graphic displays a unified terrain map divided into these four quadrants, while the surrounding panels provide detailed, ground-level views of the specific assets and environments generated for each season.

**Figure 1.** Overview of WorldClaw. Given an open-ended text prompt, WorldClaw constructs an explicit, explorable, and editable 3D world through a three-stage global-to-regional pipeline. (1) Intent Analysis and Planning translates the prompt into a structured scene specification. (2) Global Terrain Generation establishes a region-aware terrain foundation with coherent geometry, appearance, and spatial semantics. (3) Regional Object Generation and Placement populates planned regions with editable 3D assets and refines their arrangements and interactions with terrain.

Intent Analysis and Planning

Convert open‑ended prompts into a structured scene specification for downstream generation.

We first catalog exactly what the user said, then we fill in the missing details—like a librarian records a book’s metadata before the story is written.

How does this differ from a naïve “prompt‑to‑scene” model that directly generates geometry?

The naïve model tries to infer all missing geometry and style from the short prompt, which often leads to inconsistent or unintended details. Our two‑stage approach first locks down the explicit constraints, then fills in only the missing pieces required by the schema, guaranteeing coherence across terrain and object generation.

Receive user prompt $q$ (e.g., “a misty forest with a river”).

Intent analysis agent extracts explicit constraints: scene type = forest, style = misty, key region = river.

Scene‑planning agent resolves ambiguities (e.g., “river” → location, width) and adds required fields (terrain type, object densities) according to the schema.

Emit structured specification $P = (R, C_{\text{terrain}}, C_{\text{object}})$ for downstream modules.

Intent analysis extracts explicit constraints: scene type = beach, style = sunny, key region = shoreline, objects = palm trees.

Scene‑planning fills missing attributes: $R$ = {shoreline, dunes}; $C_{\text{terrain}}$ = {sand texture, gentle slope}; $C_{\text{object}}$ = {palm tree count ≈ 5, spacing ≈ 3 m}.

Structured specification $P$ is assembled as the tuple of those three components.

The two‑stage process guarantees that the “sunny” style is applied uniformly to both terrain and objects, preventing the downstream generators from interpreting style inconsistently.

Global Terrain Generation

Layered terrain synthesis resolves the limits of planar world bases.

Most existing generators flatten the ground to a plane or a weakly varying surface, which cannot express mountains, canyons, or the semantic region structure needed for coherent worlds. This planar assumption forces downstream object placement to ignore terrain‑driven semantics, limiting realism and navigability.

Think of the terrain as a layered canvas where each region paints its own height and texture, then the layers blend smoothly to form a continuous landscape.

How does this differ from a classic height‑map generator that simply adds noise to a flat plane?

Classic generators treat the whole world as a single mask, so a single noise field determines every point. Here each semantic region carries its own mask $m_r(x)$ and its own set of noise $N_{r,k}$ and operators $G_{r,j}$, allowing mountains in one region, dunes in another, and smooth blending where they meet.

Compute region $r_1$ contribution at cell (1,2): $0.8\bigl[10 + 0.5\cdot1 + 0.3\cdot2\bigr]=0.8\cdot12.1=9.68$.

Compute region $r_2$ contribution at the same cell: $0.2\bigl[2 + 0.5\cdot1 + 0.3\cdot0\bigr]=0.2\cdot2.5=0.5$.

Sum contributions → final height $H=9.68+0.5=10.18$, a smooth blend of mountain base and plain.

Repeat for all cells; the central column yields heights around $6$, creating a gentle slope between the two regions.

The soft‑mask blend lets each region retain its characteristic shape while guaranteeing a continuous surface, something a single‑mask height‑map cannot achieve without manual stitching.

**Figure 2.** Overview of global terrain generation and refinement. (a) Initial Height-Field Generation constructs composite terrain geometry from the semantic layout map and region-specific terrain parameters and assigns materials to the corresponding regions. (b) Global Terrain Asset Scattering instantiates terrain-associated assets according to regional semantics and local surface conditions. (c) Terrain Refinement renders, inspects, and locally edits the terrain to correct geometric transitions, material scales, asset distributions, and rendering artifacts.

Regional Object Generation

Generate detailed objects region‑by‑region to avoid a costly single‑pass.

Processing an entire 3‑D scene in one pass overwhelms memory and makes fine‑grained control impossible, so the system isolates only those regions that still need objects.

The agent treats each selected region like a miniature stage: first it sketches a 2‑D layout on top of the terrain, then it pulls each sketch element into a 3‑D mesh, and finally it drops the mesh back onto the terrain with the right size and pose.

Regional Planning: examine $P$ and $T$, select $R^{+}$, emit $P_{\text{regional}}$.

Region Composition: render terrain view $I_{\text{terrain}}^{r}$, generate layout image $I_{\text{comp}}^{r}$.

Object Generation: segment $I_{\text{comp}}^{r}$, reconstruct each segment into $(M_{i},U_{i},T_{i})$, calibrate scale.

Object Placement: compute scale $s_{i}$ and transform $T_{i}^{\text{place}}$, drop objects onto terrain.

Scene Refinement: optional post‑processing of the assembled scene.

Compute scale factor $s_{i}= (6.0/2.0)\times(50/35)=3\times1.428\approx4.28$.

Assume the relative rotation $R_{i}$ is identity (camera axes aligned).

Set $P_{o}=(0,0,0)$ (object origin) and $P_{t}=(10,\,5,\,6)$ m (terrain anchor).

Form the placement matrix $T_{i}^{\text{place}}=\begin{bmatrix}4.28\,I & 4.28\,(10,5,6)\\0&1\end{bmatrix}$, i.e. scale by 4.28 and translate to $(42.8,\,21.4,\,25.7)$ m.

The scale factor aligns the object's apparent size across two different cameras; without it the statue would appear too tiny or too large on the hill.

Why not place objects directly on the terrain using the original scene spec instead of generating $I_{\text{comp}}^{r}$?

Direct placement lacks a concrete spatial prior; the composition image encodes both the terrain’s geometry and the user‑specified layout, ensuring that generated meshes respect occlusions, density constraints, and style cues before any 3‑D reconstruction occurs.

Scene Refinement and Results

We evaluate WorldClaw’s scene‑refinement step and compare it to prior 3‑D generators.

Generating a full 3D world in a single pass is intractable; WorldClaw therefore decomposes generation into staged modules and finishes with a dedicated scene‑refinement step that enforces global coherence.

A post‑generation agent iteratively fixes object poses, mesh quality, and terrain contacts so that the final world is both visually plausible and semantically consistent.

How does Scene Refinement differ from a simple mesh‑cleanup post‑process?

Unlike a generic cleanup that only smooths geometry, Scene Refinement jointly evaluates semantic consistency (pose, scale, category), texture quality, and physical plausibility (object‑terrain contact). It uses a task queue, re‑renders after each edit, and stops only when all diagnostics succeed, guaranteeing both visual and functional coherence.

**Figure 3. Scene refinement.** (a) *Object Refinement* processes objects from a report queue, evaluates pose, mesh quality, and scale against semantic and regional context, applies targeted edits, and verifies the result through re-rendering. (b) *Terrain Refinement* examines support-surface quality and object-terrain collisions, applies local co-deformation to defects such as floating, and updates the report after re-rendering.

Across diverse prompts—tropical island, river canyon, desert battlefield, and snow‑covered valley—WorldClaw consistently produces coherent global terrain, region‑specific objects with correct scales and orientations, and stable object‑terrain contacts, as evident in both orbit and walk‑view renderings.

Related Work

Survey of existing 3D scene generation approaches and their key design axes.

We organize prior work on static 3D scene synthesis into four distinct paradigms: procedural generation, visual‑prior‑based generation, native 3D generation, and LLM‑driven generation.

Procedural generation synthesizes content through hand‑crafted rules, stochastic grammars, or executable programs. Classical pipelines excel at terrains, vegetation, and cityscapes (e.g., Infinigen, Infinigen Indoors, Infinigen‑Articulated) and are prized for controllability and repeatability.

LLM‑guided procedural generation replaces manual parameter tuning with language models that plan and orchestrate modular generators. Notable systems include CityX, SceneX, LatticeWorld, and LandCraft, which translate multimodal instructions into symbolic layouts executed in game engines.

Visual‑prior‑based methods lift pretrained image or video generators into 3D via depth estimation, multi‑view fusion, or explicit representation optimization. They split into progressive scene expansion, global scene initialization, structured generation, and video‑guided world synthesis.

Progressive expansion builds a scene incrementally along camera trajectories. Representative works include Infinite Nature, InfiniteNature‑Zero, Persistent Nature, DiffDreamer, SceneScape, WonderJourney, and ScenePainter, all focusing on long‑range consistency.

Several systems integrate each new observation into a persistent 3D representation: Text2Room, LucidDreamer, RealmDreamer, WonderTurbo, WonderWorld, Infinicity, CityDreamer, and GaussianCity.

Global scene initialization first creates a coherent visual proxy—often a panoramic image—and then lifts it into 3D. Key examples are PERF, FastScene, DreamScene360, HoloDreamer, SceneDreamer360, LayerPano3D, Scene4U, and HunyuanWorld 1.0.

**Figure 8.** Qualitative comparison with the representative text-driven 3D scene generation methods. All methods are conditioned on prompts that share the same medieval-village theme and comparable scene requirements, with the wording adapted when necessary to each method's input format.

Collectively, these works illustrate a spectrum from monolithic, end‑to‑end generators to staged pipelines that decompose world creation, motivating the hierarchical Agentic Framework introduced in our method.

Structured Scene Generation

Survey of recent modular, video‑guided, native, and LLM‑driven 3D scene generation approaches.

Structured scene generation methods now treat scenes as collections of semantically meaningful instances rather than monolithic meshes. Approaches such as SAM3D, MIDI, and SceneTransporter reconstruct individual objects and model their spatial relationships; modular pipelines like SceneMaker, CAST, Sketch2Scene, and WorldGen further decompose generation into object‑level stages, enabling instance‑level editing.

Video‑guided world generation leverages temporally coherent video diffusion to provide dense multi‑view evidence. Methods include Scene Splatter, Wonderland, WorldExplorer, FlexWorld, HY‑World 2.0, and WorldMirror 2.0, each integrating video latents with 3D reconstruction pipelines.

Native 3D scene generation can be grouped into holistic, progressive, and synchronized strategies. Holistic methods such as DiffInDScene, XCube, Frankenstein, L3DG, CADD, CymbaDiff, and NuiWorld generate entire scenes within a fixed spatial support; progressive approaches like BlockFusion, SceneFactor, SemCity, NuiScene, WorldGrow, and GaussianGPT expand scenes conditionally from a seed region; synchronized techniques including LT3SD, TRELLISWorld, Extend3D, and WorldFlow3D jointly optimize overlapping regions for global consistency.

LLM‑driven 3D scene generation is categorized as language‑guided planning, hierarchical interactive construction, and agentic generation. Early planners such as Holodeck, I‑Design, 3D‑GPT, and SceneCraft translate text into scene layouts; hierarchical systems like HSM, OptiScene, CityCraft, MajutsuCity, Yo’City, MANSION, SceneThesis, VLM agents, and HOLODECK 2.0 add multi‑scale planning and visual grounding; agentic frameworks such as SceneWeaver, SAGE, SceneSmith, VIGA, and SceneCode close the loop with tool invocation, execution, and feedback.

Limitations and Conclusion

WorldClaw’s promise is tempered by model reliance, code stability, and efficiency costs.

Generating large‑scale 3D worlds in a single pass is intractable; WorldClaw instead breaks the task into a coarse‑to‑fine agentic pipeline that first plans intent, then builds global terrain, and finally places regional objects.

First, the system is highly dependent on the capabilities of the underlying models—large language models for planning, image generators for semantic layouts, and 3D generators for asset reconstruction.

The pipeline’s overall quality collapses when any component model underperforms; open‑source alternatives often fail to produce executable terrain or coherent layout maps, forcing reliance on high‑end models such as Claude Opus 4.8.

Second, stability risks arise from LLM‑generated code that drives terrain construction, material synthesis, and object placement; small errors in scale, parameters, or node connectivity manifest as inconsistent landforms or misaligned objects.

Third, the multi‑stage pipeline incurs substantial inference latency and compute cost, especially as the number of objects and refinement iterations grows, making it inefficient for simpler scenes that could be generated holistically.

Instead of outputting static meshes, the system can emit code—such as Blender material‑node graphs or shader scripts—that programmatically defines geometry and appearance, yielding directly editable assets.

Integrating WorldClaw with mature production environments—such as Unreal Engine’s procedural‑content pipeline—could address scalability and real‑time interaction gaps that Blender alone cannot satisfy.

Looking ahead, we anticipate generative systems that output fully executable worlds: geometry, materials, animation rigs, and interaction scripts all defined through code, enabling seamless editing and reuse.

In the long term, creators would no longer need to assemble asset libraries or hand‑craft shader graphs; agents would handle those technical steps, leaving creators to focus solely on the narrative and design of their imagined worlds.

Additional Results

Additional generated scenes and contributor credits.

**Figure 9** Medieval-style village across diverse terrain. The scene combines snow-capped mountains, forested plains, waterways, and desert areas, with village buildings, windmills, vegetation, and animals distributed across the different regions. The figure shows the global layout, regional close-ups, local walk views, and their corresponding instance, depth, and normal renderings.

**Figure 10** Snow-covered riverside village. The village extends along both sides of a frozen river within a mountainous landscape. The figure shows the global layout, regional close-ups, and local walk views together with instance, depth, and normal renderings.

**Figure 11** Desert adventure camp surrounded by dragons. The scene combines layered desert terrain settlements, watchtowers, and dragons distributed around the camp. Global, regional, and local walk views are shown with the corresponding instance, depth, and normal renderings.

**Figure 12** Island with Japanese-style towns. Multiple settlements are distributed across an island containing coastlines, vegetation, hills, and water. The figure presents the global and regional organization as well as local walk views and their instance, depth, and normal renderings.

**Figure 13. Volcanic demon lair.** Lava flows, volcanic landforms, and demonic structures form a unified environment around the central lair. Global and regional views are complemented by local walk views and the corresponding instance, depth, and normal renderings.

**Figure 14** Gemstone mining site. The scene organizes excavation regions, exposed gemstone deposits, construction structures, and mining equipment within a terraced mine. Global, regional, and local walk views are shown with instance, depth, and normal renderings.

**Figure 15.** Mountain valley with Hobbit-style villages. Scattered settlements are embedded within a valley surrounded by steep hills, vegetation, and water. The figure shows the global composition, regional close-ups, local walk views, and their instance, depth, and normal renderings.

Project Leaders — Chunchao Guo, Yang Li. Local Scene Generation — Jinpeng Li, Yang Li, Zilong Huang. Global Terrain Generation — Zilong Huang, Yang Li, Jinpeng Li.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers