Principia: Relational Physics Tests for Video Models

Varun Varma Thozhiyoor, Shivam Tripathi, Venkatesh Babu Radhakrishnan, Anand Bhattad

Principia evaluates video generators on physical consistency using relational invariants that bypass camera and scale ambiguity.

Do current video generators and vision-language models actually understand Newtonian physics, or are they just producing visually plausible motion that violates physical laws?

Evaluating physical reasoning in video models is difficult because absolute measurements like velocity or mass are often ambiguous or unavailable in generated content. The authors introduce Principia, a benchmark that tests physical laws by measuring the relational consistency between paired objects in a scene. Because these relationships—such as ratios of fall times or arrival times—are invariant to camera calibration and metric scale, they can be computed directly in image space. Across six state-of-the-art video generators, no model exceeds a consistency score of 0.42, despite all models scoring around 0.8 on standard visual quality benchmarks.

Paper Primer

Principia evaluates eight Newtonian phenomena—including gravity, friction, and rotational inertia—by enforcing specific relational invariants between two objects released under identical conditions. The core move is to measure whether the generated motion satisfies these expected ratios or equalities, effectively treating the video as a controlled physics experiment rather than a visual rendering task.

Visual quality and physical fidelity are decoupled in modern video generators.

All six evaluated models achieve high scores (~0.8) on VBench (visual quality) but low scores (0.14–0.42) on Principia (physical consistency). The gap suggests that current models learn to render realistic textures without learning the underlying physical constraints.

Scaling model size does not guarantee improved physical reasoning.

In both the Cosmos and Wan model families, increasing parameter counts led to performance regressions on specific physical phenomena. Scaling within a fixed architecture does not uniformly improve physical fidelity and may actively degrade performance on previously handled regimes.

Why is this benchmark more robust than existing physics evaluations?

Most benchmarks require absolute metric quantities like mass or camera intrinsics, which are often unknown in generated video. Principia uses relational invariants—such as the ratio of rebound heights—that remain valid regardless of camera angle, frame rate, or scale.

Can vision-language models (VLMs) reliably detect these physical violations?

No; even the best-performing VLM achieves only 67% accuracy in distinguishing physically consistent videos from those with violations, performing near chance level across most phenomena.

Physical fidelity in video generation is not a byproduct of visual realism or scale. Future progress requires training signals or architectural biases specifically designed to enforce relational physical invariants.

The Physics Gap in Video Models

We define Principia, a relational‑invariant benchmark exposing physical inconsistencies missed by visual quality metrics.

Evaluating physical reasoning in video models is difficult because absolute motion measurements depend on frame rate, object scale, and camera calibration—quantities often unavailable in generated video.

Prior benchmarks either ask whether motion looks plausible, compare generated trajectories to a single recorded video, or require recovering metric quantities to test physical laws; each approach either remains subjective or demands unavailable measurements.

Principia measures physical fidelity by checking that paired objects obey expected relational invariants, eliminating any need for absolute measurements.

Consider two blocks of different mass placed on identical inclined planes with the same angle and friction coefficient; physics predicts they reach the bottom simultaneously because acceleration on an inclined plane with friction is mass‑independent.

Visual plausibility alone does not guarantee physical fidelity; relational invariants expose hidden violations that look realistic.

The Principia Benchmark

Describes the Principia benchmark setup, its variations, and evaluation criteria.

Select a paired‑object scene from the 500+ pool, ensuring it belongs to one of the eight Newtonian phenomena.

Fix all scene parameters (camera pose, lighting, background) and hold both objects identical except for the target variable.

Vary the target variable (mass, height, length, or moment of inertia) across a predefined range to isolate the law.

Render or capture the video sequence for each variation.

Compute the relational invariant score using the quantitative physics formula associated with the phenomenon.

Average the scores over all variations and all scenes to obtain the final Principia score for a model.

Generate three videos with $m_2 = 0.5$, $1.0$, and $2.0$ kg while keeping drop height $h = 1\,$m constant.

Measure the bounce height of each ball; the relational invariant is the ratio $r = h_{m_2} / h_{m_1}$.

Compute scores: for $m_2=0.5$ kg, $r=1.02$ (near‑perfect); for $m_2=1.0$ kg, $r=1.00$; for $m_2=2.0$ kg, $r=0.98$ (slight loss of energy).

Average the three ratios to obtain a scene‑level score of $0.667$ (≈ 0.67).

The invariant isolates the effect of mass on bounce height, showing that the physical law (energy conservation) holds despite differing masses.

**Figure 1.** **Principia** tests physical reasoning in video models through paired-object relational tests. Each panel shows a real-world experimental setup encoding a relational constraint between two objects obeying the same physical law (a-g); these constraints hold independent of camera, scale, or frame rate. We test whether modern video generators and vision-language models preserve them.

Constructing Relational Invariants

We detail how videos were captured, filtered, and augmented to form the Principia dataset.

Record ~750 raw videos of paired‑object experiments across multiple physical phenomena.

Apply four constraints during collection: matched geometry, synchronized release, controlled surface material, and minimized external forces.

Segment and track all objects with SAM3, initializing with hand‑annotated points.

Convert trajectories to pixel‑space measurements; relational consistency scores rely only on ratios/equalities.

Manually inspect each take for monotonic descent, clean transitions, or periodic motion, discarding failures such as drift or slip.

Filter the remaining videos, yielding 529 high‑quality scenes, then augment diversity using editing models.

**Figure 2.** (a) A generated video of a single block sliding down an incline, shown at three timesteps and as a stroboscopic composite (top right): motion appears plausible. (b) A generated video of two blocks with different masses released simultaneously on identical inclines, where the blocks arrive at different times, violating the mass-independence invariant of gravitational acceleration. This violation only becomes detectable through relative comparison.

Newtonian Dynamics Coverage

Physical phenomena define relational invariants for Newtonian dynamics.

The section enumerates the Newtonian phenomena used to construct relational invariants that underpin the Principia benchmark.

Newtonian Phenomena are the four families of motion—translational, rotational, collisional, and oscillatory—that obey Newton’s laws and can be captured by simple algebraic relations.

How does “Newtonian Phenomena” differ from a generic statement about physical constraints?

“Newtonian Phenomena” groups four specific dynamical regimes—translational, rotational, collisional, and oscillatory—each with a concrete algebraic invariant. A generic “physical constraint” could refer to any law (e.g., thermodynamics), but here the term pinpoints the exact Newtonian families that the benchmark tests.

Select a pair of objects that share the same physical constants (e.g., gravity $g$, ramp angle $\theta$).

Vary a single variable between the objects (height $h$, mass $m$, length $l$, etc.) while keeping all other factors fixed.

Record the observable tied to the variable (fall time $t$, travel distance $x$, period $T$, etc.).

Form the relational invariant by equating the functional expressions $F_{\phi}(o_{1})$ and $F_{\phi}(o_{2})$ derived from the governing equation.

Verify that the invariant holds across different camera viewpoints, scales, and frame rates.

Compute the dimensionless inertia ratios: $I_{1}/(mR^{2})=0.5$, $I_{2}/(mR^{2})=1.0$.

Apply the pure‑rolling acceleration formula $a = \frac{g\sin\theta}{1 + I/(mR^{2})}$ to each cylinder, yielding $a_{1}= \frac{9.8\sin30^{\circ}}{1+0.5}=6.53\,$m/s² and $a_{2}= \frac{9.8\sin30^{\circ}}{1+1}=3.27\,$m/s².

For a fixed descent distance, the travel times satisfy $t_{1}/t_{2}= \sqrt{a_{2}/a_{1}} = \sqrt{3.27/6.53}=0.75$.

Thus the relational invariant $t_{1}/t_{2}= (1+I_{1}/(mR^{2}))/(1+I_{2}/(mR^{2})) = 0.75$ holds.

The cylinder with larger moment of inertia arrives later, and the invariant captures this ratio independent of the ramp length or camera speed.

The table lists various physical scenarios, their associated constants, variables, and the corresponding relational invariants. The scenarios included are Restitution, Gravity, Friction, Rotational Inertia, Projectile, Momentum (height), Momentum (mass), Pendulum, and Mass-Spring.

The four core dynamics—translational, rotational, collisional, and oscillatory—provide a complete basis for describing everyday motion in the Principia benchmark.

Defining Relational Invariants

How the experiments enforce and measure relational invariants across physical scenarios.

The experimental protocol varies the physical parameters that appear in the relational invariants while keeping all other factors fixed, then records the resulting measurements.

Relations that remain constant when the underlying objects are scaled, rotated, or otherwise transformed – they capture the physics that should hold regardless of camera viewpoint or frame‑rate.

Select a physical scenario (e.g., mass‑spring, pendulum, projectile) and instantiate the required objects.

Fix all global constants (gravity $g$, material density, surface friction coefficient $\mu$) across trials.

Choose a pair of parameter values to vary (e.g., masses $m_1$, $m_2$ or drop heights $h_1$, $h_2$) while keeping the complementary parameters identical.

Run the simulation or real‑world experiment, recording the observable quantities (heights, times, extensions, torques) for each member of the pair.

Compute the ratios defined by the relational invariants and check whether they match the theoretical expectations.

Log any deviation as a measure of physical fidelity for the video model under test.

Compute extensions: $x_1 = \frac{m_1 g}{k} = \frac{2 \times 9.8}{10} = 1.96\,$m, $x_2 = \frac{m_2 g}{k} = \frac{4 \times 9.8}{10} = 3.92\,$m.

Form the ratio $x_1/x_2 = 1.96/3.92 = 0.5$.

Form the mass ratio $m_1/m_2 = 2/4 = 0.5$ and verify $\frac{x_1}{x_2} = \frac{m_1}{m_2}$.

The linear scaling of spring extension with suspended mass directly demonstrates the relational invariant $x_1/x_2 = m_1/m_2$, confirming that the experiment respects the physical law.

Video Generator Performance

We report how video generators and VLMs perform on visual quality and physical fidelity benchmarks.

Visual quality and physical fidelity are nearly orthogonal across video generators.

All six generators score ≈0.8 on VBench while ranging only 0.14–0.42 on Principia (see Figure 7a).

We evaluate six video generators (Omni [21], Veo‑3.1 [38], Wan2.2‑5B/14B [44], Cosmos‑2.5‑2B/14B [41]) and four vision‑language models (Gemini‑3.1‑Pro, Gemini‑3‑Flash [14], Qwen‑32B, Qwen‑4B [37]). Each generator receives a text prompt and the first frame of a real recording, with experimenter and apparatus inpainted out using Nano Banana 2 [20] so the model conditions on physics rather than the setup. We sample multiple random seeds per scenario and average the scores; total inference compute exceeds 2,600 A100‑hours.

It quantifies both how realistic a generated video looks and how faithfully it obeys relational invariants.

How does this evaluation differ from using only a visual quality metric?

Visual quality metrics like VBench ignore whether the motion respects physics; the Principia score adds a relational invariant check, exposing failures that look realistic but violate basic laws.

**Figure 3.** Visual quality decouples from physical fidelity, and different model classes fail on different phenomena. (a) All six video generators score around 0.8 on VBench but below 0.5 on Principia, clustering in the high-visual-quality, low-physics-fidelity region. (b) Per-phenomenon continuous consistency scores $S_{\phi}$ for video generators. (c) Per-phenomenon agreement scores for vision-language models. VLM polygons are smoother and more uniform than generator polygons (compare b vs c).

High visual quality does not guarantee physical fidelity in generated videos.

VLM Physical Reasoning

Vision‑language models struggle to detect physics violations, with the best achieving only 66 % agreement.

Vision‑language models fail to reliably detect relational physics violations, with none exceeding a 0.7 average agreement.

Table 4 shows the highest overall agreement is 0.665, indicating near‑chance performance.

Assess whether a model can spot violations of relational physics in videos rather than generate physically correct motion.

Generator Evaluation Details

Additional details on the Principia evaluation protocol and model configurations.

The evaluation dataset comprises 401 real‑world scenes captured in‑house; after removing the experimenter and apparatus and augmenting for visual diversity, the final set contains 529 scenes.

All quantitative measurements are extracted automatically: SAM3 tracks each object’s centroid (or bottom pixel) throughout the generated video, and phenomenon‑specific events are detected from these trajectories.

For restitution and gravity, impact is detected when the centroid falls below a ground threshold and vertical velocity flips sign; the rebound apex is the next frame where vertical velocity changes from negative to positive, yielding drop height, rebound height, and time of flight.

Friction and rotational inertia use the arrival time of the bottom‑most pixel crossing a predefined threshold at the end of the incline; these arrival times feed the respective invariants.

Projectile motion detects ground impact when vertical velocity becomes positive after descent and the centroid crosses the ground threshold; the horizontal range is the distance between this impact point and the annotated launch point.

Pendulum period is estimated by detecting a half‑period when horizontal velocity reverses after the bob crosses its vertical midpoint; the full period is twice this value.

Spring extension is measured at the first oscillation turning point (vertical velocity flips from positive to negative); object masses are inferred from cube side lengths ($m \propto l^{3}$), and the measured displacement ratio is compared to the expected mass ratio.

Momentum is evaluated by measuring the maximum post‑collision vertical displacement of each target block and comparing the ordering of these displacements across the two interactions.

Prompt templates for each phenomenon are listed (e.g., “two identical balls dropped from different heights” for restitution/gravity, “two blocks sliding down identical inclined planes” for friction, etc.), each video is conditioned on the first frame and its corresponding textual description.

Qualitative Failure Modes

Limits of scaling and robustness for physical fidelity in video models.

The paper’s core claim is that video models often generate visually plausible motion yet violate fundamental physical constraints; the Principia benchmark measures this by checking relational invariants independent of camera, scale, or frame rate.

It quantifies how closely the direction of a generated object's motion matches the expected direction, ignoring speed.

Generating the Principia corpus required massive compute: Wan2.2‑14B needed about 1,100 A100‑hours (46 days) and Cosmos‑2.5‑14B about 1,300 A100‑hours (54 days); inference across all four open‑weight models consumed roughly 2,800 A100‑hours, equivalent to 120 days on a single A100.

Scaling model size does not uniformly improve physical fidelity. Within each family, Cosmos‑14B gains on friction (+0.29) and pendulum (+0.24) but loses on restitution (–0.09); Wan‑14B improves friction (+0.40), inertia (+0.26), and pendulum (+0.17) but degrades momentum (–0.05). At least one phenomenon worsens with scale, indicating that larger models alone cannot solve the physics gap.

The evaluation protocol assumes a static camera, yet models sometimes violate this. Adding synthetic translational camera motion leaves six of seven phenomena unchanged; only the Projectile scenario is affected, as horizontal pan flips its range ordering. This shows that low Principia scores stem from poor physics rather than camera motion.

VLM Scaling and Robustness

We assess VLMs on Principia‑synth, report scaling effects and detailed physics agreement results.

We evaluate four Vision‑Language Models (VLMs)—Qwen‑4B, Qwen‑32B, Gemini‑3‑Flash, and Gemini‑3.1‑Pro—on the Principia‑synth benchmark, measuring their ability to detect physical violations in videos.

Section B.1 introduces synthetic “anti‑physics” videos that deliberately break relational invariants; representative examples are shown in Figure 12.

**Figure 8.** Representative anti-physics examples from Principia-Synth. Each scene intentionally violates the relational invariant of the corresponding physical phenomenon. From left to right, top to bottom: restitution (inconsistent restitution), friction (mass-dependent sliding despite identical surface properties), projectile motion (incorrect range ordering), gravity (incorrect fall-time relationship), rotational inertia (hollow cylinder reaches the bottom before the solid cylinder), momentum (a ball with greater initial momentum transfers less momentum to the corresponding block), pendulum (the longer pendulum oscillates with a shorter period), and spring (the lighter mass exhibits a larger equilibrium displacement than the heavier mass).

Section B.2 describes the evaluation protocol: for every scene the model receives a phenomenon‑specific prompt and must return a PASS/FAIL verdict with a brief explanation in strict JSON format.

Section B.3 reports the scaling study; Figure 13 reveals that increasing model size does not improve physical‑reasoning performance.

**Figure 9: VLM scaling effects.** Per-phenomenon agreement changes from Qwen-4B to Qwen-32B (left, 8× scale) and Gemini-3-Flash to Gemini-3.1-Pro (right). Scaling does not lead to an increase in performance.

Table 4 summarizes per‑phenomenon agreement scores for the four models; Gemini‑3‑Flash dominates most categories, while Gemini‑3.1‑Pro leads only on Inertia.

Table 6 evaluates robustness to simulated camera motion; most phenomena remain stable, but Projectile suffers a sharp drop under left‑pan motion.

**Figure 5.** **Additional Qualitative results on Principia.** Stroboscopic composites of generated videos across all seven phenomena (columns) and the five evaluated video generators alongside ground truth (rows). For each phenomenon, we show two sets with different physical configurations and visual contexts. **Green markings** indicate motion that satisfies the relational invariant; **red markings** indicate physics violations. Across both sets, we include samples where the real recorded videos are edited to vary backgrounds, lighting, and scene appearance (e.g., indoor vs. outdoor environments, different surface textures, and illumination conditions), allowing the dataset to be scaled and diversified without requiring new physical recordings. For these samples, we additionally provide the original ground-truth first frame prior to augmentation.

**Figure 6.** Dataset statistics across physical phenomena. The figure summarizes the number of samples available for each physical phenomenon, separated into real-world and augmented scenarios.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers