UniMate: One Unified Model to Animate Diverse Skeletons

LINZHAN MOU, JIAHUI LEI, ZHIYANG DOU, CHENYUE CAI, CHAOYUE SONG, ADAM FINKELSTEIN, SZYMON RUSINKIEWICZ

UniMate is a unified foundation model that synthesizes articulated motion for arbitrary skeletal topologies from text prompts.

How can we generate motion for arbitrary 3D skeletons using a single model, rather than training separate models for each specific topology?

Existing motion generators are either constrained to specific skeleton templates like humans or require costly per-skeleton fine-tuning, making them unable to animate diverse 3D assets at scale. UniMate introduces a Topology-Aware Diffusion Transformer (TADiT) that treats skeletal structure as an explicit input, using graph-aware attention and spectral position embeddings to generalize across heterogeneous kinematic trees. On a diverse benchmark of 13,006 sequences, UniMate outperforms state-of-the-art baselines in motion quality and generalization while enabling zero-shot tasks like cross-topology motion transfer.

Paper Primer

The core challenge is that real-world skeletons—ranging from humans and insects to articulated rigid objects—have distinct joint counts and kinematic trees. UniMate solves this by decoupling motion synthesis from fixed templates, using a shared token stream that jointly processes rest-pose structure and dynamic motion.

UniMate is a flow-matching diffusion model: it injects structural context via a graph-aware attention bias, uses spectral rotary position embeddings (Spec-RoPE) to encode joint positions based on graph Laplacian eigenvalues, and pools global topology into every transformer block via AdaLN-Zero.

UniMate achieves superior motion fidelity and generalization compared to topology-agnostic baselines.

In tests on unseen skeletal topologies, UniMate reduced the Fréchet Inception Distance (FID) from 2.711 to 0.757. A 72% reduction in FID, indicating significantly higher motion quality.

UniMate provides faster, more expressive mesh animation than existing feed-forward or optimization-based methods.

In user studies, UniMate scored 4.615/5.0 on average across text-motion agreement and plausibility, compared to 2.808 for the next-best baseline. UniMate generates animations in ~1.2 seconds, compared to over 15 seconds for feed-forward baselines and over 1.6 hours for optimization-based methods.

Why is a "topology-aware" approach necessary instead of just treating skeletons as point clouds?

Treating skeletons as raw geometry sacrifices the kinematic constraints (like joint hierarchies) that define articulated motion, leading to jittery or physically implausible results that break standard rig-driven animation pipelines.

What is the role of the UniML3D dataset in this architecture?

UniML3D provides the necessary scale and diversity—13,006 sequences across heterogeneous species—to allow the model to learn shared motion priors that transfer across different skeletal structures, which is impossible with human-only or species-specific datasets.

UniMate shifts motion synthesis from per-asset optimization to a single, feed-forward foundation model, enabling scalable animation for any rigged 3D asset.

Introduction

We expose why animation is the last bottleneck and how UniMate unifies motion synthesis.

Current animation pipelines tie motion generation to specific skeleton topologies, preventing a single model from handling diverse characters.

Recent advances in automatic rigging now deliver animation‑ready 3D assets at scale, yet generating the motion to drive them remains a bottleneck.

Existing learned animators are topology‑constrained: they rely on category‑specific templates or require per‑skeleton fine‑tuning and reference motions at inference.

We introduce UniMate, a unified foundation model that synthesizes articulated motion for arbitrary skeletons from a rigged 3D asset and a text prompt, with no test‑time optimization or per‑skeleton retraining.

UniMate’s core is a topology‑aware diffusion transformer that weaves skeletal structure into attention through three mechanisms: (1) a graph‑aware attention bias derived from pairwise joint relations and geodesic distances; (2) a spectral rotary position embedding that extends RoPE to arbitrary kinematic trees via the graph Laplacian; and (3) a global topological conditioner obtained by attention‑pooling the rest‑pose skeleton.

To train this model we curated UniML3D, a dataset of 13,006 motion sequences covering bipedal, quadrupedal, avian, marine, insectoid, serpentine, and articulated rigid objects, each paired with a canonicalized rig and a descriptive text prompt.

Trained on UniML3D, UniMate outperforms state‑of‑the‑art baselines in quality, generalization, and efficiency, and supports zero‑shot cross‑topology transfer, in‑betweening, expansion, and text‑guided editing.

**Fig. 1.** Given a rigged 3D asset and a text prompt, UniMate generates animations for characters with diverse skeletal topologies within a single unified model.

A single unified animation model can generate high‑quality motion for any skeletal topology.

The Topology Bottleneck

Motivating the need for a unified animator that handles arbitrary skeletal topologies.

State‑of‑the‑art learned animators are topology‑constrained, relying on fixed category‑specific skeleton templates such as SMPL or SMAL, while topology‑agnostic models still need per‑skeleton fine‑tuning or reference motions, and mesh‑based methods either depend on costly per‑asset distillation or regress kinematically unconstrained vertex‑wise deformations.

These limitations motivate a unified foundation model that can synthesize motion for arbitrary skeletal topologies directly from high‑level descriptions.

Building such a model faces two core challenges: (1) modeling heterogeneous skeletons—bipedal humans, multi‑legged insects, winged animals, and articulated rigid objects all exhibit distinct kinematic trees, joint counts, and motion patterns—so the topology must be an explicit input rather than baked into the architecture; and (2) data scarcity—text‑paired motion corpora spanning diverse topologies are scarce, and existing rigged 4D assets are noisy, inconsistent, and lack unified preprocessing.

We address these by introducing UniMate, which leverages the Topology‑Aware Diffusion Transformer (TADiT) equipped with a graph‑aware attention bias, the Spec‑RoPE spectral rotary position embedding, and a global topological conditioner via AdaLN‑Zero, and by curating the UniML3D dataset of 13,006 sequences covering bipedal, quadrupedal, avian, marine, insectoid, serpentine, and rigid‑object rigs paired with 3,584 diverse text prompts.

Our contributions are: (1) UniMate, a unified foundation model for arbitrary skeletons; (2) TADiT with its three key design choices; (3) the UniML3D dataset; and (4) extensive experiments demonstrating superior quality, generalization, and zero‑shot capabilities such as cross‑topology motion transfer and text‑guided editing.

Related Work

We situate UniMate among prior 3D animation and cross‑topology motion methods.

Recent 3D animation research falls into two camps: generative pipelines that optimize per‑asset at test time, and feed‑forward networks that regress deformations directly on raw geometry. Both approaches either incur heavy optimization costs or abandon the compact skeletal representations that drive existing animation pipelines.

AnyTop is a diffusion model that learns a single latent space capable of generating motion for many heterogeneous skeletons.

Neural generator that learns motion on arbitrary topologies but trains a distinct model per skeleton, limiting cross‑skeleton reuse.

SinMDM adopts a diffusion framework for motion synthesis on arbitrary topologies, yet each skeleton requires its own trained model.

Example‑based method that transfers motion across topologies via sparse correspondences, avoiding neural training.

Patch‑nearest‑neighbor synthesis that adapts the Drop‑the‑GAN idea to motion, stitching fragments from a motion library.

Generative adversarial approach that treats motion generation as a patch‑matching problem, analogous to image synthesis.

Parametric human body model that provides a fixed skeletal template for many motion‑generation pipelines.

Parametric animal model extending SMPL to quadrupeds, still based on a fixed topology per species.

Dataset of animal skeletons used to train AnyTop; limited to a small set of species.

**Fig. 2. Animations generated by UniMate.** Our method generalizes across heterogeneous skeletons and diverse motion prompts.

In contrast to these prior approaches, UniMate learns a single diffusion model that operates directly on arbitrary skeletons, accepts free‑form text, and produces end‑to‑end rigged animations without requiring exemplar motions.

The TADiT Architecture

We describe a unified skeleton tokenization and a topology‑aware diffusion transformer that drives animation from text.

Given a rigged 3D asset and a text prompt, we need a representation that lets a single transformer handle arbitrary skeletons.

We turn any kinematic tree into a flat list of joints by walking breadth‑first from the root, so the transformer sees a consistent token order across different characters.

Start at root $j_1$ (index 1).

Visit its child $j_2$ (index 2).

Visit $j_2$'s children $j_3$ and $j_4$, assigning indices 3 and 4 respectively.

Resulting BFS order: $[j_1, j_2, j_3, j_4]$ with parent list $[1,1,2,2]$.

The ordering keeps the hierarchical structure while giving every skeleton the same token positions, enabling weight sharing across diverse topologies.

Why do we set the root’s parent index to itself ($pa_1 = 1$) instead of leaving it undefined?

Using the self‑parent convention eliminates a special‑case entry in the parent‑index array, so the transformer can treat every joint uniformly as “has a parent”. This simplifies the positional encoding and avoids a separate mask for the root.

We divide all joint positions by the longest geodesic distance in the skeleton, stripping absolute size while preserving relative limb proportions.

Compute $d_{\text{topo}} = \max\{2,3,5\}=5$.

Scale each position: $\mathbf{r}_1' = (0,0,0)$, $\mathbf{r}_2' = (0,0.4,0)$, $\mathbf{r}_3' = (0,1,0)$.

The normalized skeleton retains the same relative limb ratios (2:3) but fits within a unit‑diameter space.

After normalization, a tall character and a short character occupy the same canonical space, allowing the transformer to learn geometry‑independent motion patterns.

What would happen if we omitted the topology‑diameter normalization?

Without scaling, large skeletons would produce joint coordinates with magnitudes orders of magnitude larger than small ones. The transformer’s learned weights would then be biased toward the scale of the most common skeleton, harming generalization to unseen topologies.

**Fig. 3. UniMate pipeline.** The proposed TADiT operates on a joint token space that combines per-frame motion features with rest-pose skeletal descriptors, while injecting skeletal graph-structured bias into both positional encoding and attention computation. Conditioned on input text prompts, our unified model produces realistic, coherent animations while demonstrating strong cross-topology generalization and efficient runtime performance.

Spectral Joint Descriptors

We encode skeleton topology with spectral descriptors and inject it via graph‑aware bias and Spec‑RoPE.

The core difficulty is that a single model must reason over arbitrary kinematic trees, each with its own joint ordering and connectivity.

We embed each joint in a continuous space derived from the graph Laplacian, so the model sees the joint’s position on the skeleton’s intrinsic geometry rather than a raw coordinate.

How does this differ from simply feeding raw joint coordinates to the transformer?

Raw coordinates are tied to a particular world frame and depend on the joint ordering; the spectral descriptors are invariant to rigid transforms and encode the graph’s intrinsic connectivity, allowing the model to transfer to unseen topologies.

Eigenvector $u_{1} = (0.5, 0.5, -0.5, -0.5)$, eigenvector $u_{2} = (0.5, -0.5, -0.5, 0.5)$.

For $m=2$, the descriptor for joint 2 is $FS_{2} = [u_{12}, u_{22}] = [0.5, -0.5]$.

Joint 3 receives $FS_{3} = [-0.5, -0.5]$, distinguishing its position on the chain.

The descriptors separate joints by their spectral role on the graph, making it easy for attention to reason about “ends” versus “middle” joints regardless of absolute coordinates.

We add a learned scalar bias to each joint‑attention logit that encodes the pairwise graph‑distance and relation type, so the attention scores directly reflect the skeleton’s structure.

Why not simply concatenate the distance matrix to the token embeddings instead of using a bias?

Appending the matrix inflates token dimensionality and forces the model to learn a linear combination; a bias directly modifies the attention logits, preserving the original embedding size and guaranteeing that the structural signal influences the attention distribution regardless of embedding scale.

We replace the conventional 1‑D rotary angles (based on joint index) with angles derived from the Laplacian eigenvalues, giving each joint a topology‑aware positional code.

How does Spec‑RoPE differ from the usual 1‑D RoPE applied to joint indices?

Standard RoPE treats joint indices as a linear sequence, which is undefined for trees; Spec‑RoPE derives angles from the Laplacian eigenvalues, providing a rotation that reflects the true graph connectivity rather than an arbitrary ordering.

**Fig. 4. Cross-topology animation.** UniMate generates prompt-aligned motions for diverse characters and articulated objects within a single unified model.

**Fig. 5. One skeleton, diverse prompts.** Given a single skeleton, UniMate synthesizes distinct, prompt-faithful motions for different input text prompts.

**Fig. 6. One skeleton, one prompt, diverse motions.** Given the same skeleton and text prompt, UniMate generates diverse plausible motion samples.

**Fig. 7. Spectral visualization.** From left to right, Laplacian eigenvectors increase in frequency. Low-frequency modes capture global kinematic structure, while higher-frequency modes encode finer local relationships.

Together, spectral joint descriptors, graph‑aware bias, and Spec‑RoPE give the transformer a unified, topology‑sensitive view of any skeleton, enabling a single model to animate diverse characters.

Theoretical Analysis

Provides the appendix details on dataset construction, implementation, and theoretical analysis.

Section A.1 lists the three source datasets—Truebones, Mixamo, and Objaverse‑XL—and reports that UniML3D contains 13,006 motion sequences totaling over 2.1 million frames across thousands of skeletons.

Section A.2 describes a joint‑name standardization pipeline that maps heterogeneous raw labels to a fixed anatomical vocabulary using a deterministic LLM prompt, ensuring a one‑to‑one correspondence for every joint.

Section A.3 selects a bilateral joint pair per rig to define the initial facing direction, preferring thigh or shoulder pairs and falling back to head‑tail axes for serpentine rigs, with a rule‑based resolver handling missing pairs.

Section A.4 renders each retained clip from four orthogonal cameras (0°, 90°, 180°, 270°) at a fixed 2 m radius and 33.9° field of view, producing synchronized videos for captioning and visual inspection.

Section A.5 generates motion captions by prompting a multimodal LLM (Qwen3.5‑9B) with the four‑view videos, using subject‑specific prompts and a manual quality‑control pass to filter inaccurate captions.

Section A.6 addresses the long‑tailed distribution of skeleton types with a re‑weighted sampler (weight $w_i = n_i^{-\\alpha}$, $\\alpha=0.5$) and applies four kinematics‑preserving augmentations—joint removal, joint addition, skeleton pooling, and bone‑length perturbation—recomputing topology descriptors after each augmentation.

Section B.1 details the motion diffusion transformer architecture: eight Skeletal‑Temporal Transformer blocks, hidden size $d=512$, eight attention heads with per‑head dimension $d_h=64$, and a SwiGLU feed‑forward network of dimension $4d=2048$, plus graph‑distance embeddings of size $d_e=128$ and Spec‑RoPE using the leading $m=8$ Laplacian eigenvectors.

Section B.2 explains conditioning: the Topology‑Aware Diffusion Transformer (TADiT) receives timestep embeddings, frozen FLAN‑T5‑Base text embeddings, and topology embeddings, which are fused and injected via AdaLN‑Zero into every sublayer.

Section C provides a self‑contained theoretical analysis of Spec‑RoPE, covering its relative‑coordinate structure, permutation equivariance, connection to standard RoPE, and an effective‑resistance interpretation.

Results and Ablations

UniMate unifies animation across diverse skeletons, removing the need for category‑specific templates.

UniMate attains the highest mean user rating, surpassing the nearest competitor by 0.3 points.

Mean Likert score 4.6 for UniMate versus 4.3 for V2M4 and 4.2 for AnimateAnyMesh (Fig. 21).

**Fig. 11.** **Comparison with AnyTop on unseen Truebones skeletons.** Ours (left) vs. AnyTop (right): our deer falls and lies on its side and our raptor walks forward as prompted, whereas AnyTop’s deer never falls and its raptor remains nearly stationary.

**Fig. 21.** User study results on text-conditioned mesh animation. Mean Likert ratings (1 = very poor, 5 = excellent, with error bars) per criterion and averaged overall; UniMate receives the highest rating on all four criteria—text-to-motion agreement, motion plausibility, motion expressiveness, and shape preservation—as well as overall.

**Fig. 24.** Qualitative ablation of the topology-aware components. The same rig and prompt rendered from the already-trained ablation models of the main paper, with each variant's FID and diversity inset. The full model executes a clear forward attack; without the graph-aware attention bias the wing-body coordination degrades; without Spec-RoPE the motion loses the prompted action and collapses toward in-place wing flailing; without the global topological conditioner the motion stays dynamic but becomes unstable, with jittery, poorly grounded poses.

**Fig. 25.** Before/after qualitative examples for the curation stages. The same held-out rig and prompt rendered from models trained with the full curation pipeline (top) and with one stage removed. Without implausibility filtering, the implausible clips that survive in the corpus surface at generation time: the reindeer tumbles through airborne, twisted poses instead of attacking. Without dataset-level statistics normalization, the attack is roughly executed but the motion drifts and hovers with degraded ground contact. The full pipeline performs the prompted antler attack with stable, grounded motion.

Implementation Details

We present the experimental validation of UniMate, covering quantitative benchmarks, qualitative demos, and ablations.

This section reports the empirical evaluation of UniMate, highlighting its quantitative superiority, diverse qualitative outputs, and the impact of each topology‑aware component.

**Fig. 8. Long-horizon generation.** A variant trained at 180 frames generates long sequences that stay temporally coherent and drift-free.

**Fig. 9. Samples from the UniML3D dataset.** Our dataset spans diverse skeletons across bipedal, quadrupedal, avian, marine, insectoid, serpentine, and articulated rigid objects, with detailed skeleton annotations and coherent text prompts paired with motion sequences.

**Fig. 10. UniML3D data-processing pipeline.** Source assets pass through skeleton-based filtering, semantic annotation, and motion canonicalization; balanced sampling and augmentation happen on the fly during training. (1) *Single-tree pruning.* Multiple or disconnected kinematic trees are pruned, keeping only the primary skeleton: the tree with the largest cumulative skinning weight. (2) *Root-joint realignment.* Spurious or misaligned root joints are corrected by propagating their global transformation onto the semantic root via forward kinematics. (3) *Phantom-joint removal.* Non-functional joints, e.g., IK controllers and helper bones with zero skinning weight, are recursively pruned to streamline the topology. (4) *Static-clip removal.* Clips with negligible activity, quantified by bone-length-normalized global displacement, are discarded. (5) *Implausibility filtering.* Clips with out-of-distribution root velocities or per-joint angular jitter above an anatomical threshold are discarded.

**Fig. 12. Text-mediated motion transfer.** The source behavior is abstracted into a text prompt (shown atop each group); the same prompt then animates four target rigs of differing topology, conditioned directly on each target skeleton.

**Fig. 13. Motion in-betweening.** Given the start and end poses and a text prompt, UniMate synthesizes smooth and plausible intermediate motions; the opaque poses mark the given boundary constraints, while the translucent poses are the synthesized in-betweens.

**Fig. 16. Foot sliding.** In a generated walk, the hind-paw contacts drift along the ground (red traces) instead of staying planted.

**Fig. 17. Rare-topology failure.** Prompted to fall onto the desk, the lamp only dips and recovers its head; the base never tips over.

The ablation study (Table 4) confirms that each topology‑aware component—graph‑aware attention bias, Spec‑RoPE, and the global topological conditioner—contributes to lower FID and higher diversity, with the full model achieving the best trade‑off.

Limitations include foot‑sliding and occasional drift in contact‑rich motions, as well as degraded performance on rare skeletal topologies and out‑of‑distribution prompts.

The UniML3D Dataset

This section details the UniML3D dataset composition, preprocessing pipeline, and augmentation strategy.

UniML3D aggregates three complementary motion sources: Truebones contributes 1,094 animal sequences across 74 skeletons, Mixamo adds 2,425 high‑quality human sequences, and Objaverse‑XL provides 6,965 articulated assets with animation annotations, yielding a total of 13,006 sequences and 2,140,232 frames.

The dataset is heavily long‑tailed: bipedal characters dominate at 84.1 %, while serpentine (0.3 %) and marine (0.8 %) categories are sparsely populated, motivating a square‑root‑balanced sampler.

The joint‑count distribution is bimodal, with a primary peak around 30 joints and a secondary peak near 55, reflecting a wide range of skeletal complexities.

Joint Name Standardization resolves the myriad raw labels (e.g., mixamorig:, QuickRigCharacter_, Spine1, Finger01) to a fixed anatomical vocabulary using a deterministic LLM prompt, ensuring a one‑to‑one mapping for every rig.

Algorithm 1 outlines the three‑stage curation pipeline: skeleton‑level filtering and semantic annotation, clip‑level filtering, canonicalization, and dataset‑level feature normalization.

Facing‑Direction Joint‑Pair Selection aligns each clip’s initial facing direction with the positive z‑axis by selecting a bilaterally symmetric joint pair per rig; when no mirrored pair exists, head‑and‑tail joints define a longitudinal axis.

Motion Rendering produces a four‑view synchronized video for each clip using four cameras placed at elevation 0° and azimuths 0°, 90°, 180°, 270° on a sphere of radius 2 m, with a 33.9° field of view, after resampling to 30 FPS.

Motion Captioning queries a multimodal LLM (Qwen3.5‑9B) on the four‑view rendering, applying dataset‑specific prompts and a manual quality‑control pass to ensure captions correctly describe the dominant action and root motion.

Balanced Sampling assigns each skeletal type $i$ a weight $w_i = n_i^{-\alpha}$ with $\alpha = 0.5$, and four kinematics‑preserving augmentations (joint removal, joint addition, skeleton pooling, bone‑length perturbation) are applied on‑the‑fly during training.

UniML3D is a large, diverse collection of motion sequences paired with standardized skeletal rigs and textual captions, covering a wide range of creature morphologies.

**Fig. 18. Morphology distribution of UniML3D.** Share of motion sequences per skeletal morphology; bipedal characters dominate, while serpentine and marine rigs are rare.

**Fig. 19. Joint-count distribution across skeletons.** Histogram and kernel density of the number of joints per retained skeleton in UniML3D (median 38, mean 40.4); the bimodal spread shows that the dataset covers rigs of widely varying complexity.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers