MULTIMODAL PERCEPTION · 2026

UniT

Unified Geometry Learning with Group Causal Transformer

  1. 1Intelligent Transportation Thrust, Systems Hub, HKUST(GZ)
  2. 2National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi'an Jiaotong University
  3. 3Applied Science, Amazon.com, Inc.
  4. †Corresponding authors

TL;DR One feed-forward model reconstructs 3D geometry online or offline, from any number of views, with optional camera and depth inputs, at metric scale and over long sequences.

OVERVIEW

One model for geometry perception

METHOD

Group causal transformer

Geometry as next-group prediction: attention is bidirectional inside a group of G views and causal across groups. G = 1 runs online, G = N offline.

A toy example: N = 4 frames, cache queue Q = 2. Rows are query frames, columns key frames. Pick a group size to step through it.

Training data

EXAMPLES

Reconstructions in the wild

Point clouds reconstructed by UniT, from campus scenes at HKUST(GZ) to public driving and synthetic sequences. Each scene loads only when you select it.

Drag to rotate, scroll or pinch to zoom, and switch scenes with the thumbnails. Drag to rotate, pinch to zoom, and tap a thumbnail below to switch scenes. Swipe the strip to see more scenes. Turn on Measure to check the real-world distance between any two points.

Click a thumbnail to load the point cloud.

LIVE DEMO

Run UniT on your own images

The Hugging Face Space runs the released model on your images, no setup required.

  1. 01Upload images or a video, or load one of five packaged examples.
  2. 02Set the group size: how many frames UniT reconstructs together.
  3. 03Reconstruct, then explore the metric-scale point cloud.
Open in a new tab

Runs on Hugging Face Spaces. If the Space is asleep it takes about a minute to start.

KEY INSIGHTS

Three challenges, one formulation

Three obstacles keep geometry models apart. UniT answers each with one idea.

01Challenge · View configurations

Online and offline models are separate families

  1. Problem

    Online models fuse many views poorly. Offline models recompute everything for each new frame.

  2. Insight · Group causal modeling

    Predict the next group of G views. G = 1 is online, G = N is offline, with the same weights.

  3. Result

    0.087average error of one unified model; three specialized models reach 0.092

Motivation No existing paradigm covers all five capabilities Fig. 2

02Challenge · Streaming memory

A streaming model's memory grows with every frame

  1. Problem

    The KV cache keeps every past frame, and old entries cannot be dropped: the model keeps reading them.

  2. Insight · Queue-style KV caching

    Anchor-free relative poses remove the first-frame reference, so the cache becomes a fixed-length queue.

  3. Result

    5.1 kmof driving streamed in one pass; StreamVGGT runs out of memory on 9 of 11 sequences

Motivation An anchored model keeps looking back at the first frames Fig. 6

15.1%StreamVGGT, anchored

8.0%UniT, anchor-free

Attention deviation per query frame over 64 streamed frames.

03Challenge · Metric scale

Metric scale is much harder to learn than relative geometry

  1. Problem

    Absolute scale has a large dynamic range. Most models avoid it or add a separate scale module.

  2. Insight · Scale-adaptive geometry loss

    Scale-invariant constraints plus a partial metric term: scale gets a closed form, with no extra module.

  3. Result

    7 / 7metric-scale settings where UniT has the best average rank

Motivation Relative geometry is learned first; metric scale follows Fig. 8

Normalized point loss: an easy-to-hard curriculum, with no schedule.

Scale you can see Raw point clouds at one scene scale, no alignment
Three scenes reconstructed as point clouds, left to right: ground truth, UniT offline, DepthAnything3, UniT online and CUT3R, all drawn at the same scene scale.

+Plug-in · Auxiliary modalities

Camera parameters and depth plug in, in any combination

  1. Problem

    Platforms often know their intrinsics or poses, or carry a depth sensor.

  2. Design · Modal attention

    Depth and camera parameters become tokens fused with image tokens, for about 3% more parameters.

  3. Result

    5 / 7input combinations where UniT has the best average rank

RESULTS

Evaluated across tasks and settings

Offline, streaming and chunked baselines, at scale-invariant and metric scale.

Benchmarks
11
  • 7-Scenes
  • NRGBD
  • DTU
  • TUM-Dynamic
  • ScanNetV2
  • Sintel
  • KITTI
  • NYUv2
  • ETH3D
  • Bonn
  • KITTI-Odometry
Tasks
7
  • Multi-view reconstruction
  • Camera pose
  • Video depth
  • Monocular depth
  • Long horizon
  • Multi-modal
  • Depth completion
Baselines
9
  • VGGTCVPR 2025
  • π³ICLR 2026
  • MapAnything3DV 2026
  • DepthAnything3ICLR 2026
  • CUT3RCVPR 2025
  • StreamVGGTICLR 2026
  • TTT3RICLR 2026
  • Stream3RICLR 2026
  • VGGT-LongICRA 2026

CITATION

@misc{wang2026unit,
  title={UniT: Unified Geometry Learning with Group Causal Transformer},
  author={Haotian Wang and Yusong Huang and Zhaonian Kuang and Hongliang Lu and Xinhu Zheng and Meng Yang and Gang Hua},
  year={2026},
  eprint={2605.21131},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2605.21131},
}