A research team has developed AutoWorld, a multi-agent traffic simulation framework that learns from unlabeled LiDAR sensor data rather than costly labeled annotations, and has demonstrated state-of-the-art performance on a public benchmark. The paper was submitted to arXiv on March 30, 2026 by Mozhgan Pourkeshavarz, Tianran Liu, and Nicholas Rhinehart.
- AutoWorld learns a world model from unlabeled LiDAR occupancy data, removing the need for expensive semantic annotations or labeled trajectories.
- The framework ranked first on the Waymo Open Sim Agents Challenge (WOSAC) leaderboard by the primary Realism Meta Metric (RMM).
- A cascaded Determinantal Point Process (DPP) framework guides sampling at both the world model and motion generation stages to promote scenario diversity.
- Simulation performance improved consistently as more unlabeled LiDAR data was incorporated, suggesting the approach scales with available sensor data.
What Happened
Mozhgan Pourkeshavarz, Tianran Liu, and Nicholas Rhinehart submitted AutoWorld: Scaling Multi-Agent Traffic Simulation with Self-Supervised World Models to arXiv on March 30, 2026. The paper introduces a traffic simulation framework that trains a world model on unlabeled occupancy representations derived from LiDAR sensor data — the raw sensor output that autonomous vehicle fleets generate continuously but that most existing simulators do not use.
The framework ranked first on the Waymo Open Sim Agents Challenge (WOSAC) leaderboard according to the primary Realism Meta Metric (RMM), a composite score the benchmark uses to evaluate how realistically simulated agents behave in traffic scenarios.
Why It Matters
Most current data-driven traffic simulators depend on supervised learning from labeled data — annotated driving trajectories, semantic segmentation maps, or object-level bounding box labels. Producing this labeled data at scale is expensive and operationally complex, which constrains how much training data developers can practically use.
Autonomous vehicle fleets already collect large volumes of raw sensor data during normal operation, but this unlabeled data has remained largely unused by existing simulation frameworks. AutoWorld addresses this gap by treating unlabeled LiDAR occupancy grids as a training signal. The authors state in the paper that their method “paves the way for scaling traffic simulation realism without additional labeling.”
Technical Details
AutoWorld’s architecture centers on a world model trained on occupancy representations of LiDAR data — grid-based encodings of which cells in a scene are occupied by physical objects, derived without object-level labeling. From world model samples, the system constructs a coarse-to-fine predictive scene context, which is passed as input to a multi-agent motion generation model that produces agent trajectories.
To address sample diversity — a known limitation in generative simulation systems that can produce repetitive or homogeneous outputs — the researchers introduced a cascaded Determinantal Point Process (DPP) framework. DPPs are probabilistic models that favor diversity in selected subsets; the cascade applies DPP-guided sampling at both the world model stage and the motion model stage independently.
A third component is a motion-aware latent supervision objective, a training signal designed to improve how AutoWorld’s representations capture scene dynamics and inter-agent interactions rather than only static scene structure. Ablation experiments confirmed that each of these three components contributes to the final benchmark score, and that RMM performance improved consistently as more unlabeled LiDAR data was incorporated into training.
Who’s Affected
The primary audience is autonomous driving developers who rely on simulation environments to test vehicle behavior at scale before real-world deployment. AutoWorld is particularly relevant for organizations with large AV fleets that already accumulate substantial unlabeled sensor logs — the framework can convert that data directly into simulation training signal without a separate annotation pipeline.
Simulation platform developers and AV safety teams who currently maintain expensive labeling workflows may find the self-supervised approach reduces that operational cost while improving scenario realism and diversity.
What’s Next
The research team has released code and additional visualizations on the project page linked from the arXiv submission. The ablation study results provide a clear breakdown of which components contribute most to performance, offering a roadmap for further development.
The paper does not claim that AutoWorld generalizes beyond LiDAR occupancy representations to other sensor modalities such as cameras or radar. Whether the self-supervised world model approach transfers to those domains, or to driving environments with different traffic densities and infrastructure, is not addressed in this work.
