Benchmarking Action-Conditioned World Models against Real-World Executions

Website · Paper · Code · Datasets · Leaderboard

WorldOracle evaluates whether action-conditioned world models predict what happens in the real world. Given the same initial observation and low-level action trajectory, it compares predicted rollouts with paired real-world executions, including physical outcomes and task-state transitions.

• Across embodiments: AgiBot G2, YAM, and human-hand interactions, including both successful and failed executions.
• Across action distributions and time horizons: Human-operated and policy-rollout trajectories, with long, multi-segment episodes and synchronized multi-view observations.
• Across four complementary dimensions: Physical State Correctness, Interaction Spatial Alignment, Key Interaction Area Fidelity, and Multi-View Action Coherence.

Explore the datasets, evaluate your model with the evaluation code, and visit the leaderboard for results and submission instructions.

Questions, feedback, and contributions are welcome through GitHub Issues.