World models are increasingly used as policy-in-the-loop imagination environments, where reliable rollouts require fine-grained controllability with respect to low-level robot actions. A key obstacle to scaling such models in robotics is that actions are not a universal language in pixel space: changes in visual environment, camera view, robot placement, or embodiment alter how the same numerical action manifests visually, leading to conflicting supervision under mixed training and brittle generalization at deployment. We introduce SyncWorld, an action-conditioned world model that serves as a zero-shot simulator across unseen environments without any additional training. SyncWorld leverages a visual calibration episode—paired frames and actions that showcase all the controllable degrees of freedom—to specify the setup-specific Action–Visual Mapping in context. Training with visual calibration contexts teaches the model to interpret actions through visual evidence and to leverage interaction history when explicit calibration is unavailable. Experiments show that SyncWorld can accurately simulate action outcomes in previously unseen settings, and that its capability of simulating rollouts enables test-time policy improvement without training.
SyncWorld is an action-conditioned world model that uses in-context visual evidence to simulate how numerical robot controls change a scene. A single pretrained model adapts to unseen camera views, environments, and robot embodiments, predicting the visual outcome of low-level actions without fine-tuning or any downstream training. Below, each prediction is paired with its ground-truth outcome.
Self-Collected - xArm (Example 1)
Self-Collected - xArm (Example 2)
ARX5 Manipulation Dataset - ARX5
RH20T - KUKA
Maniskill - Franka Panda
Robomind - UR5e
The same numerical action can produce different pixel-space motion under a new camera, scene, robot placement, or embodiment. Visual calibration resolves this ambiguity with a short paired video–action episode in which the robot moves along each controllable DoF, directly revealing the setup-specific Action–Visual Mapping.
During pretraining, the ordered calibration segments are prepended as context alongside interaction history and future actions. This teaches the Diffusion Transformer to infer action meaning from visual evidence and generate controllable future frames in a new setup without parameter updates.
Representative calibration episodes
Real-world evaluation
ManiSkill evaluation
RLBench pretraining
At test time, a VLA policy proposes several action chunks, SyncWorld imagines their visual outcomes from multiple views, and a VLM ranks the rollouts against the instruction. Executing the highest-scoring candidate turns the pretrained world model into a simulator for policy improvement—without updating either the policy or SyncWorld.
Visual example · Pick up the BBQ sauce
Selected imagined rollouts
Improved policy trajectory
Baseline policy trajectory
We test whether six numerical motion controls remain visually grounded after domain shift. Select a DoF to inspect its isolated outcome for a seen embodiment in a new environment and for an entirely unseen robot embodiment.
Choose one translational or rotational action to view its isolated effect.
The same six action primitives are applied to a robot absent from training.
Reliable action simulation should preserve a shared 3D scene across viewpoints. For the same actions, we reconstruct point clouds from SyncWorld rollouts generated in two camera views; their geometry agrees with each other and with ground truth, indicating 3D-consistent dynamics rather than view-specific appearance matching.
Generated rollout · SyncWorld
Ground truth
If you find SyncWorld useful, please cite our work:
@misc{yang2026syncworldvisualcalibrationenablesworld,
title = {{SyncWorld}: Visual Calibration Enables World Models as Zero-Shot Simulators},
author = {Yuncong Yang and Zhengtao Han and Furkan Ozyurt and Zeyuan Yang and Han Yang and Junyi Cao and Haoyu Zhen and Yilun Du and Chuang Gan},
year = {2026},
eprint = {2609.09155},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2609.09155}
}