RoboWits: Unexpected Challenges for Robotic Creative Problem Solving
* Equal contribution
Abstract
Benchmarks measure skill.
RoboWits measures wits.
The ability to reason, adapt, and creatively solve problems under unexpected challenges is essential for robots operating in real-world environments. However, current robotic benchmarks primarily emphasize skill-level execution and provide limited insight into such cognitive reasoning capabilities. We introduce RoboWits, a bi-manual robotic benchmark designed to systematically evaluate cognitive reasoning, creative tool use, and robustness to unexpected conditions.
To enable scalable construction of high-quality reasoning-centric unexpected scenarios, we propose an automated task generation pipeline formulated as a multi-agent cooperative framework, comprising agents for seed task generation and verification, metric generation, scene generation, and task mutation. Using the pipeline, we curated 30 diverse seed tasks and 200 mutated tasks with graded difficulty across geometry, material, and assembly-based reasoning.
We benchmark popular robot policy models, pre-trained VLAs, and oracle state-based planners in both single-task fine-tuning and multi-task learning settings. Our results show that while pre-trained VLAs show some preliminary performance on the seed task under single-task fine-tuning settings, they struggle to perform on mutated tasks, implying their brittleness in manipulation tasks requiring non-trivial reasoning, strategy adaptation, and robustness to deceptive or constrained environments.
Task generation
A multi-agent pipeline that writes its own tasks
Seed tasks are proposed and verified at the verbal level, turned into executable success metrics, mutated into graded variants, and only then instantiated as 3D scenes in the simulator.
Gallery
Task gallery
Mutation examples
Same instruction. Different scene.
A mutation keeps the goal and changes what the scene affords: it adds distractors or obstacles, replaces a tool with one of different affordance, or fixes an object the seed solution relied on moving. The policy is never told what changed.
Benchmark
State-of-the-art policies do not get far
Success rate and average progress score over 50 episodes per seed task and 20 episodes per mutation, for three policies fine-tuned on RoboWits demonstrations (ACT, π₀, π₀.₅), alongside Astra-Policy and Fable 5.2. Numbers are the Genesis World 1.3.1 re-run of the full benchmark. Astra-Policy and Fable 5.2 ran on smaller budgets (Fable 5.2: 5 seed and 1-3 mutation episodes per cell), so read their columns as a ranking rather than as per-task estimates.
Average success rate
Mean over the RoboWits-10 tasks
Average progress score
Mean over the RoboWits-10 tasks
Citation
Cite RoboWits
@article{lin2026robowits,
title={RoboWits: Unexpected Challenges for Robotic Creative Problem Solving},
author={Lin, Chunru and Zhang, Hongxin and Yu, Fenghao and Chen, Zhehuan and
Griffiths, Thomas L and Choi, Yejin and Held, David and Gan, Chuang},
journal={arXiv preprint arXiv:2605.30326},
year={2026}
}