RoboWits: Unexpected Challenges for Robotic Creative Problem Solving

NeurIPS 2026 poster Thu 10 Dec · 10:00 AM – 1:00 PM EST · Hall C1, Atlanta

Details Add to calendar
1University of Massachusetts Amherst 2Princeton University 3Stanford University 4Carnegie Mellon University

* Equal contribution

Bi-manual tabletop manipulation in Genesis World 1.3.1 30 seed tasks · 170 mutated variants

Abstract

Benchmarks measure skill.
RoboWits measures wits.

The ability to reason, adapt, and creatively solve problems under unexpected challenges is essential for robots operating in real-world environments. However, current robotic benchmarks primarily emphasize skill-level execution and provide limited insight into such cognitive reasoning capabilities. We introduce RoboWits, a bi-manual robotic benchmark designed to systematically evaluate cognitive reasoning, creative tool use, and robustness to unexpected conditions.

To enable scalable construction of high-quality reasoning-centric unexpected scenarios, we propose an automated task generation pipeline formulated as a multi-agent cooperative framework, comprising agents for seed task generation and verification, metric generation, scene generation, and task mutation. Using the pipeline, we curated 30 diverse seed tasks and 200 mutated tasks with graded difficulty across geometry, material, and assembly-based reasoning.

We benchmark popular robot policy models, pre-trained VLAs, and oracle state-based planners in both single-task fine-tuning and multi-task learning settings. Our results show that while pre-trained VLAs show some preliminary performance on the seed task under single-task fine-tuning settings, they struggle to perform on mutated tasks, implying their brittleness in manipulation tasks requiring non-trivial reasoning, strategy adaptation, and robustness to deceptive or constrained environments.

Task generation

A multi-agent pipeline that writes its own tasks

Seed tasks are proposed and verified at the verbal level, turned into executable success metrics, mutated into graded variants, and only then instantiated as 3D scenes in the simulator.

0:00
Left: specification-level seed generation and verification. Right: metric generation, iterative mutation and verification, and scene instantiation.

    Gallery

    Task gallery

    Mutation examples

    Same instruction. Different scene.

    A mutation keeps the goal and changes what the scene affords: it adds distractors or obstacles, replaces a tool with one of different affordance, or fixes an object the seed solution relied on moving. The policy is never told what changed.

    Benchmark

    State-of-the-art policies do not get far

    Success rate and average progress score over 50 episodes per seed task and 20 episodes per mutation, for three policies fine-tuned on RoboWits demonstrations (ACT, π₀, π₀.₅), alongside Astra-Policy and Fable 5.2. Numbers are the Genesis World 1.3.1 re-run of the full benchmark. Astra-Policy and Fable 5.2 ran on smaller budgets (Fable 5.2: 5 seed and 1-3 mutation episodes per cell), so read their columns as a ranking rather than as per-task estimates.

    Suite
    Seed task Mutated variants

    Average success rate

    Mean over the RoboWits-10 tasks

    Average progress score

    Mean over the RoboWits-10 tasks

    Per-task results

    ● marks the best policy in the row · bar length encodes the success rate

    Citation

    Cite RoboWits

    bibtex
    @article{lin2026robowits,
      title={RoboWits: Unexpected Challenges for Robotic Creative Problem Solving},
      author={Lin, Chunru and Zhang, Hongxin and Yu, Fenghao and Chen, Zhehuan and
              Griffiths, Thomas L and Choi, Yejin and Held, David and Gan, Chuang},
      journal={arXiv preprint arXiv:2605.30326},
      year={2026}
    }