Embodied AGI

3D-LLM: Injecting the 3D World into Large Language Models

Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, Chuang Gan

NeurIPS 2023 (Spotlight)

Adaptive online replanning with diffusion models

Siyuan Zhou, Yilun Du, Shun Zhang, Mengdi Xu, Yikang Shen, Wei Xiao, Dit-Yan Yeung, Chuang Gan

NeurIPS 2023

DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics

Zhiao Huang, Feng Chen, Yewen Pu, Chunru Lin, Hao Su, Chuang Gan

NeurIPS 2023

DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models

Tsun-Hsuan Wang, Juntian Zheng, Pingchuan Ma, Yilun Du, Byungchul Kim, Andrew Spielberg, Joshua Tenenbaum, Chuang Gan, Daniela Rus

NeurIPS 2023 (Oral)

Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties

Hsiao-Yu Tung, Mingyu Ding, Zhenfang Chen, Daniel Bear, Chuang Gan, Joshua B. Tenenbaum, Daniel LK Yamins, Judith E Fan, Kevin A. Smith

NeurIPS 2023 Dataset Track

TextPSG: Panoptic Scene Graph Generation from Textual Descriptions

Chengyang Zhao, Yikang Shen, Zhenfang Chen, Mingyu Ding, Chuang Gan

ICCV 2023

EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction

Han Cai, Junyan Li, Muyan Hu, Chuang Gan, Song Han

ICCV 2023

Learning Vision-and-Language Navigation from YouTube Videos

Kunyang Lin, Peihao Chen, Diwei Huang, Thomas H. Li, Mingkui Tan, Chuang Gan

ICCV 2023

Sparse Universal Transformer

Shawn Tan, Yikang Shen, Zhenfang Chen, Aaron Courville, Chuang Gan

EMNLP 2023

Learning Neural Constitutive Laws from Motion Observations for Generalizable PDE Dynamics

Pingchuan Ma, Peter Yichen Chen, Bolei Deng, Joshua B. Tenenbaum, Tao Du, Chuang Gan, Wojciech Matusik

ICML 2023

Reparameterized Policy Learning for Multimodal Trajectory Optimization

Zhiao Huang, Litian Liang, Zhan Ling, Xuanlin Li, Chuang Gan, Hao Su

ICML 2023 (Oral)

On the Forward Invariance of Neural ODEs

Wei Xiao, Tsun-Hsuan Wang, Ramin Hasani, Mathias Lechner, Yutong Ban, Chuang Gan, Daniela Rus

ICML 2023

Roboninja: Learning an Adaptive Cutting Policy for Multi-material Objects

Zhenjia Xu, Zhou Xian, Xingyu Lin, Cheng Chi, Zhiao Huang, Chuang Gan, Shuran Song

RSS 2023

JECC: Commonsense Reasoning Tasks Derived from Interactive Fictions

Mo Yu*, Yi Gu*, Xiaoxiao Guo, Yufei Feng, Xiaodan Zhu, Michael Greenspan, Murray Campbell, Chuang Gan

ACL 2023 (Findings)

3D Concept Learning and Reasoning from Multi-View Images

Yining Hong, Chunru Lin, Yilun Du, Zhenfang Chen, Joshua B. Tenenbaum, Chuang Gan

CVPR 2023

Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learners

Zitian Chen, Yikang Shen, Mingyu Ding, Zhenfang Chen, Hengshuang Zhao, Erik Learned-Miller, Chuang Gan

CVPR 2023

EC^ 2: Emergent Communication for Embodied Control

Yao Mu, Shunyu Yao, Mingyu Ding, Ping Luo, Chuang Gan

CVPR 2023

Physics-Driven Diffusion Models for Impact Sound Synthesis from Videos

Kun Su, Kaizhi Qian, Eli Shlizerman, Antonio Torralba, Chuang Gan

CVPR 2023

Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention

Mingyu Ding, Yikang Shen, Lijie Fan, Zhenfang Chen, Zitian Chen, Ping Luo, Joshua B Tenenbaum, Chuang Gan

CVPR 2023

Masked Motion Encoding for Self-Supervised Video Representation Learning

Xinyu Sun, Peihao Chen, Liangwei Chen, Changhao Li, Thomas H. Li, Mingkui Tan, Chuang Gan

CVPR 2023

FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation

Zhou Xian, Bo Zhu, Zhenjia Xu, Hsiao-Yu Tung, Antonio Torralba, Katerina Fragkiadaki, Chuang Gan

ICLR 2023 (Spotlight)

PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification

Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, Chuang Gan

ICLR 2023 (Spotlight)

SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environments

Tsun-Hsuan Wang, Pingchuan Ma, Andrew Everett Spielberg, Zhou Xian, Hao Zhang, Joshua B Tenenbaum, Daniela Rus, Chuang Gan

ICLR 2023

Planning with Large Language Models for Code Generation

Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B Tenenbaum, Chuang Gan

ICLR 2023

DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physics

Sizhe Li*, Zhiao Huang*, Tao Chen, Tao Du, Hao Su, Joshua B Tenenbaum, Chuang Gan

ICLR 2023

Hyper-Decision Transformer for Efficient Online Policy Adaptation

Mengdi Xu, Yuchen Lu, Yikang Shen, Shun Zhang, Ding Zhao, Chuang Gan

ICLR 2023