Xiaojian Ma

According to our database, Xiaojian Ma authored at least 61 papers between 2018 and 2025.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2025

Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigation.

Ziyu Zhu, Xilin Wang, Yixuan Li, Zhuofan Zhang, Xiaojian Ma, Yixin Chen, Baoxiong Jia, Wei Liang, Qian Yu, Zhidong Deng, Siyuan Huang, Qing Li

CoRR, July, 2025

LEO-VL: Towards 3D Vision-Language Generalists via Data Scaling with Efficient Representation.

Jiangyong Huang, Xiaojian Ma, Xiongkun Linghu, Yue Fan, Junchao He, Wenxin Tan, Qing Li, Song-Chun Zhu, Yixin Chen, Baoxiong Jia, Siyuan Huang

CoRR, June, 2025

From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes.

Tianxu Wang, Zhuofan Zhang, Ziyu Zhu, Yue Fan, Jing Xiong, Pengxiang Li, Xiaojian Ma, Qing Li

CoRR, June, 2025

FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulation.

Jun Guo, Xiaojian Ma, Yikai Wang, Min Yang, Huaping Liu, Qing Li

CoRR, May, 2025

Iterative Trajectory Exploration for Multimodal Agents.

Pengxiang Li, Zhi Gao, Bofei Zhang, Yapeng Mi, Xiaojian Ma, Chenrui Shi, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, Qing Li

CoRR, April, 2025

TongUI: Building Generalized GUI Agents by Learning from Multimodal Web Tutorials.

Bofei Zhang, Zirui Shang, Zhi Gao, Wang Zhang, Rui Xie, Xiaojian Ma, Tao Yuan, Xinxiao Wu, Song-Chun Zhu, Qing Li

CoRR, April, 2025

Building LLM Agents by Incorporating Insights from Computer Systems.

Yapeng Mi, Zhi Gao, Xiaojian Ma, Qing Li

CoRR, April, 2025

JARVIS-1: Open-World Multi-Task Agents With Memory-Augmented Multimodal Language Models.

Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jinbing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang, Xiaojian Ma, Yitao Liang

IEEE Trans. Pattern Anal. Mach. Intell., March, 2025

Fast Visuomotor Policies via Partial Denoising.

Haojun Chen, Minghao Liu, Xiaojian Ma, Zailin Ma, Huimin Wu, Chengdong Ma, Yuanpei Chen, Yifan Zhong, Mingzhi Wang, Qing Li, Yaodong Yang

CoRR, March, 2025

LongViTU: Instruction Tuning for Long-Form Video Understanding.

Rujie Wu, Xiaojian Ma, Hai Ci, Yue Fan, Yuxuan Wang, Haozhe Zhao, Qing Li, Yizhou Wang

CoRR, January, 2025

Embodied VideoAgent: Persistent Memory from Egocentric Videos and Embodied Sensors Enables Dynamic Scene Understanding.

Yue Fan, Xiaojian Ma, Rongpeng Su, Jun Guo, Rujie Wu, Xi Chen, Qing Li

CoRR, January, 2025

Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage.

Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaojian Ma, Tao Yuan, Yue Fan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, Qing Li

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

GROOT-2: Weakly Supervised Multimodal Instruction Following Agents.

Shaofei Cai, Bowei Zhang, Zihao Wang, Haowei Lin, Xiaojian Ma, Anji Liu, Yitao Liang

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

ROCKET-1: Mastering Open-World Interaction with Visual-Temporal Context Prompting.

Shaofei Cai, Zihao Wang, Kewei Lian, Zhancun Mu, Xiaojian Ma, Anji Liu, Yitao Liang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse.

Muyao Li, Zihao Wang, Kaichen He, Xiaojian Ma, Yitao Liang

Proceedings of the Findings of the Association for Computational Linguistics, 2025

2024

GROOT-2: Weakly Supervised Multi-Modal Instruction Following Agents.

Shaofei Cai, Bowei Zhang, Zihao Wang, Haowei Lin, Xiaojian Ma, Anji Liu, Yitao Liang

CoRR, 2024

ROCKET-1: Master Open-World Interaction with Visual-Temporal Context Prompting.

Shaofei Cai, Zihao Wang, Kewei Lian, Zhancun Mu, Xiaojian Ma, Anji Liu, Yitao Liang

CoRR, 2024

Task-oriented Sequential Grounding in 3D Scenes.

Zhuofan Zhang, Ziyu Zhu, Pengxiang Li, Tengyu Liu, Xiaojian Ma, Yixin Chen, Baoxiong Jia, Siyuan Huang, Qing Li

CoRR, 2024

Latent Energy-Based Odyssey: Black-Box Optimization via Expanded Exploration in the Energy-Based Latent Space.

Peiyu Yu, Dinghuai Zhang, Hengzhi He, Xiaojian Ma, Ruiyao Miao, Yifan Lu, Yasi Zhang, Deqian Kong, Ruiqi Gao, Jianwen Xie, Guang Cheng, Ying Nian Wu

CoRR, 2024

Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splatting.

Jun Guo, Xiaojian Ma, Yue Fan, Huaping Liu, Qing Li

CoRR, 2024

VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding.

Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, Qing Li

CoRR, 2024

RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation.

Zihao Wang, Anji Liu, Haowei Lin, Jiaqi Li, Xiaojian Ma, Yitao Liang

CoRR, 2024

UltraEdit: Instruction-based Fine-Grained Image Editing at Scale.

Haozhe Zhao, Xiaojian (Shawn) Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, Baobao Chang

Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, 2024

OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents.

Zihao Wang, Shaofei Cai, Zhancun Mu, Haowei Lin, Ceyao Zhang, Xuejie Liu, Qing Li, Anji Liu, Xiaojian (Shawn) Ma, Yitao Liang

Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, 2024

Multi-modal Situated Reasoning in 3D Scenes.

Xiongkun Linghu, Jiangyong Huang, Xuesong Niu, Xiaojian (Shawn) Ma, Baoxiong Jia, Siyuan Huang

Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, 2024

MindAgent: Emergent Gaming Interaction.

Ran Gong, Qiuyuan Huang, Xiaojian Ma, Yusuke Noda, Zane Durante, Zilong Zheng, Demetri Terzopoulos, Li Fei-Fei, Jianfeng Gao, Hoi Vo

Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2024, 2024

An Embodied Generalist Agent in 3D World.

Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, Siyuan Huang

Proceedings of the Forty-first International Conference on Machine Learning, 2024

MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning.

Haozhe Zhao, Zefan Cai, Shuzheng Si, Xiaojian Ma, Kaikai An, Liang Chen, Zixuan Liu, Sheng Wang, Wenjuan Han, Baobao Chang

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Bongard-OpenWorld: Few-Shot Reasoning for Free-form Visual Concepts in the Real World.

Rujie Wu, Xiaojian Ma, Zhenliang Zhang, Wei Wang, Qing Li, Song-Chun Zhu, Yizhou Wang

Proceedings of the Twelfth International Conference on Learning Representations, 2024

GROOT: Learning to Follow Instructions by Watching Gameplay Videos.

Shaofei Cai, Bowei Zhang, Zihao Wang, Xiaojian Ma, Anji Liu, Yitao Liang

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Unifying 3D Vision-Language Understanding via Promptable Queries.

Ziyu Zhu, Zhuofan Zhang, Xiaojian Ma, Xuesong Niu, Yixin Chen, Baoxiong Jia, Zhidong Deng, Siyuan Huang, Qing Li

Proceedings of the Computer Vision - ECCV 2024, 2024

[inline-graphic not available: see fulltext]VideoAgent: A Memory-Augmented Multimodal Agent for Video Understanding.

Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, Qing Li

Proceedings of the Computer Vision - ECCV 2024, 2024

CLOVA: A Closed-LOop Visual Assistant with Tool Usage and Update.

Zhi Gao, Yuntao Du, Xintong Zhang, Xiaojian Ma, Wenjuan Han, Song-Chun Zhu, Qing Li

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

2023

MindAgent: Emergent Gaming Interaction.

Ran Gong, Qiuyuan Huang, Xiaojian Ma, Hoi Vo, Zane Durante, Yusuke Noda, Zilong Zheng, Song-Chun Zhu, Demetri Terzopoulos, Li Fei-Fei, Jianfeng Gao

CoRR, 2023

Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents.

Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma, Yitao Liang

CoRR, 2023

Learning Energy-Based Prior Model with Diffusion-Amortized MCMC.

Peiyu Yu, Yaxuan Zhu, Sirui Xie, Xiaojian Ma, Ruiqi Gao, Song-Chun Zhu, Ying Nian Wu

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Describe, Explain, Plan and Select: Interactive Planning with LLMs Enables Open-World Multi-Task Agents.

Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, Yitao Liang

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

SQA3D: Situated Question Answering in 3D Scenes.

Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, Siyuan Huang

Proceedings of the Eleventh International Conference on Learning Representations, 2023

3D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignment.

Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, Qing Li

Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023

Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction.

Shaofei Cai, Zihao Wang, Xiaojian Ma, Anji Liu, Yitao Liang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

2022

Perceive, Ground, Reason, and Act: A Benchmark for General-purpose Visual Representation.

Jiangyong Huang, William Yicheng Zhu, Baoxiong Jia, Zan Wang, Xiaojian Ma, Qing Li, Siyuan Huang

CoRR, 2022

Latent Diffusion Energy-Based Model for Interpretable Text Modeling.

Peiyu Yu, Sirui Xie, Xiaojian Ma, Baoxiong Jia, Bo Pang, Ruiqi Gao, Yixin Zhu, Song-Chun Zhu, Ying Nian Wu

CoRR, 2022

Latent Diffusion Energy-Based Model for Interpretable Text Modelling.

Peiyu Yu, Sirui Xie, Xiaojian Ma, Baoxiong Jia, Bo Pang, Ruiqi Gao, Yixin Zhu, Song-Chun Zhu, Ying Nian Wu

Proceedings of the International Conference on Machine Learning, 2022

RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning.

Xiaojian Ma, Weili Nie, Zhiding Yu, Huaizu Jiang, Chaowei Xiao, Yuke Zhu, Song-Chun Zhu, Anima Anandkumar

Proceedings of the Tenth International Conference on Learning Representations, 2022

Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions.

Huaizu Jiang, Xiaojian Ma, Weili Nie, Zhiding Yu, Yuke Zhu, Anima Anandkumar

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

2021

HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving.

Sirui Xie, Xiaojian Ma, Peiyu Yu, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu

CoRR, 2021

Unsupervised Foreground Extraction via Deep Region Competition.

Peiyu Yu, Sirui Xie, Xiaojian Ma, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Adversarial Option-Aware Hierarchical Imitation Learning.

Mingxuan Jing, Wenbing Huang, Fuchun Sun, Xiaojian Ma, Tao Kong, Chuang Gan, Lei Li

Proceedings of the 38th International Conference on Machine Learning, 2021

2020

Robust Robotic Pouring using Audition and Haptics.

Hongzhuo Liang, Chuangchuang Zhou, Shuang Li, Xiaojian Ma, Norman Hendrich, Timo Gerkmann, Fuchun Sun, Jianwei Zhang

CoRR, 2020

Robust Robotic Pouring using Audition and Haptics.

Hongzhuo Liang, Chuangchuang Zhou, Shuang Li, Xiaojian Ma, Norman Hendrich, Timo Gerkmann, Fuchun Sun, Marcus Stoffel, Jianwei Zhang

Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020

A Mobile Robot Hand-Arm Teleoperation System by Vision and IMU.

Shuang Li, Jiaxi Jiang, Philipp Ruppel, Hongzhuo Liang, Xiaojian Ma, Norman Hendrich, Fuchun Sun, Jianwei Zhang

Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020

Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidance.

Mingxuan Jing, Xiaojian Ma, Wenbing Huang, Fuchun Sun, Chao Yang, Bin Fang, Huaping Liu

Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020

Theory-Based Causal Transfer: Integrating Instance-Level Induction and Abstract-Level Structure Learning.

Mark Edmonds, Xiaojian Ma, Siyuan Qi, Yixin Zhu, Hongjing Lu, Song-Chun Zhu

Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020

2019

Making Sense of Audio Vibration for Liquid Height Estimation in Robotic Pouring.

Hongzhuo Liang, Shuang Li, Xiaojian Ma, Norman Hendrich, Timo Gerkmann, Jianwei Zhang

CoRR, 2019

Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement.

Chao Yang, Xiaojian Ma, Wenbing Huang, Fuchun Sun, Huaping Liu, Junzhou Huang, Chuang Gan

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Making Sense of Audio Vibration for Liquid Height Estimation in Robotic Pouring.

Hongzhuo Liang, Shuang Li, Xiaojian Ma, Norman Hendrich, Timo Gerkmann, Fuchun Sun, Jianwei Zhang

Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2019

PointNetGPD: Detecting Grasp Configurations from Point Sets.

Hongzhuo Liang, Xiaojian Ma, Shuang Li, Michael Görner, Song Tang, Bin Fang, Fuchun Sun, Jianwei Zhang

Proceedings of the International Conference on Robotics and Automation, 2019

Vision-based Teleoperation of Shadow Dexterous Hand using End-to-End Deep Neural Network.

Shuang Li, Xiaojian Ma, Hongzhuo Liang, Michael Görner, Philipp Ruppel, Bin Fang, Fuchun Sun, Jianwei Zhang

Proceedings of the International Conference on Robotics and Automation, 2019

Task Transfer by Preference-Based Cost Learning.

Mingxuan Jing, Xiaojian Ma, Wen-bing Huang, Fuchun Sun, Huaping Liu

Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, 2019

2018

Learning and Inference Movement with Deep Generative Model.

Mingxuan Jing, Xiaojian Ma, Fuchun Sun, Huaping Liu

CoRR, 2018

Adversarial Task Transfer from Preference.

Xiaojian Ma, Mingxuan Jing, Fuchun Sun, Huaping Liu

CoRR, 2018