Shengbang Tong

According to our database, Shengbang Tong authored at least 38 papers between 2021 and 2026.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2026

VisionFoundry: Teaching VLMs Visual Perception with Synthetic Images.

Guanyu Zhou, Yida Yin, Wenhao Chai, Shengbang Tong, Xingyu Fu, Zhuang Liu

CoRR, April, 2026

Beyond Language Modeling: An Exploration of Multimodal Pretraining.

Shengbang Tong, David Fan, John Nguyen, Ellis Brown, Gaoyue Zhou, Shengyi Qian, Boyang Zheng, Théophane Vallaeys, Junlin Han, Rob Fergus, Naila Murray, Marjan Ghazvininejad, Mike Lewis, Nicolas Ballas, Amir Bar, Michael Rabbat, Jakob Verbeek, Luke Zettlemoyer, Koustuv Sinha, Yann LeCun, Saining Xie

CoRR, March, 2026

Asymmetric Idiosyncrasies in Multimodal Models.

Muzi Tao, Chufan Shi, Huijuan Wang, Shengbang Tong, Xuezhe Ma

CoRR, February, 2026

Reliable and Responsible Foundation Models: A Comprehensive Survey.

Xinyu Yang, Junlin Han, Rishi Bommasani, Jinqi Luo, Wenjie Qu, Wangchunshu Zhou, Adel Bibi, Xiyao Wang, Jaehong Yoon, Elias Stengel-Eskin, Shengbang Tong, Lingfeng Shen, Rafael Rafailov, Runjia Li, Zhaoyang Wang, Yiyang Zhou, Chenhang Cui, Yu Wang, Wenhao Zheng, Huichi Zhou, Jindong Gu, Zhaorun Chen, Peng Xia, Tony Lee, Thomas P. Zollo, Vikash Sehwag, Jixuan Leng, Jiuhai Chen, Yuxin Wen, Huan Zhang, Zhun Deng, Linjun Zhang, Pavel Izmailov, Pang Wei Koh, Yulia Tsvetkov, Andrew Wilson, Jiaheng Zhang, James Zou, Cihang Xie, Hao Wang, Philip Torr, Julian J. McAuley, David Alvarez-Melis, Florian Tramèr, Kaidi Xu, Suman Jana, Chris Callison-Burch, René Vidal, Filippos Kokkinos, Mohit Bansal, Beidi Chen, Huaxiu Yao

CoRR, February, 2026

Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders.

Shengbang Tong, Boyang Zheng, Ziteng Wang, Bingda Tang, Nanye Ma, Ellis Brown, Jihan Yang, Rob Fergus, Yann LeCun, Saining Xie

CoRR, January, 2026

Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs.

Chun-Hsiao Yeh, Chenyu Wang, Shengbang Tong, Ta Ying Cheng, Ruoyu Wang, Tianzhe Chu, Yuexiang Zhai, Yubei Chen, Shenghua Gao, Yi Ma

Proceedings of the Fortieth AAAI Conference on Artificial Intelligence, 2026

2025

From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images.

Yiming Chen, Junlin Han, Tianyi Bai, Shengbang Tong, Filippos Kokkinos, Philip Torr

CoRR, November, 2025

Cambrian-S: Towards Spatial Supersensing in Video.

Shusheng Yang, Jihan Yang, Pinzhi Huang, Ellis Brown, Zihao Yang, Yue Yu, Shengbang Tong, Zihan Zheng, Yifan Xu, Muhan Wang, Daohan Lu, Rob Fergus, Yann LeCun, Li Fei-Fei, Saining Xie

CoRR, November, 2025

Diffusion Transformers with Representation Autoencoders.

Boyang Zheng, Nanye Ma, Shengbang Tong, Saining Xie

CoRR, October, 2025

Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training.

Junlin Han, Shengbang Tong, David Fan, Yufan Ren, Koustuv Sinha, Philip Torr, Filippos Kokkinos

CoRR, September, 2025

From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models.

Irving Fang, Juexiao Zhang, Shengbang Tong, Chen Feng

CoRR, June, 2025

Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction.

Junhong Shen, Hao Bai, Lunjun Zhang, Yifei Zhou, Amrith Setlur, Shengbang Tong, Diego Caples, Nan Jiang, Tong Zhang, Ameet Talwalkar, Aviral Kumar

CoRR, June, 2025

Reliable and Responsible Foundation Models.

Xinyu Yang, Junlin Han, Rishi Bommasani, Jinqi Luo, Wenjie Qu, Wangchunshu Zhou, Adel Bibi, Xiyao Wang, Jaehong Yoon, Elias Stengel-Eskin, Shengbang Tong, Lingfeng Shen, Rafael Rafailov, Runjia Li, Zhaoyang Wang, Yiyang Zhou, Chenhang Cui, Yu Wang, Wenhao Zheng, Huichi Zhou, Jindong Gu, Zhaorun Chen, Peng Xia, Tony Lee, Thomas P. Zollo, Vikash Sehwag, Jixuan Leng, Jiuhai Chen, Yuxin Wen, Huan Zhang, Zhun Deng, Linjun Zhang, Pavel Izmailov, Pang Wei Koh, Yulia Tsvetkov, Andrew Gordon Wilson, Jiaheng Zhang, James Zou, Cihang Xie, Hao Wang, Philip Torr, Julian J. McAuley, David Alvarez-Melis, Florian Tramèr, Kaidi Xu, Suman Jana, Chris Callison-Burch, René Vidal, Filippos Kokkinos, Mohit Bansal, Beidi Chen, Huaxiu Yao

Trans. Mach. Learn. Res., 2025

SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training.

Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, Yi Ma

Proceedings of the Forty-second International Conference on Machine Learning, 2025

MetaMorph: Multimodal Understanding and Generation via Instruction Tuning.

Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, Zhuang Liu

Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025

Scaling Language-Free Visual Representation Learning.

David Fan, Shengbang Tong, Jiachen Zhu, Koustuv Sinha, Zhuang Liu, Xinlei Chen, Michael Rabbat, Nicolas Ballas, Yann LeCun, Amir Bar, Saining Xie

Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025

MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark.

Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, Graham Neubig

Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025

2024

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?

Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Hao Bai, Yuexiang Zhai, Benjamin D. Haeffele, Yi Ma

J. Mach. Learn. Res., 2024

Connecting Joint-Embedding Predictive Architecture with Contrastive Self-supervised Learning.

Shentong Mo, Shengbang Tong

CoRR, 2024

Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs.

Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, Saining Xie

CoRR, 2024

Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning.

Yuexiang Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Shengbang Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, Sergey Levine

CoRR, 2024

Ctrl123: Consistent Novel View Synthesis via Closed-Loop Transcription.

Hongxiang Zhao, Xili Dai, Jianan Wang, Shengbang Tong, Jingyuan Zhang, Weida Wang, Lei Zhang, Yi Ma

CoRR, 2024

Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models.

Tianzhe Chu, Shengbang Tong, Tianjiao Ding, Xili Dai, Benjamin David Haeffele, René Vidal, Yi Ma

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs.

Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, Saining Xie

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

Investigating the Catastrophic Forgetting in Multimodal Large Language Model Fine-Tuning.

Yuexiang Zhai, Shengbang Tong, Xiao Li, Mu Cai, Qing Qu, Yong Jae Lee, Yi Ma

Proceedings of the Conference on Parsimony and Learning, 2024

Emergence of Segmentation with Minimalistic White-Box Transformers.

Yaodong Yu, Tianzhe Chu, Shengbang Tong, Ziyang Wu, Druv Pai, Sam Buchanan, Yi Ma

Proceedings of the Conference on Parsimony and Learning, 2024

Unsupervised Learning of Structured Representation via Closed-Loop Transcription.

Shengbang Tong, Xili Dai, Yubei Chen, Mingyang Li, Zengyi Li, Brent Yi, Yann LeCun, Yi Ma

Proceedings of the Conference on Parsimony and Learning, 2024

Closed-Loop Transcription via Convolutional Sparse Coding.

Xili Dai, Ke Chen, Shengbang Tong, Jingyuan Zhang, Xingjian Gao, Mingyang Li, Druv Pai, Yuexiang Zhai, Xiaojun Yuan, Heung-Yeung Shum, Lionel M. Ni, Yi Ma

Proceedings of the Conference on Parsimony and Learning, 2024

2023

Investigating the Catastrophic Forgetting in Multimodal Large Language Models.

Yuexiang Zhai, Shengbang Tong, Xiao Li, Mu Cai, Qing Qu, Yong Jae Lee, Yi Ma

CoRR, 2023

EMP-SSL: Towards Self-Supervised Learning in One Training Epoch.

Shengbang Tong, Yubei Chen, Yi Ma, Yann LeCun

CoRR, 2023

White-Box Transformers via Sparse Rate Reduction.

Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Benjamin D. Haeffele, Yi Ma

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Mass-Producing Failures of Multimodal Systems with Language Models.

Shengbang Tong, Erik Jones, Jacob Steinhardt

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Incremental Learning of Structured Memory via Closed-Loop Transcription.

Shengbang Tong, Xili Dai, Ziyang Wu, Mingyang Li, Brent Yi, Yi Ma

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Unsupervised Manifold Linearizing and Clustering.

Tianjiao Ding, Shengbang Tong, Kwan Ho Ryan Chan, Xili Dai, Yi Ma, Benjamin D. Haeffele

Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023

2022

CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction.

Xili Dai, Shengbang Tong, Mingyang Li, Ziyang Wu, Michael Psenka, Kwan Ho Ryan Chan, Pengyuan Zhai, Yaodong Yu, Xiaojun Yuan, Heung-Yeung Shum, Yi Ma

Entropy, 2022

Unsupervised Learning of Structured Representations via Closed-Loop Transcription.

Shengbang Tong, Xili Dai, Yubei Chen, Mingyang Li, Zengyi Li, Brent Yi, Yann LeCun, Yi Ma

CoRR, 2022

Revisiting Sparse Convolutional Model for Visual Recognition.

Xili Dai, Mingyang Li, Pengyuan Zhai, Shengbang Tong, Xingjian Gao, Shao-Lun Huang, Zhihui Zhu, Chong You, Yi Ma

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

2021

Closed-Loop Data Transcription to an LDR via Minimaxing Rate Reduction.

Xili Dai, Shengbang Tong, Mingyang Li, Ziyang Wu, Kwan Ho Ryan Chan, Pengyuan Zhai, Yaodong Yu, Michael Psenka, Xiaojun Yuan, Heung-Yeung Shum, Yi Ma

CoRR, 2021