Integrating Language and Vision to Generate Natural Language Descriptions of Videos in the Wild. - Jesse Thomason; Subhashini Venugopalan; Sergio Guadarrama; Kate Saenko; Raymond J. Mooney (COLING, 2014)

Learning to Interpret Natural Language Commands through Human-Robot Dialog. - Jesse Thomason; Shiqi Zhang 001; Raymond J. Mooney; Peter Stone 001 (IJCAI, 2015)

Learning Multi-Modal Grounded Linguistic Semantics by Playing "I Spy". - Jesse Thomason; Jivko Sinapov; Maxwell Svetlik; Peter Stone 001; Raymond J. Mooney (IJCAI, 2016)

Guiding Interaction Behaviors for Multi-modal Grounded Language Learning. - Jesse Thomason; Jivko Sinapov; Raymond J. Mooney (RoboNLP@ACL, 2017)

Opportunistic Active Learning for Grounding Natural Language Descriptions. - Jesse Thomason; Aishwarya Padmakumar; Jivko Sinapov; Justin W. Hart; Peter Stone 001; Raymond J. Mooney (CoRL, 2017)

Integrated Learning of Dialog Strategies and Semantic Parsing. - Aishwarya Padmakumar; Jesse Thomason; Raymond J. Mooney (EACL (1), 2017)

Multi-Modal Word Synset Induction. - Jesse Thomason; Raymond J. Mooney (IJCAI, 2017)

Improving Black-box Speech Recognition using Semantic Parsing. - Rodolfo Corona; Jesse Thomason; Raymond J. Mooney (IJCNLP(2), 2017)

BWIBots: A platform for bridging the gap between AI and human-robot interaction research. - Piyush Khandelwal; Shiqi Zhang 001; Jivko Sinapov; Matteo Leonetti; Jesse Thomason; Fangkai Yang; Ilaria Gori; Maxwell Svetlik; Priyanka Khante; Vladimir Lifschitz; J. K. Aggarwal; Raymond J. Mooney; et al. (Int. J. Robotics Res., 2017)

Maximum-Variance Total Variation Denoising for Interpretable Spatial Smoothing. - Wesley Tansey; Jesse Thomason; James G. Scott (AAAI, 2018)

Guiding Exploratory Behaviors for Multi-Modal Grounding of Linguistic Descriptions. - Jesse Thomason; Jivko Sinapov; Raymond J. Mooney; Peter Stone 001 (AAAI, 2018)

Robot Behavioral Exploration and Multi-modal Perception using Dynamically Constructed Controllers. - Saeid Amiri; Suhua Wei; Shiqi Zhang 001; Jivko Sinapov; Jesse Thomason; Peter Stone 001 (AAAI Spring Symposia, 2018)

Multi-modal Predicate Identification using Dynamically Learned Robot Controllers. - Saeid Amiri; Suhua Wei; Shiqi Zhang 001; Jivko Sinapov; Jesse Thomason; Peter Stone 001 (IJCAI, 2018)

Interaction and Autonomy in RoboCup@Home and Building-Wide Intelligence. - Justin W. Hart; Harel Yedidsion; Yuqian Jiang; Nick Walker 001; Rishi Shah; Jesse Thomason; Aishwarya Padmakumar; Rolando Fernandez; Jivko Sinapov; Raymond J. Mooney; Peter Stone 001 (CoRR, 2018)

Vision-and-Dialog Navigation. - Jesse Thomason; Michael Murray; Maya Cakmak; Luke Zettlemoyer (CoRL, 2019)

Prospection: Interpretable plans from language by predicting the future. - Chris Paxton 001; Yonatan Bisk; Jesse Thomason; Arunkumar Byravan; Dieter Fox (ICRA, 2019)

Improving Grounded Natural Language Understanding through Human-Robot Dialog. - Jesse Thomason; Aishwarya Padmakumar; Jivko Sinapov; Nick Walker 001; Yuqian Jiang; Harel Yedidsion; Justin W. Hart; Peter Stone 001; Raymond J. Mooney (ICRA, 2019)

Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog. - Saeid Amiri; Sujay Bajracharya; Cihangir Goktolga; Jesse Thomason; Shiqi Zhang 001 (IROS, 2019)

Improving Robot Success Detection using Static Object Data. - Rosario Scalise; Jesse Thomason; Yonatan Bisk; Siddhartha S. Srinivasa (IROS, 2019)

Shifting the Baseline: Single Modality Performance on Visual Navigation & QA. - Jesse Thomason; Daniel Gordon; Yonatan Bisk (NAACL-HLT (1), 2019)

Interpreting Black Box Models with Statistical Guarantees. - Collin Burns; Jesse Thomason; Wesley Tansey (CoRR, 2019)

The RobotSlang Benchmark: Dialog-guided Robot Localization and Navigation. - Shurjo Banerjee; Jesse Thomason; Jason J. Corso (CoRL, 2020)

ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks. - Mohit Shridhar; Jesse Thomason; Daniel Gordon; Yonatan Bisk; Winson Han; Roozbeh Mottaghi; Luke Zettlemoyer; Dieter Fox (CVPR, 2020)

Experience Grounds Language. - Yonatan Bisk; Ari Holtzman; Jesse Thomason; Jacob Andreas; Yoshua Bengio; Joyce Chai; Mirella Lapata; Angeliki Lazaridou; Jonathan May; Aleksandr Nisnevich; Nicolas Pinto; Joseph P. Turian (EMNLP (1), 2020)

RMM: A Recursive Mental Model for Dialog Navigation. - Homero Roman Roman; Yonatan Bisk; Jesse Thomason; Asli Celikyilmaz; Jianfeng Gao 001 (EMNLP (Findings), 2020)

Interpreting Black Box Models via Hypothesis Testing. - Collin Burns; Jesse Thomason; Wesley Tansey (FODS, 2020)

Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog. - Jesse Thomason; Aishwarya Padmakumar; Jivko Sinapov; Nick Walker 001; Yuqian Jiang; Harel Yedidsion; Justin W. Hart; Peter Stone 001; Raymond J. Mooney (J. Artif. Intell. Res., 2020)

Language Grounding with 3D Objects. - Jesse Thomason; Mohit Shridhar; Yonatan Bisk; Chris Paxton 001; Luke Zettlemoyer (CoRL, 2021)

Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion. - Alessandro Suglia; Qiaozi Gao; Jesse Thomason; Govind Thattai; Gaurav S. Sukhatme (CoRR, 2021)

LUMINOUS: Indoor Scene Generation for Embodied AI Challenges. - Yizhou Zhao; Kaixiang Lin; Zhiwei Jia; Qiaozi Gao; Govind Thattai; Jesse Thomason; Gaurav S. Sukhatme (CoRR, 2021)

TEACh: Task-Driven Embodied Agents That Chat. - Aishwarya Padmakumar; Jesse Thomason; Ayush Shrivastava; Patrick Lange; Anjali Narayan-Chen; Spandana Gella; Robinson Piramuthu; Gökhan Tür; Dilek Hakkani-Tür (AAAI, 2022)

Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions. - Jing Gu; Eliana Stefani; Qi Wu 001; Jesse Thomason; Xin Wang 061 (ACL (1), 2022)

Generalization Differences between End-to-End and Neuro-Symbolic Vision-Language Reasoning Systems. - Wang Zhu 001; Jesse Thomason; Robin Jia (EMNLP (Findings), 2022)

ALFRED-L: Investigating the Role of Language for Action Learning in Interactive Visual Environments. - Arjun R. Akula; Spandana Gella; Aishwarya Padmakumar; Mahdi Namazifar; Mohit Bansal; Jesse Thomason; Dilek Hakkani-Tur (EMNLP, 2022)

SSL Enables Learning from Sparse Rewards in Image-Goal Navigation. - Arjun Majumdar; Gunnar A. Sigurdsson; Robinson Piramuthu; Jesse Thomason; Dhruv Batra; Gaurav S. Sukhatme (ICML, 2022)

CLiMB: A Continual Learning Benchmark for Vision-and-Language Tasks. - Tejas Srinivasan; Ting-Yun Chang; Leticia Leonor Pinto Alva; Georgios Chochlakis; Mohammad Rostami; Jesse Thomason (NeurIPS, 2022)

Interactive Learning from Natural Language and Demonstrations using Signal Temporal Logic. - Sara Mohammadinejad; Jesse Thomason; Jyotirmoy V. Deshmukh (CoRR, 2022)

VAuLT: Augmenting the Vision-and-Language Transformer with the Propagation of Deep Language Representations. - Georgios Chochlakis; Tejas Srinivasan; Jesse Thomason; Shrikanth Narayanan (CoRR, 2022)

Retrospectives on the Embodied AI Workshop. - Matt Deitke; Dhruv Batra; Yonatan Bisk; Tommaso Campari; Angel X. Chang; Devendra Singh Chaplot; Changan Chen; Claudia Pérez-D'Arpino; Kiana Ehsani; Ali Farhadi; Li Fei-Fei 001; Anthony G. Francis; et al. (CoRR, 2022)

CLIP-Nav: Using CLIP for Zero-Shot Vision-and-Language Navigation. - Vishnu Sashank Dorbala; Gunnar A. Sigurdsson; Robinson Piramuthu; Jesse Thomason; Gaurav S. Sukhatme (CoRR, 2022)

The Sem-Lex Benchmark: Modeling ASL Signs and their Phonemes. - Lee Kezar; Jesse Thomason; Naomi Caselli; Zed Sevcikova Sehyr; Elana Pontecorvo (ASSETS, 2023)

I2I: Initializing Adapters with Improvised Knowledge. - Tejas Srinivasan; Furong Jia 002; Mohammad Rostami; Jesse Thomason (CoLLAs, 2023)

Iterative Vision-and-Language Navigation. - Jacob Krantz; Shurjo Banerjee; Wang Zhu 001; Jason J. Corso; Peter Anderson; Stefan Lee; Jesse Thomason (CVPR, 2023)

Curriculum Learning for Data-Efficient Vision-Language Alignment. - Tejas Srinivasan; Xiang Ren; Jesse Thomason (CVPR Workshops, 2023)

Improving Sign Recognition with Phonology. - Lee Kezar; Jesse Thomason; Zed Sevcikova Sehyr (EACL, 2023)

Chain-of-Questions Training with Latent Answers for Robust Multistep Question Answering. - Wang Zhu 001; Jesse Thomason; Robin Jia (EMNLP, 2023)

Task-Attentive Transformer Architecture for Continual Learning of Vision-and-Language Tasks Using Knowledge Distillation. - Yuliang Cai; Jesse Thomason; Mohammad Rostami (EMNLP (Findings), 2023)

Exploring Strategies for Modeling Sign Language Phonology. - Lee Kezar; Tejas Srinivasan; Riley Carlin; Jesse Thomason; Zed Sevcikova Sehyr; Naomi Caselli (ESANN, 2023)

ProgPrompt: Generating Situated Robot Task Plans using Large Language Models. - Ishika Singh; Valts Blukis; Arsalan Mousavian; Ankit Goyal 001; Danfei Xu; Jonathan Tremblay; Dieter Fox; Jesse Thomason; Animesh Garg (ICRA, 2023)

Multimodal Speech Recognition for Language-Guided Embodied Agents. - Allen Chang; Xiaoyuan Zhu; Aarav Monga; Seoho Ahn; Tejas Srinivasan; Jesse Thomason (INTERSPEECH, 2023)

RREx-BoT: Remote Referring Expressions with a Bag of Tricks. - Gunnar A. Sigurdsson; Jesse Thomason; Gaurav S. Sukhatme; Robinson Piramuthu (IROS, 2023)

Geolocated Social Media Posts are Happier: Understanding the Characteristics of Check-in Posts on Twitter. - Julie Jiang; Jesse Thomason; Francesco Barbieri; Emilio Ferrara (WebSci, 2023)

The Sem-Lex Benchmark: Modeling ASL Signs and Their Phonemes. - Lee Kezar; Elana Pontecorvo; Adele Daniels; Connor Baer; Ruth Ferster; Lauren Berger; Jesse Thomason; Zed Sevcikova Sehyr; Naomi Caselli (CoRR, 2023)

Comparative Multi-View Language Grounding. - Chancharik Mitra; Abrar Anwar; Rodolfo Corona; Dan Klein 001; Trevor Darrell; Jesse Thomason (CoRR, 2023)

Do Localization Methods Actually Localize Memorized Data in LLMs? - Ting-Yun Chang; Jesse Thomason; Robin Jia (CoRR, 2023)

Does VLN Pretraining Work with Nonsensical or Irrelevant Instructions? - Wang Zhu 001; Ishika Singh; Yuan Huang; Robin Jia; Jesse Thomason (CoRR, 2023)

Multimodal embodied attribute learning by robots for object-centric action policies. - Xiaohan Zhang 002; Saeid Amiri; Jivko Sinapov; Jesse Thomason; Peter Stone 001; Shiqi Zhang 001 (Auton. Robots, 2023)

ProgPrompt: program generation for situated robot task planning using large language models. - Ishika Singh; Valts Blukis; Arsalan Mousavian; Ankit Goyal 001; Danfei Xu; Jonathan Tremblay; Dieter Fox; Jesse Thomason; Animesh Garg (Auton. Robots, 2023)

Selective "Selective Prediction": Reducing Unnecessary Abstention in Vision-Language Reasoning. - Tejas Srinivasan; Jack Hessel; Tanmay Gupta; Bill Yuchen Lin; Yejin Choi 001; Jesse Thomason; Khyathi Raghavi Chandu (ACL (Findings), 2024)

Contrast Sets for Evaluating Language-Guided Robot Policies. - Abrar Anwar; Rohan Gupta; Jesse Thomason (CoRL, 2024)

When Parts Are Greater Than Sums: Individual LLM Components Can Outperform Full Models. - Ting-Yun Chang; Jesse Thomason; Robin Jia (EMNLP, 2024)

ViSaRL: Visual Reinforcement Learning Guided by Human Saliency. - Anthony Liang; Jesse Thomason; Erdem Biyik (IROS, 2024)

Efficient End-to-End Visual Document Understanding with Rationale Distillation. - Wang Zhu 001; Alekh Agarwal; Mandar Joshi; Robin Jia; Jesse Thomason; Kristina Toutanova (NAACL-HLT, 2024)

Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks. - Ting-Yun Chang; Jesse Thomason; Robin Jia (NAACL-HLT, 2024)

Which One? Leveraging Context Between Objects and Multiple Views for Language Grounding. - Chancharik Mitra; Abrar Anwar; Rodolfo Corona; Dan Klein 001; Trevor Darrell; Jesse Thomason (NAACL-HLT, 2024)

THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation. - Wilbert Pumacay; Ishika Singh; Jiafei Duan; Ranjay Krishna; Jesse Thomason; Dieter Fox (Robotics: Science and Systems, 2024)

Systematic Translation from Natural Language Robot Task Descriptions to STL. - Sara Mohammadinejad; Sheryl Paul; Yuan Xia; Vidisha Kudalkar; Jesse Thomason; Jyotirmoy V. Deshmukh (AISoLA, 2024)

WinoViz: Probing Visual Properties of Objects Under Different States. - Woojeong Jin 001; Tejas Srinivasan; Jesse Thomason; Xiang Ren 001 (CoRR, 2024)

TwoStep: Multi-agent Task Planning using Classical Planners and Large Language Models. - Ishika Singh; David Traum; Jesse Thomason (CoRR, 2024)

Language Models can Infer Action Semantics for Classical Planners from Environment Feedback. - Wang Zhu 001; Ishika Singh; Robin Jia; Jesse Thomason (CoRR, 2024)

Generating Contextually-Relevant Navigation Instructions for Blind and Low Vision People. - Zain Merchant; Abrar Anwar; Emily Wang; Souti Chattopadhyay; Jesse Thomason (CoRR, 2024)

Evaluating Creativity and Deception in Large Language Models: A Simulation Framework for Multi-Agent Balderdash. - Parsa Hejabi; Elnaz Rahmati; Alireza S. Ziabari; Preni Golazizian; Jesse Thomason; Morteza Dehghani (CoRR, 2024)

Promoting Cognitive Health in Elder Care with Large Language Model-Powered Socially Assistive Robots. - Maria R. Lima; Amy O'Connell; Feiyang Zhou; Alethea Nagahara; Avni Hulyalkar; Anura Deshpande; Jesse Thomason; Ravi Vaidyanathan; Maja J. Mataric (CHI, 2025)

The American Sign Language Knowledge Graph: Infusing ASL Models with Linguistic Knowledge. - Lee Kezar; Nidhi Munikote; Zian Zeng; Zed Sevcikova Sehyr; Naomi Caselli; Jesse Thomason (NAACL (Findings), 2025)

Language Models Can Infer Action Semantics for Symbolic Planners from Environment Feedback. - Wang Bill Zhu; Ishika Singh; Robin Jia; Jesse Thomason (NAACL (Long Papers), 2025)

M3PT: A Transformer for Multimodal, Multi-Party Social Signal Prediction with Person-aware Blockwise Attention. - Yiming Tang; Abrar Anwar; Jesse Thomason (CoRR, 2025)

Better Slow than Sorry: Introducing Positive Friction for Reliable Dialogue Systems. - Mert Inan; Anthony Sicilia; Suvodip Dey; Vardhan Dongre; Tejas Srinivasan; Jesse Thomason; Gökhan Tür; Dilek Hakkani-Tür; Malihe Alikhani (CoRR, 2025)

Efficient Evaluation of Multi-Task Robot Policies With Active Experiment Selection. - Abrar Anwar; Rohan Gupta; Zain Merchant; Sayan Ghosh; Willie Neiswanger; Jesse Thomason (CoRR, 2025)

Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance. - Tejas Srinivasan; Jesse Thomason (CoRR, 2025)

Multi-modal Synthetic Data Training and Model Collapse: Insights from VLMs and Diffusion Models. - Zizhao Hu; Mohammad Rostami; Jesse Thomason (CoRR, 2025)

ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrations. - Jiahui Zhang; Yusen Luo; Abrar Anwar; Sumedh Anand Sontakke; Joseph J. Lim; Jesse Thomason; Erdem Biyik; Jesse Zhang (CoRR, 2025)

Zero-Shot Iterative Formalization and Planning in Partially Observable Environments. - Liancheng Gong; Wang Zhu 001; Jesse Thomason; Li Zhang (CoRR, 2025)

Large Language Models Do Multi-Label Classification Differently. - Marcus Ma; Georgios Chochlakis; Niyantha Maruthu Pandiyan; Jesse Thomason; Shrikanth Narayanan (CoRR, 2025)

TNG-CLIP:Training-Time Negation Data Generation for Negation Awareness of CLIP. - Yuliang Cai; Jesse Thomason; Mohammad Rostami (CoRR, 2025)

HAND Me the Data: Fast Robot Adaptation via Hand Path Retrieval. - Matthew Hong 002; Anthony Liang; Kevin Kim; Harshitha Rajaprakash; Jesse Thomason; Erdem Biyik; Jesse Zhang (CoRR, 2025)

From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered. - Siddartha Devic; Tejas Srinivasan; Jesse Thomason; Willie Neiswanger; Vatsal Sharan (CoRR, 2025)

Why Do Some Inputs Break Low-Bit LLM Quantization? - Ting-Yun Chang; Muru Zhang; Jesse Thomason; Robin Jia (CoRR, 2025)

PSALM-V: Automating Symbolic Planning in Interactive Visual Environments with Large Language Models. - Wang Bill Zhu; Miaosen Chai; Ishika Singh; Robin Jia; Jesse Thomason (CoRR, 2025)