WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents. - Shunyu Yao; Howard Chen 003; John Yang 002; Karthik Narasimhan (NeurIPS, 2022)

InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback. - John Yang 002; Akshara Prabhakar; Karthik Narasimhan; Shunyu Yao (NeurIPS, 2023)

Referral Augmentation for Zero-Shot Information Retrieval. - Michael Tang; Shunyu Yao; John Yang 002; Karthik Narasimhan (ACL (Findings), 2024)

Disentangled Prompt Learning for Transferable, Multimodal, Few-Shot Image Classification. - John Yang 002; Alessandro Magnani; Binwei Yang (IEEE Big Data, 2024)

SWE-bench: Can Language Models Resolve Real-world Github Issues? - Carlos E. Jimenez; John Yang 002; Alexander Wettig; Shunyu Yao; Kexin Pei; Ofir Press; Karthik R. Narasimhan (ICLR, 2024)

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. - John Yang 002; Carlos E. Jimenez; Alexander Wettig; Kilian Lieret; Shunyu Yao; Karthik Narasimhan; Ofir Press (NeurIPS, 2024)

DevBench: A Comprehensive Benchmark for Software Development. - Bowen Li 002; Wenhan Wu; Ziwei Tang; Lin Shi; John Yang 002; Jinyang Li 003; Shunyu Yao; Chen Qian 006; Binyuan Hui; Qicheng Zhang; Zhiyin Yu; He Du; et al. (CoRR, 2024)

EnIGMA: Enhanced Interactive Generative Model Agent for CTF Challenges. - Talor Abramovich; Meet Udeshi; Minghao Shao; Kilian Lieret; Haoran Xi; Kimberly Milner; Sofija Jancheska; John Yang 002; Carlos E. Jimenez; Farshad Khorrami; Prashanth Krishnamurthy; Brendan Dolan-Gavitt; et al. (CoRR, 2024)

Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration. - Yijia Shao; Vinay Samuel; Yucheng Jiang; John Yang 002; Diyi Yang (CoRR, 2024)

Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study. - Bowen Li 002; Wenhan Wu; Ziwei Tang; Lin Shi; John Yang 002; Jinyang Li 003; Shunyu Yao; Chen Qian 006; Binyuan Hui; Qicheng Zhang; Zhiyin Yu; He Du; et al. (COLING, 2025)

SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains? - John Yang 002; Carlos E. Jimenez; Alex L. Zhang; Kilian Lieret; Joyce Yang; Xindi Wu; Ori Press; Niklas Muennighoff; Gabriel Synnaeve; Karthik R. Narasimhan; Diyi Yang; Sida Wang 001; et al. (ICLR, 2025)

MMTEB: Massive Multilingual Text Embedding Benchmark. - Kenneth C. Enevoldsen; Isaac Chung; Imene Kerboua; Márton Kardos; Ashwin Mathur; David Stap; Jay Gala; Wissam Siblini; Dominik Krzeminski; Genta Indra Winata; Saba Sturua; Saiteja Utpala; et al. (CoRR, 2025)

SWE-smith: Scaling Data for Software Engineering Agents. - John Yang 002; Kilian Leret; Carlos E. Jimenez; Alexander Wettig; Kabir Khandpur; Yanzhe Zhang; Binyuan Hui; Ofir Press; Ludwig Schmidt; Diyi Yang (CoRR, 2025)

LongCodeBench: Evaluating Coding LLMs at 1M Context Windows. - Stefano Rando; Luca Romani; Alessio Sampieri; Yuta Kyuragi; Luca Franco; Fabio Galasso; Tatsunori Hashimoto; John Yang 002 (CoRR, 2025)

OpenThoughts: Data Recipes for Reasoning Models. - Etash Kumar Guha; Ryan Marten; Sedrick Keh; Negin Raoof; Georgios Smyrnis; Hritik Bansal; Marianna Nezhurina; Jean Mercat; Trung Vu 002; Zayne Sprague; Ashima Suvarna; Benjamin Feuer; et al. (CoRR, 2025)