Offline Meta-Reinforcement Learning with Advantage Weighting. - Eric Mitchell; Rafael Rafailov; Xue Bin Peng; Sergey Levine; Chelsea Finn (ICML, 2021)

Offline Reinforcement Learning from Images with Latent Space Models. - Rafael Rafailov; Tianhe Yu; Aravind Rajeswaran; Chelsea Finn (L4DC, 2021)

Visual Adversarial Imitation Learning using Variational Models. - Rafael Rafailov; Tianhe Yu; Aravind Rajeswaran; Chelsea Finn (NeurIPS, 2021)

COMBO: Conservative Offline Model-Based Policy Optimization. - Tianhe Yu; Aviral Kumar; Rafael Rafailov; Aravind Rajeswaran; Sergey Levine; Chelsea Finn (NeurIPS, 2021)

Vision-Based Manipulators Need to Also See from Their Hands. - Kyle Hsu; Moo Jin Kim; Rafael Rafailov; Jiajun Wu 001; Chelsea Finn (ICLR, 2022)

MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learning. - Rafael Rafailov; Kyle Beltran Hatch; Victor Kolev; John D. Martin; Mariano Phielipp; Chelsea Finn (CoRL, 2023)

Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback. - Katherine Tian; Eric Mitchell; Allan Zhou; Archit Sharma; Rafael Rafailov; Huaxiu Yao; Chelsea Finn; Christopher D. Manning (EMNLP, 2023)

Contrastive Example-Based Control. - Kyle Beltran Hatch; Benjamin Eysenbach; Rafael Rafailov; Tianhe Yu; Ruslan Salakhutdinov; Sergey Levine; Chelsea Finn (L4DC, 2023)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model. - Rafael Rafailov; Archit Sharma; Eric Mitchell; Christopher D. Manning; Stefano Ermon; Chelsea Finn (NeurIPS, 2023)

Offline Retraining for Online RL: Decoupled Policy Learning to Mitigate Exploration Bias. - Max Sobol Mark; Archit Sharma; Fahim Tajwar; Rafael Rafailov; Sergey Levine; Chelsea Finn (CoRR, 2023)

Contrastive Preference Learning: Learning from Human Feedback without RL. - Joey Hejna; Rafael Rafailov; Harshit Sikchi; Chelsea Finn; Scott Niekum; W. Bradley Knox; Dorsa Sadigh (CoRR, 2023)

Disentangling Length from Quality in Direct Preference Optimization. - Ryan Park; Rafael Rafailov; Stefano Ermon; Chelsea Finn (ACL (Findings), 2024)

OpenVLA: An Open-Source Vision-Language-Action Model. - Moo Jin Kim; Karl Pertsch; Siddharth Karamcheti; Ted Xiao; Ashwin Balakrishna; Suraj Nair 003; Rafael Rafailov; Ethan Paul Foster; Pannag R. Sanketi; Quan Vuong; Thomas Kollar; Benjamin Burchfiel; et al. (CoRL, 2024)

Diffusion Model Alignment Using Direct Preference Optimization. - Bram Wallace; Meihua Dang; Rafael Rafailov; Linqi Zhou; Aaron Lou; Senthil Purushwalkam; Stefano Ermon; Caiming Xiong; Shafiq Joty; Nikhil Naik (CVPR, 2024)

Contrastive Preference Learning: Learning from Human Feedback without Reinforcement Learning. - Joey Hejna; Rafael Rafailov; Harshit Sikchi; Chelsea Finn; Scott Niekum; W. Bradley Knox; Dorsa Sadigh (ICLR, 2024)

An Emulator for Fine-tuning Large Language Models using Small Language Models. - Eric Mitchell; Rafael Rafailov; Archit Sharma; Chelsea Finn; Christopher D. Manning (ICLR, 2024)

Language Model Detectors Are Easily Optimized Against. - Charlotte Nicks; Eric Mitchell; Rafael Rafailov; Archit Sharma; Christopher D. Manning; Chelsea Finn; Stefano Ermon (ICLR, 2024)

Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data. - Fahim Tajwar; Anikait Singh; Archit Sharma; Rafael Rafailov; Jeff Schneider 001; Tengyang Xie; Stefano Ermon; Chelsea Finn; Aviral Kumar (ICML, 2024)

Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration. - Abby O'Neill; Abdul Rehman; Abhiram Maddukuri; Abhishek Gupta 004; Abhishek Padalkar; Abraham Lee; Acorn Pooley; Agrim Gupta; Ajay Mandlekar; Ajinkya Jain; Albert Tung; Alex Bewley; et al. (ICRA, 2024)

Efficient imitation learning with conservative world models. - Victor Kolev; Rafael Rafailov; Kyle Hatch; Jiajun Wu 001; Chelsea Finn (L4DC, 2024)

Self-Supervised Alignment with Mutual Information: Learning to Follow Principles without Preference Labels. - Jan-Philipp Fränken; Eric Zelikman; Rafael Rafailov; Kanishk Gandhi; Tobias Gerstenberg; Noah D. Goodman (NeurIPS, 2024)

Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms. - Rafael Rafailov; Yaswanth Chittepu; Ryan Park; Harshit Sikchi; Joey Hejna; W. Bradley Knox; Chelsea Finn; Scott Niekum (NeurIPS, 2024)

D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning. - Rafael Rafailov; Kyle Beltran Hatch; Anikait Singh; Aviral Kumar; Laura M. Smith; Ilya Kostrikov; Philippe Hansen-Estruch; Victor Kolev; Philip J. Ball; Jiajun Wu 001; Sergey Levine; Chelsea Finn (RLJ, 2024)

Aligning Modalities in Vision Large Language Models via Preference Fine-tuning. - Yiyang Zhou; Chenhang Cui; Rafael Rafailov; Chelsea Finn; Huaxiu Yao (CoRR, 2024)

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data. - Matthias Gerstgrasser; Rylan Schaeffer; Apratim Dey; Rafael Rafailov; Henry Sleight; John Hughes; Tomasz Korbak; Rajashree Agrawal; Dhruv Pai; Andrey Gromov; Daniel A. Roberts; Diyi Yang; et al. (CoRR, 2024)

From r to Q*: Your Language Model is Secretly a Q-Function. - Rafael Rafailov; Joey Hejna; Ryan Park; Chelsea Finn (CoRR, 2024)

Offline Regularised Reinforcement Learning for Large Language Models Alignment. - Pierre Harvey Richemond; Yunhao Tang; Daniel Guo 001; Daniele Calandriello; Mohammad Gheshlaghi Azar; Rafael Rafailov; Bernardo Ávila Pires; Eugene Tarassov; Lucas Spangher; Will Ellsworth; Aliaksei Severyn; Jonathan Mallinson; et al. (CoRR, 2024)

Scalable Ensembling For Mitigating Reward Overoptimisation. - Ahmed M. Ahmed; Rafael Rafailov; Stepan Sharkov; Xuechen Li; Sanmi Koyejo (CoRR, 2024)

OpenVLA: An Open-Source Vision-Language-Action Model. - Moo Jin Kim; Karl Pertsch; Siddharth Karamcheti; Ted Xiao; Ashwin Balakrishna; Suraj Nair 003; Rafael Rafailov; Ethan Paul Foster; Grace Lam; Pannag Sanketi; Quan Vuong; Thomas Kollar; et al. (CoRR, 2024)

MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? - Zhaorun Chen; Yichao Du; Zichen Wen; Yiyang Zhou; Chenhang Cui; Zhenzhen Weng; Haoqin Tu; Chaoqi Wang; Zhengwei Tong; Qinglan Huang; Canyu Chen; Qinghao Ye; et al. (CoRR, 2024)

Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents. - Pranav Putta; Edmund Mills; Naman Garg; Sumeet Motwani; Chelsea Finn; Divyansh Garg; Rafael Rafailov (CoRR, 2024)

Generative Reward Models. - Dakota Mahan; Duy Phung; Rafael Rafailov; Chase Blagden; Nathan Lile; Louis Castricato; Jan-Philipp Fränken; Chelsea Finn; Alon Albalak (CoRR, 2024)

Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World. - Joshua Kazdan; Rylan Schaeffer; Apratim Dey; Matthias Gerstgrasser; Rafael Rafailov; David L. Donoho; Sanmi Koyejo (CoRR, 2024)

PERSONA: A Reproducible Testbed for Pluralistic Alignment. - Louis Castricato; Nathan Lile; Rafael Rafailov; Jan-Philipp Fränken; Chelsea Finn (COLING, 2025)

Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought. - Violet Xiang; Charlie Snell; Kanishk Gandhi; Alon Albalak; Anikait Singh; Chase Blagden; Duy Phung; Rafael Rafailov; Nathan Lile; Dakota Mahan; Louis Castricato; Jan-Philipp Fränken; et al. (CoRR, 2025)

MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation. - Haibo Tong; Zhaoyang Wang 004; Zhaorun Chen; Haonian Ji; Shi Qiu; Siwei Han; Kexin Geng; Zhongkai Xue; Yiyang Zhou; Peng Xia 005; Mingyu Ding; Rafael Rafailov; et al. (CoRR, 2025)

Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models. - Alon Albalak; Duy Phung; Nathan Lile; Rafael Rafailov; Kanishk Gandhi; Louis Castricato; Anikait Singh; Chase Blagden; Violet Xiang; Dakota Mahan; Nick Haber (CoRR, 2025)

Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning. - Violet Xiang; Chase Blagden; Rafael Rafailov; Nathan Lile; Sang T. Truong; Chelsea Finn; Nick Haber (CoRR, 2025)

LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing. - Daniel Fein; Sebastian Russo; Violet Xiang; Kabir Jolly; Rafael Rafailov; Nick Haber (CoRR, 2025)