Methods for Computing State Similarity in Markov Decision Processes. - Norm Ferns; Pablo Samuel Castro; Doina Precup; Prakash Panangaden (UAI, 2006)

Using Linear Programming for Bayesian Exploration in Markov Decision Processes. - Pablo Samuel Castro; Doina Precup (IJCAI, 2007)

Equivalence Relations in Fully and Partially Observable Markov Decision Processes. - Pablo Samuel Castro; Prakash Panangaden; Doina Precup (IJCAI, 2009)

Using Bisimulation for Policy Transfer in MDPs. - Pablo Samuel Castro; Doina Precup (AAAI, 2010)

Smarter Sampling in Model-Based Bayesian Reinforcement Learning. - Pablo Samuel Castro; Doina Precup (ECML/PKDD (1), 2010)

Automatic Construction of Temporally Extended Actions for MDPs Using Bisimulation Metrics. - Pablo Samuel Castro; Doina Precup (EWRL, 2011)

Real-Time Detection of Anomalous Taxi Trajectories from GPS Traces. - Chao Chen 004; Daqing Zhang 001; Pablo Samuel Castro; Nan Li 019; Lin Sun 009; Shijian Li (MobiQuitous, 2011)

Urban Traffic Modelling and Prediction Using Large Scale Taxi GPS Traces. - Pablo Samuel Castro; Daqing Zhang 001; Shijian Li (Pervasive, 2012)

From taxi GPS traces to social and community dynamics: A survey. - Pablo Samuel Castro; Daqing Zhang 001; Chao Chen 004; Shijian Li; Gang Pan 001 (ACM Comput. Surv., 2013)

Real Time Anomalous Trajectory Detection and Analysis. - Lin Sun 009; Daqing Zhang 001; Chao Chen 004; Pablo Samuel Castro; Shijian Li; Zonghui Wang (Mob. Networks Appl., 2013)

iBOAT: Isolation-Based Online Anomalous Trajectory Detection. - Chao Chen 004; Daqing Zhang 001; Pablo Samuel Castro; Nan Li 019; Lin Sun 009; Shijian Li; Zonghui Wang (IEEE Trans. Intell. Transp. Syst., 2013)

Combining Learned Lyrical Structures and Vocabulary for Improved Lyric Generation. - Pablo Samuel Castro; Maria Attarian (CoRR, 2018)

Dopamine: A Research Framework for Deep Reinforcement Learning. - Pablo Samuel Castro; Subhodeep Moitra; Carles Gelada; Saurabh Kumar 004; Marc G. Bellemare (CoRR, 2018)

An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents. - Felipe Petroski Such; Vashisht Madhavan; Rosanne Liu; Rui Wang 052; Pablo Samuel Castro; Yulun Li; Ludwig Schubert; Marc G. Bellemare; Jeff Clune; Joel Lehman (CoRR, 2018)

A Comparative Analysis of Expected and Distributional Reinforcement Learning. - Clare Lyle; Marc G. Bellemare; Pablo Samuel Castro (AAAI, 2019)

Distributional reinforcement learning with linear function approximation. - Marc G. Bellemare; Nicolas Le Roux; Pablo Samuel Castro; Subhodeep Moitra (AISTATS, 2019)

Performing Structured Improvisations with Pre-trained Deep Learning Models. - Pablo Samuel Castro (ICCC, 2019)

An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents. - Felipe Petroski Such; Vashisht Madhavan; Rosanne Liu; Rui Wang 052; Pablo Samuel Castro; Yulun Li; Jiale Zhi; Ludwig Schubert; Marc G. Bellemare; Jeff Clune; Joel Lehman (IJCAI, 2019)

A Geometric Perspective on Optimal Representations for Reinforcement Learning. - Marc G. Bellemare; Will Dabney; Robert Dadashi; Adrien Ali Taïga; Pablo Samuel Castro; Nicolas Le Roux; Dale Schuurmans; Tor Lattimore; Clare Lyle (NeurIPS, 2019)

Shaping the Narrative Arc: An Information-Theoretic Approach to Collaborative Dialogue. - Kory W. Mathewson; Pablo Samuel Castro; Colin Cherry; George F. Foster; Marc G. Bellemare (CoRR, 2019)

Inverse Reinforcement Learning with Multiple Ranked Experts. - Pablo Samuel Castro; Shijian Li; Daqing Zhang 001 (CoRR, 2019)

Scalable Methods for Computing State Similarity in Deterministic Markov Decision Processes. - Pablo Samuel Castro (AAAI, 2020)

Shaping the Narrative Arc: Information-Theoretic Collaborative DialoguePaper type: Technical Paper. - Kory Wallace Mathewson; Pablo Samuel Castro; Colin Cherry; George F. Foster; Marc G. Bellemare (ICCC, 2020)

Rigging the Lottery: Making All Tickets Winners. - Utku Evci; Trevor Gale; Jacob Menick; Pablo Samuel Castro; Erich Elsen (ICML, 2020)

GANterpretations. - Pablo Samuel Castro (CoRR, 2020)

Autonomous navigation of stratospheric balloons using reinforcement learning. - Marc G. Bellemare; Salvatore Candido; Pablo Samuel Castro; Jun Gong; Marlos C. Machado; Subhodeep Moitra; Sameera S. Ponda; Ziyu Wang (Nat., 2020)

Metrics and Continuity in Reinforcement Learning. - Charline Le Lan; Marc G. Bellemare; Pablo Samuel Castro (AAAI, 2021)

Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning. - Rishabh Agarwal; Marlos C. Machado; Pablo Samuel Castro; Marc G. Bellemare (ICLR, 2021)

Revisiting Rainbow: Promoting more insightful and inclusive deep reinforcement learning research. - Johan S. Obando-Ceron; Pablo Samuel Castro (ICML, 2021)

Deep Reinforcement Learning at the Edge of the Statistical Precipice. - Rishabh Agarwal; Max Schwarzer; Pablo Samuel Castro; Aaron C. Courville; Marc G. Bellemare (NeurIPS, 2021)

MICo: Improved representations via sampling-based state similarity for Markov decision processes. - Pablo Samuel Castro; Tyler Kastner; Prakash Panangaden; Mark Rowland 001 (NeurIPS, 2021)

The Difficulty of Passive Learning in Deep Reinforcement Learning. - Georg Ostrovski; Pablo Samuel Castro; Will Dabney (NeurIPS, 2021)

MICo: Learning improved representations via sampling-based state similarity for Markov decision processes. - Pablo Samuel Castro; Tyler Kastner; Prakash Panangaden; Mark Rowland 001 (CoRR, 2021)

A functional mirror ascent view of policy gradient methods with function approximation. - Sharan Vaswani; Olivier Bachem; Simone Totaro; Robert Mueller; Matthieu Geist; Marlos C. Machado; Pablo Samuel Castro; Nicolas Le Roux (CoRR, 2021)

Losses, Dissonances, and Distortions. - Pablo Samuel Castro (CoRR, 2021)

A general class of surrogate functions for stable and efficient reinforcement learning. - Sharan Vaswani; Olivier Bachem; Simone Totaro; Robert Müller; Shivam Garg; Matthieu Geist; Marlos C. Machado; Pablo Samuel Castro; Nicolas Le Roux (AISTATS, 2022)

The State of Sparse Training in Deep Reinforcement Learning. - Laura Graesser; Utku Evci; Erich Elsen; Pablo Samuel Castro (ICML, 2022)

Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress. - Rishabh Agarwal; Max Schwarzer; Pablo Samuel Castro; Aaron C. Courville; Marc G. Bellemare (NeurIPS, 2022)

Beyond Tabula Rasa: Reincarnating Reinforcement Learning. - Rishabh Agarwal; Max Schwarzer; Pablo Samuel Castro; Aaron C. Courville; Marc G. Bellemare (CoRR, 2022)

Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks. - Jesse Farebrother; Joshua Greaves; Rishabh Agarwal; Charline Le Lan; Ross Goroshin; Pablo Samuel Castro; Marc G. Bellemare (ICLR, 2023)

The Small Batch Size Anomaly in Multistep Deep Reinforcement Learning. - Johan S. Obando-Ceron; Marc G. Bellemare; Pablo Samuel Castro (Tiny Papers @ ICLR, 2023)

Bigger, Better, Faster: Human-level Atari with human-level efficiency. - Max Schwarzer; Johan S. Obando-Ceron; Aaron C. Courville; Marc G. Bellemare; Rishabh Agarwal; Pablo Samuel Castro (ICML, 2023)

The Dormant Neuron Phenomenon in Deep Reinforcement Learning. - Ghada Sokar; Rishabh Agarwal; Pablo Samuel Castro; Utku Evci (ICML, 2023)

Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks. - Maxime Chevalier-Boisvert; Bolun Dai; Mark Towers; Rodrigo Perez-Vicente; Lucas Willems; Salem Lahlou; Suman Pal; Pablo Samuel Castro; Jordan K. Terry (NeurIPS, 2023)

Small batch deep reinforcement learning. - Johan S. Obando-Ceron; Marc G. Bellemare; Pablo Samuel Castro (NeurIPS, 2023)

Offline Reinforcement Learning with On-Policy Q-Function Regularization. - Laixi Shi; Robert Dadashi; Yuejie Chi; Pablo Samuel Castro; Matthieu Geist (ECML/PKDD (4), 2023)

Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks. - Maxime Chevalier-Boisvert; Bolun Dai; Mark Towers; Rodrigo de Lazcano; Lucas Willems; Salem Lahlou; Suman Pal; Pablo Samuel Castro; Jordan K. Terry (CoRR, 2023)

Learning and Controlling Silicon Dopant Transitions in Graphene using Scanning Transmission Electron Microscopy. - Max Schwarzer; Jesse Farebrother; Joshua Greaves; Ekin Dogus Cubuk; Rishabh Agarwal; Aaron C. Courville; Marc G. Bellemare; Sergei V. Kalinin; Igor Mordatch; Pablo Samuel Castro; Kevin M. Roccapriore (CoRR, 2023)

A Kernel Perspective on Behavioural Metrics for Markov Decision Processes. - Pablo Samuel Castro; Tyler Kastner; Prakash Panangaden; Mark Rowland 001 (Trans. Mach. Learn. Res., 2023)

Jaxpruner: A Concise Library for Sparsity Research. - Joo Hyung Lee; Wonpyo Park; Nicole Mitchell; Jonathan Pilault; Johan Samir Obando Ceron; Han-Byul Kim; Namhoon Lee; Elias Frantar; Yun Long; Amir Yazdanbakhsh; Woohyun Han; Shivani Agrawal; et al. (CPAL, 2024)

Stop Regressing: Training Value Functions via Classification for Scalable Deep RL. - Jesse Farebrother; Jordi Orbay; Quan Vuong; Adrien Ali Taïga; Yevgen Chebotar; Ted Xiao; Alex Irpan; Sergey Levine; Pablo Samuel Castro; Aleksandra Faust; Aviral Kumar; Rishabh Agarwal (ICML, 2024)

In value-based deep reinforcement learning, a pruned network is a good network. - Johan S. Obando-Ceron; Aaron C. Courville; Pablo Samuel Castro (ICML, 2024)

Mixtures of Experts Unlock Parameter Scaling for Deep RL. - Johan S. Obando-Ceron; Ghada Sokar; Timon Willi; Clare Lyle; Jesse Farebrother; Jakob Nicolaus Foerster; Gintare Karolina Dziugaite; Doina Precup; Pablo Samuel Castro (ICML, 2024)

Adaptive Accompaniment with ReaLchords. - Yusong Wu; Tim Cooijmans; Kyle Kastner; Adam Roberts; Ian Simon; Alexander Scarlatos; Chris Donahue; Cassie Tarakajian; Shayegan Omidshafiei; Aaron C. Courville; Pablo Samuel Castro; Natasha Jaques; et al. (ICML, 2024)

CALE: Continuous Arcade Learning Environment. - Jesse Farebrother; Pablo Samuel Castro (NeurIPS, 2024)

On the consistency of hyper-parameter selection in value-based deep reinforcement learning. - Johan S. Obando-Ceron; João Guilherme Madeira Araújo; Aaron C. Courville; Pablo Samuel Castro (RLJ, 2024)

Mixture of Experts in a Mixture of RL settings. - Timon Willi; Johan S. Obando-Ceron; Jakob Nicolaus Foerster; Gintare Karolina Dziugaite; Pablo Samuel Castro (RLJ, 2024)

In deep reinforcement learning, a pruned network is a good network. - Johan S. Obando-Ceron; Aaron C. Courville; Pablo Samuel Castro (CoRR, 2024)

NAVIX: Scaling MiniGrid Environments with JAX. - Eduardo Pignatelli; Jarek Liesen; Robert Tjarko Lange; Chris Lu 001; Pablo Samuel Castro; Laura Toni (CoRR, 2024)

A density estimation perspective on learning from pairwise human preferences. - Vincent Dumoulin; Daniel D. Johnson 001; Pablo Samuel Castro; Hugo Larochelle; Yann N. Dauphin (Trans. Mach. Learn. Res., 2024)

Studying the Interplay Between the Actor and Critic Representations in Reinforcement Learning. - Samuel Garcin; Trevor McInroe; Pablo Samuel Castro; Christopher G. Lucas; David Abel; Prakash Panangaden; Stefano V. Albrecht (ICLR, 2025)

Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RL. - Ghada Sokar; Johan S. Obando-Ceron; Aaron C. Courville; Hugo Larochelle; Pablo Samuel Castro (ICLR, 2025)

A Survey of State Representation Learning for Deep Reinforcement Learning. - Ayoub Echchahed; Pablo Samuel Castro (Trans. Mach. Learn. Res., 2025)

Multi-Task Reinforcement Learning Enables Parameter Scaling. - Reginald McLean; Evangelos Chatzaroulas; Jordan K. Terry; Isaac Woungang; Nariman Farsad; Pablo Samuel Castro (CoRR, 2025)

Estimating Policy Functions in Payment Systems Using Reinforcement Learning. - Pablo Samuel Castro; Ajit Desai; Han Du; Rodney Garratt; Francisco Rivadeneyra (ACM Trans. Economics and Comput., 2025)

Meta-World+: An Improved, Standardized, RL Benchmark. - Reginald McLean; Evangelos Chatzaroulas; Luc McCutcheon; Frank Röder; Tianhe Yu; Zhanpeng He; K. R. Zentner; Ryan Julian; Jordan K. Terry; Isaac Woungang; Nariman Farsad; Pablo Samuel Castro (CoRR, 2025)

Mind the GAP! The Challenges of Scale in Pixel-based Deep Reinforcement Learning. - Ghada Sokar; Pablo Samuel Castro (CoRR, 2025)

Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning. - Jiashun Liu; Zihao Wu; Johan S. Obando-Ceron; Pablo Samuel Castro; Aaron C. Courville; Ling Pan (CoRR, 2025)

Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churn. - Hongyao Tang; Johan S. Obando-Ceron; Pablo Samuel Castro; Aaron C. Courville; Glen Berseth (CoRR, 2025)

Optimistic critics can empower small actors. - Olya Mastikhina; Dhruv Sreenivas; Pablo Samuel Castro (CoRR, 2025)

Continual Learning in Vision-Language Models via Aligned Model Merging. - Ghada Sokar; Gintare Karolina Dziugaite; Anurag Arnab; Ahmet Iscen; Pablo Samuel Castro; Cordelia Schmid (CoRR, 2025)

The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networks. - Walter Mayor; Johan S. Obando-Ceron; Aaron C. Courville; Pablo Samuel Castro (CoRR, 2025)

The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learning. - Jiashun Liu; Johan S. Obando-Ceron; Pablo Samuel Castro; Aaron C. Courville; Ling Pan (CoRR, 2025)

Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning. - Roger Creus Castanyer; Johan S. Obando-Ceron; Lu Li; Pierre-Luc Bacon; Glen Berseth; Aaron C. Courville; Pablo Samuel Castro (CoRR, 2025)