The Lattice Harp: A New Hybrid Instrument And Controller. - Colin Raffel; Nick Kruge; Diane Douglas; Edgar Berdahl; Wendy Ju (ICMC, 2010)

Estimating timing and channel distortion across related signals. - Colin Raffel; Daniel P. W. Ellis (ICASSP, 2014)

MIR_EVAL: A Transparent Implementation of Common MIR Metrics. - Colin Raffel; Brian McFee; Eric J. Humphrey; Justin Salamon; Oriol Nieto; Dawen Liang; Daniel P. W. Ellis (ISMIR, 2014)

Large-Scale Content-Based Matching of MIDI and Audio Files. - Colin Raffel; Daniel P. W. Ellis (ISMIR, 2015)

librosa: Audio and Music Signal Analysis in Python. - Brian McFee; Colin Raffel; Dawen Liang; Daniel P. W. Ellis; Matt McVicar; Eric Battenberg; Oriol Nieto (SciPy, 2015)

Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems. - Colin Raffel; Daniel P. W. Ellis (CoRR, 2015)

Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games. - Nikolai Yakovenko; Liangliang Cao; Colin Raffel; James Fan (CoRR, 2015)

Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games Using Convolutional Networks. - Nikolai Yakovenko; Liangliang Cao; Colin Raffel; James Fan (AAAI, 2016)

Optimizing DTW-based audio-to-MIDI alignment and matching. - Colin Raffel; Daniel P. W. Ellis (ICASSP, 2016)

Pruning subsequence search with attention-based embedding. - Colin Raffel; Daniel P. W. Ellis (ICASSP, 2016)

Extracting Ground-Truth Information from MIDI Files: A MIDIfesto. - Colin Raffel; Daniel P. W. Ellis (ISMIR, 2016)

Theano: A Python framework for fast computation of mathematical expressions. - Rami Al-Rfou; Guillaume Alain; Amjad Almahairi; Christof Angermüller; Dzmitry Bahdanau; Nicolas Ballas; Frédéric Bastien; Justin Bayer; Anatoly Belikov; Alexander Belopolsky; Yoshua Bengio; Arnaud Bergeron; et al. (CoRR, 2016)

Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching. - Colin Raffel (2016)

Explaining the Learning Dynamics of Direct Feedback Alignment. - Justin Gilmer; Colin Raffel; Samuel S. Schoenholz; Maithra Raghu; Jascha Sohl-Dickstein (ICLR (Workshop), 2017)

Training a Subsampling Mechanism in Expectation. - Colin Raffel; Dieterich Lawson (ICLR (Workshop), 2017)

Online and Linear-Time Attention by Enforcing Monotonic Alignments. - Colin Raffel; Minh-Thang Luong; Peter J. Liu; Ron J. Weiss; Douglas Eck (ICML, 2017)

Learning Hard Alignments with Variational Inference. - Dieterich Lawson; Chung-Cheng Chiu; George Tucker; Colin Raffel; Kevin Swersky; Navdeep Jaitly (ICASSP, 2018)

Thermometer Encoding: One Hot Way To Resist Adversarial Examples. - Jacob Buckman; Aurko Roy; Colin Raffel; Ian J. Goodfellow (ICLR (Poster), 2018)

Monotonic Chunkwise Attention. - Chung-Cheng Chiu; Colin Raffel (ICLR (Poster), 2018)

Realistic Evaluation of Semi-Supervised Learning Algorithms. - Avital Oliver; Augustus Odena; Colin Raffel; Ekin D. Cubuk; Ian J. Goodfellow (ICLR (Workshop), 2018)

Is Generator Conditioning Causally Related to GAN Performance? - Augustus Odena; Jacob Buckman; Catherine Olsson; Tom B. Brown; Christopher Olah; Colin Raffel; Ian J. Goodfellow (ICML, 2018)

A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music. - Adam Roberts; Jesse H. Engel; Colin Raffel; Curtis Hawthorne; Douglas Eck (ICML, 2018)

Onsets and Frames: Dual-Objective Piano Transcription. - Curtis Hawthorne; Erich Elsen; Jialin Song; Adam Roberts; Ian Simon; Colin Raffel; Jesse H. Engel; Sageev Oore; Douglas Eck (ISMIR, 2018)

Realistic Evaluation of Deep Semi-Supervised Learning Algorithms. - Avital Oliver; Augustus Odena; Colin Raffel; Ekin Dogus Cubuk; Ian J. Goodfellow (NeurIPS, 2018)

Learning a Latent Space of Multitrack Measures. - Ian Simon; Adam Roberts; Colin Raffel; Jesse H. Engel; Curtis Hawthorne; Douglas Eck (CoRR, 2018)

Hickle: A HDF5-based python pickle replacement. - Danny C. Price; Sebastien Celles; Pieter T. Eendebak; Michael M. McKerns; Eben M. Olson; Colin Raffel; Bairen Yi (J. Open Source Softw., 2018)

Monotonic Infinite Lookback Attention for Simultaneous Machine Translation. - Naveen Arivazhagan; Colin Cherry; Wolfgang Macherey; Chung-Cheng Chiu; Semih Yavuz; Ruoming Pang; Wei Li 133; Colin Raffel (ACL (1), 2019)

Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer. - David Berthelot; Colin Raffel; Aurko Roy; Ian J. Goodfellow (ICLR (Poster), 2019)

Towards GAN Benchmarks Which Require Generalization. - Ishaan Gulrajani; Colin Raffel; Luke Metz (ICLR (Poster), 2019)

Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition. - Yao Qin 001; Nicholas Carlini; Garrison W. Cottrell; Ian J. Goodfellow; Colin Raffel (ICML, 2019)

MixMatch: A Holistic Approach to Semi-Supervised Learning. - David Berthelot; Nicholas Carlini; Ian J. Goodfellow; Nicolas Papernot; Avital Oliver; Colin Raffel (NeurIPS, 2019)

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling. - Jonathan Shen; Patrick Nguyen; Yonghui Wu; Zhifeng Chen; Mia Xu Chen; Ye Jia; Anjuli Kannan; Tara N. Sainath; Yuan Cao 007; Chung-Cheng Chiu; Yanzhang He; Jan Chorowski; et al. (CoRR, 2019)

ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring. - David Berthelot; Nicholas Carlini; Ekin D. Cubuk; Alex Kurakin; Kihyuk Sohn; Han Zhang 010; Colin Raffel (CoRR, 2019)

Learning-based Memory Allocation for C++ Server Workloads. - Martin Maas 001; David G. Andersen; Michael Isard; Mohammad Mahdi Javanmard; Kathryn S. McKinley; Colin Raffel (ASPLOS, 2020)

How Much Knowledge Can You Pack Into the Parameters of a Language Model? - Adam Roberts; Colin Raffel; Noam Shazeer (EMNLP (1), 2020)

ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring. - David Berthelot; Nicholas Carlini; Ekin D. Cubuk; Alex Kurakin; Kihyuk Sohn; Han Zhang 010; Colin Raffel (ICLR, 2020)

Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions. - Yao Qin 001; Nicholas Frosst; Sara Sabour; Colin Raffel; Garrison W. Cottrell; Geoffrey E. Hinton (ICLR, 2020)

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned. - Sewon Min; Jordan L. Boyd-Graber; Chris Alberti; Danqi Chen 001; Eunsol Choi; Michael Collins 001; Kelvin Guu; Hannaneh Hajishirzi; Kenton Lee; Jennimaria Palomaki; Colin Raffel; Adam Roberts; et al. (NeurIPS (Competition and Demos), 2020)

Top-k Training of GANs: Improving GAN Performance by Throwing Away Bad Samples. - Samarth Sinha; Zhengli Zhao; Anirudh Goyal; Colin Raffel; Augustus Odena (NeurIPS, 2020)

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence. - Kihyuk Sohn; David Berthelot; Nicholas Carlini; Zizhao Zhang; Han Zhang 010; Colin Raffel; Ekin Dogus Cubuk; Alexey Kurakin; Chun-Liang Li (NeurIPS, 2020)

Top-K Training of GANs: Improving Generators by Making Critics Less Critical. - Samarth Sinha; Anirudh Goyal; Colin Raffel; Augustus Odena (CoRR, 2020)

Deflecting Adversarial Attacks. - Yao Qin 001; Nicholas Frosst; Colin Raffel; Garrison W. Cottrell; Geoffrey E. Hinton (CoRR, 2020)

WT5?! Training Text-to-Text Models to Explain their Predictions. - Sharan Narang; Colin Raffel; Katherine Lee; Adam Roberts; Noah Fiedel; Karishma Malkan (CoRR, 2020)

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. - Colin Raffel; Noam Shazeer; Adam Roberts; Katherine Lee; Sharan Narang; Michael Matena; Yanqi Zhou; Wei Li 133; Peter J. Liu (J. Mach. Learn. Res., 2020)

Do Transformer Modifications Transfer Across Implementations and Applications? - Sharan Narang; Hyung Won Chung; Yi Tay; Liam Fedus; Thibault Févry; Michael Matena; Karishma Malkan; Noah Fiedel; Noam Shazeer; Zhenzhong Lan; Yanqi Zhou; Wei Li 133; et al. (EMNLP (1), 2021)

Improving and Simplifying Pattern Exploiting Training. - Derek Tam; Rakesh R. Menon; Mohit Bansal; Shashank Srivastava; Colin Raffel (EMNLP (1), 2021)

Robust and Generalizable Visual Representation Learning via Random Convolutions. - Zhenlin Xu; Deyi Liu; Junlin Yang; Colin Raffel; Marc Niethammer (ICLR, 2021)

On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition. - Ching-Yuan Bai; Hsuan-Tien Lin; Colin Raffel; Wendy Chi-wen Kan (KDD, 2021)

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. - Linting Xue; Noah Constant; Adam Roberts; Mihir Kale; Rami Al-Rfou; Aditya Siddhant; Aditya Barua; Colin Raffel (NAACL-HLT, 2021)

Training Neural Networks with Fixed Sparse Masks. - Yi-Lin Sung; Varun Nair; Colin Raffel (NeurIPS, 2021)

Extracting Training Data from Large Language Models. - Nicholas Carlini; Florian Tramèr; Eric Wallace; Matthew Jagielski; Ariel Herbert-Voss; Katherine Lee; Adam Roberts; Tom B. Brown; Dawn Song; Úlfar Erlingsson; Alina Oprea; Colin Raffel (USENIX Security Symposium, 2021)

Multitask Prompted Training Enables Zero-Shot Task Generalization. - Victor Sanh; Albert Webson; Colin Raffel; Stephen H. Bach; Lintang Sutawika; Zaid Alyafeai; Antoine Chaffin; Arnaud Stiegler; Teven Le Scao; Arun Raja; Manan Dey; M. Saiful Bari; et al. (CoRR, 2021)

Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP. - Sabrina J. Mielke; Zaid Alyafeai; Elizabeth Salesky; Colin Raffel; Manan Dey; Matthias Gallé; Arun Raja; Chenglei Si; Wilson Y. Lee; Benoît Sagot; Samson Tan (CoRR, 2021)

PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts. - Stephen H. Bach; Victor Sanh; Zheng Xin Yong; Albert Webson; Colin Raffel; Nihal V. Nayak; Abheesht Sharma; Taewoon Kim 002; M. Saiful Bari; Thibault Févry; Zaid Alyafeai; Manan Dey; et al. (ACL (demo), 2022)

Learning with Limited Text Data. - Diyi Yang; Ankur P. Parikh; Colin Raffel (ACL (tutorial), 2022)

What Language Model to Train if You Have One Million GPU Hours? - Teven Le Scao; Thomas Wang; Daniel Hesslow; Lucile Saulnier; Stas Bekman; M. Saiful Bari; Stella Biderman; Hady Elsahar; Niklas Muennighoff; Jason Phang; Ofir Press; Colin Raffel; et al. (EMNLP (Findings), 2022)

Multitask Prompted Training Enables Zero-Shot Task Generalization. - Victor Sanh; Albert Webson; Colin Raffel; Stephen H. Bach; Lintang Sutawika; Zaid Alyafeai; Antoine Chaffin; Arnaud Stiegler; Arun Raja; Manan Dey; M Saiful Bari; Canwen Xu; et al. (ICLR, 2022)

Deduplicating Training Data Mitigates Privacy Risks in Language Models. - Nikhil Kandpal; Eric Wallace; Colin Raffel (ICML, 2022)

What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization? - Thomas Wang; Adam Roberts; Daniel Hesslow; Teven Le Scao; Hyung Won Chung; Iz Beltagy; Julien Launay; Colin Raffel (ICML, 2022)

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning. - Haokun Liu; Derek Tam; Mohammed Muqeeth; Jay Mohta; Tenghao Huang; Mohit Bansal; Colin Raffel (NeurIPS, 2022)

Merging Models with Fisher-Weighted Averaging. - Michael Matena; Colin Raffel (NeurIPS, 2022)

A Combinatorial Perspective on the Optimization of Shallow ReLU Networks. - Michael Matena; Colin Raffel (NeurIPS, 2022)

Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Language. - Zhenlin Xu; Marc Niethammer; Colin Raffel (NeurIPS, 2022)

PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts. - Stephen H. Bach; Victor Sanh; Zheng Xin Yong; Albert Webson; Colin Raffel; Nihal V. Nayak; Abheesht Sharma; Taewoon Kim 002; M. Saiful Bari; Thibault Févry; Zaid Alyafeai; Manan Dey; et al. (CoRR, 2022)

Scaling Up Models and Data with t5x and seqio. - Adam Roberts; Hyung Won Chung; Anselm Levskaya; Gaurav Mishra; James Bradbury; Daniel Andor; Sharan Narang; Brian Lester; Colin Gaffney; Afroz Mohiuddin; Curtis Hawthorne; Aitor Lewkowycz; et al. (CoRR, 2022)

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? - Thomas Wang; Adam Roberts; Daniel Hesslow; Teven Le Scao; Hyung Won Chung; Iz Beltagy; Julien Launay; Colin Raffel (CoRR, 2022)

Efficient Methods for Natural Language Processing: A Survey. - Marcos V. Treviso; Tianchu Ji; Ji-Ung Lee; Betty van Aken; Qingqing Cao; Manuel R. Ciosici; Michael Hassid; Kenneth Heafield; Sara Hooker; Pedro Henrique Martins; André F. T. Martins; Peter A. Milder; et al. (CoRR, 2022)

Petals: Collaborative Inference and Fine-tuning of Large Models. - Alexander Borzunov; Dmitry Baranchuk; Tim Dettmers; Max Ryabinin; Younes Belkada; Artem Chumachenko; Pavel Samygin; Colin Raffel (CoRR, 2022)

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. - Teven Le Scao; Angela Fan; Christopher Akiki; Ellie Pavlick; Suzana Ilic; Daniel Hesslow; Roman Castagné; Alexandra Sasha Luccioni; François Yvon; Matthias Gallé; Jonathan Tow; Alexander M. Rush; et al. (CoRR, 2022)

Evaluating the Factual Consistency of Large Language Models Through Summarization. - Derek Tam; Anisha Mascarenhas; Shiyue Zhang 001; Sarah Kwan; Mohit Bansal; Colin Raffel (CoRR, 2022)

ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning. - Shachar Don-Yehiya; Elad Venezian; Colin Raffel; Noam Slonim; Yoav Katz; Leshem Choshen (CoRR, 2022)

ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models. - Linting Xue; Aditya Barua; Noah Constant; Rami Al-Rfou; Sharan Narang; Mihir Kale; Adam Roberts; Colin Raffel (Trans. Assoc. Comput. Linguistics, 2022)

Emergent Abilities of Large Language Models. - Jason Wei; Yi Tay; Rishi Bommasani; Colin Raffel; Barret Zoph; Sebastian Borgeaud; Dani Yogatama; Maarten Bosma; Denny Zhou; Donald Metzler; Ed H. Chi; Tatsunori Hashimoto; et al. (Trans. Mach. Learn. Res., 2022)

Petals: Collaborative Inference and Fine-tuning of Large Models. - Alexander Borzunov; Dmitry Baranchuk; Tim Dettmers; Maksim Riabinin; Younes Belkada; Artem Chumachenko; Pavel Samygin; Colin Raffel (ACL (demo), 2023)

ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning. - Shachar Don-Yehiya; Elad Venezian; Colin Raffel; Noam Slonim; Leshem Choshen (ACL (1), 2023)

Crosslingual Generalization through Multitask Finetuning. - Niklas Muennighoff; Thomas Wang; Lintang Sutawika; Adam Roberts; Stella Biderman; Teven Le Scao; M. Saiful Bari; Sheng Shen 001; Zheng Xin Yong; Hailey Schoelkopf; Xiangru Tang; Dragomir Radev; et al. (ACL (1), 2023)

Evaluating the Factual Consistency of Large Language Models Through News Summarization. - Derek Tam; Anisha Mascarenhas; Shiyue Zhang 001; Sarah Kwan; Mohit Bansal; Colin Raffel (ACL (Findings), 2023)

Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model. - Haikang Deng; Colin Raffel (EMNLP, 2023)

Knowledge is a Region in Weight Space for Fine-tuned Language Models. - Almog Gueta; Elad Venezian; Colin Raffel; Noam Slonim; Yoav Katz; Leshem Choshen (EMNLP (Findings), 2023)

Bidirectional Language Models Are Also Few-shot Learners. - Ajay Patel; Bryan Li; Mohammad Sadegh Rasooli; Noah Constant; Colin Raffel; Chris Callison-Burch (ICLR, 2023)

Large Language Models Struggle to Learn Long-Tail Knowledge. - Nikhil Kandpal; Haikang Deng; Adam Roberts; Eric Wallace; Colin Raffel (ICML, 2023)

Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models. - Nikhil Kandpal; Brian Lester; Mohammed Muqeeth; Anisha Mascarenhas; Monty Evans; Vishal Baskaran; Tenghao Huang; Haokun Liu; Colin Raffel (ICML, 2023)

Improving Few-Shot Generalization by Exploring and Exploiting Auxiliary Data. - Alon Albalak; Colin A. Raffel; William Yang Wang (NeurIPS, 2023)

Distributed Inference and Fine-tuning of Large Language Models Over The Internet. - Alexander Borzunov; Max Ryabinin; Artem Chumachenko; Dmitry Baranchuk; Tim Dettmers; Younes Belkada; Pavel Samygin; Colin A. Raffel (NeurIPS, 2023)

Scaling Data-Constrained Language Models. - Niklas Muennighoff; Alexander M. Rush; Boaz Barak; Teven Le Scao; Nouamane Tazi; Aleksandra Piktus; Sampo Pyysalo; Thomas Wolf 008; Colin A. Raffel (NeurIPS, 2023)

TIES-Merging: Resolving Interference When Merging Models. - Prateek Yadav; Derek Tam; Leshem Choshen; Colin A. Raffel; Mohit Bansal (NeurIPS, 2023)

Resolving Interference When Merging Models. - Prateek Yadav; Derek Tam; Leshem Choshen; Colin Raffel; Mohit Bansal (CoRR, 2023)

NPEFF: Non-Negative Per-Example Fisher Factorization. - Michael Matena; Colin Raffel (CoRR, 2023)

Efficient Online Data Mixing For Language Model Pre-Training. - Alon Albalak; Liangming Pan; Colin Raffel; William Yang Wang (CoRR, 2023)

Merging by Matching Models in Task Subspaces. - Derek Tam; Mohit Bansal; Colin Raffel (CoRR, 2023)

Scaling Up Models and Data with t5x and seqio. - Adam Roberts; Hyung Won Chung; Gaurav Mishra; Anselm Levskaya; James Bradbury; Daniel Andor; Sharan Narang; Brian Lester; Colin Gaffney; Afroz Mohiuddin; Curtis Hawthorne; Aitor Lewkowycz; et al. (J. Mach. Learn. Res., 2023)

An Empirical Survey of Data Augmentation for Limited Data Learning in NLP. - Jiaao Chen; Derek Tam; Colin Raffel; Mohit Bansal; Diyi Yang (Trans. Assoc. Comput. Linguistics, 2023)

Efficient Methods for Natural Language Processing: A Survey. - Marcos V. Treviso; Ji-Ung Lee; Tianchu Ji; Betty van Aken; Qingqing Cao; Manuel R. Ciosici; Michael Hassid; Kenneth Heafield; Sara Hooker; Colin Raffel; Pedro Henrique Martins; André F. T. Martins; et al. (Trans. Assoc. Comput. Linguistics, 2023)

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. - Aarohi Srivastava; Abhinav Rastogi; Abhishek Rao; Abu Awal Md Shoeb; Abubakar Abid; Adam Fisch; Adam R. Brown; Adam Santoro; Aditya Gupta; Adrià Garriga-Alonso; Agnieszka Kluska; Aitor Lewkowycz; et al. (Trans. Mach. Learn. Res., 2023)

Building Machine Learning Models Like Open Source Software. - Colin Raffel (Commun. ACM, 2023)

DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows. - Ajay Patel; Colin Raffel; Chris Callison-Burch (ACL (1), 2024)

Learning to Route Among Specialized Experts for Zero-Shot Generalization. - Mohammed Muqeeth; Haokun Liu; Yufan Liu; Colin Raffel (ICML, 2024)

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale. - Guilherme Penedo; Hynek Kydlícek; Loubna Ben Allal; Anton Lozhkov; Margaret Mitchell; Colin A. Raffel; Leandro von Werra; Thomas Wolf 008 (NeurIPS, 2024)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models. - Bowen Pan; Yikang Shen; Haokun Liu; Mayank Mishra; Gaoyuan Zhang; Aude Oliva; Colin Raffel; Rameswar Panda (CoRR, 2024)

Realistic Evaluation of Model Merging for Compositional Generalization. - Derek Tam; Yash Kant; Brian Lester; Igor Gilitschenski; Colin Raffel (CoRR, 2024)

A Survey on Data Selection for Language Models. - Alon Albalak; Yanai Elazar; Sang Michael Xie; Shayne Longpre; Nathan Lambert 001; Xinyi Wang 003; Niklas Muennighoff; Bairu Hou; Liangming Pan; Haewon Jeong; Colin Raffel; Shiyu Chang; et al. (Trans. Mach. Learn. Res., 2024)

Soft Merging of Experts with Adaptive Routing. - Mohammed Muqeeth; Haokun Liu; Colin Raffel (Trans. Mach. Learn. Res., 2024)

Merging by Matching Models in Task Parameter Subspaces. - Derek Tam; Mohit Bansal; Colin Raffel (Trans. Mach. Learn. Res., 2024)

Combining Machine Learning and Lifetime-Based Resource Management for Memory Allocation and Beyond. - Martin Maas 001; David G. Andersen; Michael Isard; Mohammad Mahdi Javanmard; Kathryn S. McKinley; Colin Raffel (Commun. ACM, 2024)

AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attribution. - Fengyuan Liu; Nikhil Kandpal; Colin Raffel (ICLR, 2025)

ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization. - Prateek Yadav; Leshem Choshen; Colin Raffel; Mohit Bansal (Trans. Mach. Learn. Res., 2025)

A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning. - Prateek Yadav; Colin Raffel; Mohammed Muqeeth; Lucas Caccia; Haokun Liu; Tianlong Chen 001; Mohit Bansal; Leshem Choshen; Alessandro Sordoni (Trans. Mach. Learn. Res., 2025)

SmolLM2: When Smol Goes Big - Data-Centric Training of a Small Language Model. - Loubna Ben Allal; Anton Lozhkov; Elie Bakouch; Gabriel Martín Blázquez; Guilherme Penedo; Lewis Tunstall; Andrés Marafioti; Hynek Kydlícek; Agustín Piqueres Lajarín; Vaibhav Srivastav; Joshua Lochner; Caleb Fahlgren; et al. (CoRR, 2025)

Position: The Most Expensive Part of an LLM should be its Training Data. - Nikhil Kandpal; Colin Raffel (CoRR, 2025)

Enhancing Training Data Attribution with Representational Optimization. - Weiwei Sun; Haokun Liu; Nikhil Kandpal; Colin Raffel; Yiming Yang (CoRR, 2025)

The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text. - Nikhil Kandpal; Brian Lester; Colin Raffel; Sebastian Majstorovic; Stella Biderman; Baber Abbasi; Luca Soldaini; Enrico Shippole; A. Feder Cooper; Aviya Skowron; John Kirchenbauer; Shayne Longpre; et al. (CoRR, 2025)

The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions. - Devin Kwok; Gül Sena Altintas; Colin Raffel; David Rolnick (CoRR, 2025)

FineWeb2: One Pipeline to Scale Them All - Adapting Pre-Training Data Processing to Every Language. - Guilherme Penedo; Hynek Kydlícek; Vinko Sabolcec; Bettina Messmer; Negar Foroutan; Amir Hossein Kargaran; Colin Raffel; Martin Jaggi; Leandro von Werra; Thomas Wolf 008 (CoRR, 2025)

Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator. - YuXin Li; Felix Dangel; Derek Tam; Colin Raffel (CoRR, 2025)