The (Un)reliability of saliency methods. - Pieter-Jan Kindermans; Sara Hooker; Julius Adebayo; Maximilian Alber; Kristof T. Schütt; Sven Dähne; Dumitru Erhan; Been Kim (CoRR, 2017)

Evaluating Feature Importance Estimates. - Sara Hooker; Dumitru Erhan; Pieter-Jan Kindermans; Been Kim (CoRR, 2018)

A Benchmark for Interpretability Methods in Deep Neural Networks. - Sara Hooker; Dumitru Erhan; Pieter-Jan Kindermans; Been Kim (NeurIPS, 2019)

The State of Sparsity in Deep Neural Networks. - Trevor Gale; Erich Elsen; Sara Hooker (CoRR, 2019)

Selective Brain Damage: Measuring the Disparate Impact of Model Pruning. - Sara Hooker; Aaron C. Courville; Yann N. Dauphin; Andrea Frome (CoRR, 2019)

The (Un)reliability of Saliency Methods. - Pieter-Jan Kindermans; Sara Hooker; Julius Adebayo; Maximilian Alber; Kristof T. Schütt; Sven Dähne; Dumitru Erhan; Been Kim (Explainable AI, 2019)

Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims. - Miles Brundage; Shahar Avin; Jasmine Wang; Haydn Belfield; Gretchen Krueger; Gillian K. Hadfield; Heidy Khlaaf; Jingying Yang; Helen Toner; Ruth Fong; Tegan Maharaj; Pang Wei Koh; et al. (CoRR, 2020)

Estimating Example Difficulty using Variance of Gradients. - Chirag Agarwal; Sara Hooker (CoRR, 2020)

Characterising Bias in Compressed Models. - Sara Hooker; Nyalleng Moorosi; Gregory Clark; Samy Bengio; Emily Denton (CoRR, 2020)

The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation. - Orevaoghene Ahia; Julia Kreutzer; Sara Hooker (EMNLP (Findings), 2021)

The hardware lottery. - Sara Hooker (Commun. ACM, 2021)

Keep the Gradients Flowing: Using Gradient Flow to Study Sparse Network Optimization. - Kale-ab Abebe Tessera; Sara Hooker; Benjamin Rosman (CoRR, 2021)

When does loss-based prioritization fail? - Niel Teng Hu; Xinyu Hu; Rosanne Liu; Sara Hooker; Jason Yosinski (CoRR, 2021)

A Tale Of Two Long Tails. - Daniel D'souza; Zach Nussbaum; Chirag Agarwal; Sara Hooker (CoRR, 2021)

Moving beyond "algorithmic bias is a data problem". - Sara Hooker (Patterns, 2021)

Estimating Example Difficulty using Variance of Gradients. - Chirag Agarwal; Daniel D'souza; Sara Hooker (CVPR, 2022)

Intriguing Properties of Compression on Multilingual Models. - Kelechi Ogueji; Orevaoghene Ahia; Gbemileke Onilude; Sebastian Gehrmann; Sara Hooker; Julia Kreutzer (EMNLP, 2022)

Randomness in Neural Network Training: Characterizing the Impact of Tooling. - Donglin Zhuang; Xingyao Zhang; Shuaiwen Song; Sara Hooker (MLSys, 2022)

When less is more: Simplifying inputs aids neural network understanding. - Robin Tibor Schirrmeister; Rosanne Liu; Sara Hooker; Tonio Ball (CoRR, 2022)

Robust Distillation for Worst-class Performance. - Serena Lutong Wang; Harikrishna Narasimhan; Yichen Zhou; Sara Hooker; Michal Lukasik; Aditya Krishna Menon (CoRR, 2022)

Studying the impact of magnitude pruning on contrastive learning methods. - Francesco Corti; Rahim Entezari; Sara Hooker; Davide Bacciu; Olga Saukh (CoRR, 2022)

Efficient Methods for Natural Language Processing: A Survey. - Marcos V. Treviso; Tianchu Ji; Ji-Ung Lee; Betty van Aken; Qingqing Cao; Manuel R. Ciosici; Michael Hassid; Kenneth Heafield; Sara Hooker; Pedro Henrique Martins; André F. T. Martins; Peter A. Milder; et al. (CoRR, 2022)

Large language models are not zero-shot communicators. - Laura Ruis; Akbir Khan; Stella Biderman; Sara Hooker; Tim Rocktäschel; Edward Grefenstette (CoRR, 2022)

On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research. - Luiza Pozzobon; Beyza Ermis; Patrick Lewis 002; Sara Hooker (EMNLP, 2023)

Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models. - Luiza Pozzobon; Beyza Ermis; Patrick Lewis 002; Sara Hooker (EMNLP (Findings), 2023)

Locally Differentially Private Document Generation Using Zero Shot Prompting. - Saiteja Utpala; Sara Hooker; Pin-Yu Chen (EMNLP (Findings), 2023)

Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics. - Shoaib Ahmed Siddiqui; Nitarshan Rajkumar; Tegan Maharaj; David Krueger 001; Sara Hooker (ICLR, 2023)

Intriguing Properties of Quantization at Scale. - Arash Ahmadian; Saurabh Dash; Hongyu Chen 008; Bharat Venkitesh; Stephen Zhen Gou; Phil Blunsom; Ahmet Üstün; Sara Hooker (NeurIPS, 2023)

The Grand Illusion: The Myth of Software Portability and Implications for ML Progress. - Fraser Mince; Dzung Dinh; Jonas Kgomo; Neil Thompson; Sara Hooker (NeurIPS, 2023)

The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs. - Laura Ruis; Akbir Khan; Stella Biderman; Sara Hooker; Tim Rocktäschel; Edward Grefenstette (NeurIPS, 2023)

Robust distillation for worst-class performance: on the interplay between teacher and student objectives. - Serena Wang 001; Harikrishna Narasimhan; Yichen Zhou; Sara Hooker; Michal Lukasik; Aditya Krishna Menon (UAI, 2023)

FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensembling. - Wei-Yin Ko; Daniel D'souza; Karina Nguyen; Randall Balestriero; Sara Hooker (CoRR, 2023)

Intriguing Properties of Quantization at Scale. - Arash Ahmadian; Saurabh Dash; Hongyu Chen 008; Bharat Venkitesh; Stephen Gou; Phil Blunsom; Ahmet Üstün; Sara Hooker (CoRR, 2023)

Evaluating the Social Impact of Generative AI Systems in Systems and Society. - Irene Solaiman; Zeerak Talat; William Agnew; Lama Ahmad; Dylan K. Baker; Su Lin Blodgett; Hal Daumé III; Jesse Dodge; Ellie Evans; Sara Hooker; Yacine Jernite; Alexandra Sasha Luccioni; et al. (CoRR, 2023)

Frontier AI Regulation: Managing Emerging Risks to Public Safety. - Markus Anderljung; Joslyn Barnhart; Anton Korinek; Jade Leung; Cullen O'Keefe; Jess Whittlestone; Shahar Avin; Miles Brundage; Justin Bullock; Duncan Cass-Beggs; Ben Chang; Tantum Collins; et al. (CoRR, 2023)

When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale. - Max Marion; Ahmet Üstün; Luiza Pozzobon; Alex Wang; Marzieh Fadaee; Sara Hooker (CoRR, 2023)

Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation. - Meriem Boubdir; Edward Kim; Beyza Ermis; Marzieh Fadaee; Sara Hooker (CoRR, 2023)

The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI. - Shayne Longpre; Robert Mahari; Anthony Chen; Naana Obeng-Marnu; Damien Sileo; William Brannon; Niklas Muennighoff; Nathan Khazam; Jad Kabbara; Kartik Perisetla; Xinyi Wu; Enrico Shippole; et al. (CoRR, 2023)

Generalisable Agents for Neural Network Optimisation. - Kale-ab Abebe Tessera; Callum Rhys Tilbury; Sasha Abramowitz; Ruan de Kock; Omayma Mahjoub; Benjamin Rosman; Sara Hooker; Arnu Pretorius (CoRR, 2023)

Efficient Methods for Natural Language Processing: A Survey. - Marcos V. Treviso; Ji-Ung Lee; Tianchu Ji; Betty van Aken; Qingqing Cao; Manuel R. Ciosici; Michael Hassid; Kenneth Heafield; Sara Hooker; Colin Raffel; Pedro Henrique Martins; André F. T. Martins; et al. (Trans. Assoc. Comput. Linguistics, 2023)

Back to Basics: Revisiting REINFORCE-Style Optimization for Learning from Human Feedback in LLMs. - Arash Ahmadian; Chris Cremer; Matthias Gallé; Marzieh Fadaee; Julia Kreutzer; Olivier Pietquin; Ahmet Üstün; Sara Hooker (ACL (1), 2024)

Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning. - Everlyn Chimoto; Jay Gala; Orevaoghene Ahia; Julia Kreutzer; Bruce A. Bassett; Sara Hooker (ACL (Findings), 2024)

From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models. - Beyza Ermis; Luiza Pozzobon; Sara Hooker; Patrick Lewis 002 (ACL (Findings), 2024)

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning. - Shivalika Singh; Freddie Vargus; Daniel D'souza; Börje Karlsson 001; Abinaya Mahendiran; Wei-Yin Ko; Herumb Shandilya; Jay Patel; Deividas Mataciunas; Laura O'Mahony; Mike Zhang; Ramith Hettiarachchi; et al. (ACL (1), 2024)

Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model. - Ahmet Üstün; Viraat Aryabumi; Zheng Xin Yong; Wei-Yin Ko; Daniel D'souza; Gbemileke Onilude; Neel Bhandari; Shivalika Singh; Hui-Lee Ooi; Amr Kayid; Freddie Vargus; Phil Blunsom; et al. (ACL (1), 2024)

The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm. - Aakanksha; Arash Ahmadian; Beyza Ermis; Seraphina Goldfarb-Tarrant; Julia Kreutzer; Marzieh Fadaee; Sara Hooker (EMNLP, 2024)

RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs. - John Dang; Arash Ahmadian; Kelly Marchisio; Julia Kreutzer; Ahmet Üstün; Sara Hooker (EMNLP, 2024)

How Does Quantization Affect Multilingual LLMs? - Kelly Marchisio; Saurabh Dash; Hongyu Chen 008; Dennis Aumiller; Ahmet Üstün; Sara Hooker; Sebastian Ruder (EMNLP (Findings), 2024)

LLM See, LLM Do: Leveraging Active Inheritance to Target Non-Differentiable Objectives. - Luísa Shimabucoro; Sebastian Ruder; Julia Kreutzer; Marzieh Fadaee; Sara Hooker (EMNLP, 2024)

Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning. - Ted Zadouri; Ahmet Üstün; Arash Ahmadian; Beyza Ermis; Acyr Locatelli; Sara Hooker (ICLR, 2024)

On The Fairness Impacts of Hardware Selection in Machine Learning. - Sree Harsha Nelaturu; Nishaanth Kanna Ravichandran; Cuong Tran 007; Sara Hooker; Ferdinando Fioretto (ICML, 2024)

Elo Uncovered: Robustness and Best Practices in Language Model Evaluation. - Meriem Boubdir; Edward Kim; Beyza Ermis; Sara Hooker; Marzieh Fadaee (NeurIPS, 2024)

Consent in Crisis: The Rapid Decline of the AI Data Commons. - Shayne Longpre; Robert Mahari; Ariel Lee; Campbell Lund; Hamidah Oderinwale; William Brannon; Nayan Saxena; Naana Obeng-Marnu; Tobin South; Cole Hunter; Kevin Klyman; Christopher Klamm; et al. (NeurIPS, 2024)

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning. - Shivalika Singh; Freddie Vargus; Daniel D'souza; Börje F. Karlsson; Abinaya Mahendiran; Wei-Yin Ko; Herumb Shandilya; Jay Patel; Deividas Mataciunas; Laura O'Mahony; Mike Zhang; Ramith Hettiarachchi; et al. (CoRR, 2024)

Aya 23: Open Weight Releases to Further Multilingual Progress. - Viraat Aryabumi; John Dang; Dwarak Talupuru; Saurabh Dash; David Cairuz; Hangyu Lin; Bharat Venkitesh; Madeline Smith; Jon Ander Campos; Yi Chern Tan; Kelly Marchisio; Max Bartolo; et al. (CoRR, 2024)

IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models. - David Ifeoluwa Adelani; Jessica Ojo; Israel Abebe Azime; Jian Yun Zhuang; Jesujoba O. Alabi; Xuanli He; Millicent Ochieng; Sara Hooker; Andiswa Bukula; En-Shiun Annie Lee; Chiamaka Chukwuneke; Happy Buzaaba; et al. (CoRR, 2024)

LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives. - Luísa Shimabucoro; Sebastian Ruder; Julia Kreutzer; Marzieh Fadaee; Sara Hooker (CoRR, 2024)

On the Limitations of Compute Thresholds as a Governance Strategy. - Sara Hooker (CoRR, 2024)

Consent in Crisis: The Rapid Decline of the AI Data Commons. - Shayne Longpre; Robert Mahari; Ariel Lee; Campbell Lund; Hamidah Oderinwale; William Brannon; Nayan Saxena; Naana Obeng-Marnu; Tobin South; Cole Hunter; Kevin Klyman; Christopher Klamm; et al. (CoRR, 2024)

Open Problems in Technical AI Governance. - Anka Reuel; Ben Bucknall; Stephen Casper; Timothy Fist; Lisa Soder; Onni Aarne; Lewis Hammond; Lujain Ibrahim; Alan Chan; Peter Wills; Markus Anderljung; Ben Garfinkel; et al. (CoRR, 2024)

Multilingual Arbitrage: Optimizing Data Pools to Accelerate Multilingual Progress. - Ayomide Odumakinde; Daniel D'souza; Pat Verga; Beyza Ermis; Sara Hooker (CoRR, 2024)

Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts. - Nikolas Gritsch; Qizhen Zhang; Acyr Locatelli; Sara Hooker; Ahmet Üstün (CoRR, 2024)

The Future of Open Human Feedback. - Shachar Don-Yehiya; Ben Burtenshaw; Ramón Fernandez Astudillo; Cailean Osborne; Mimansa Jaiswal; Tzu-Sheng Kuo; Wenting Zhao; Idan Shenfeld; Andi Peng; Mikhail Yurochkin; Atoosa Kasirzadeh; Yangsibo Huang; et al. (CoRR, 2024)

Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning. - Aakanksha; Arash Ahmadian; Seraphina Goldfarb-Tarrant; Beyza Ermis; Marzieh Fadaee; Sara Hooker (CoRR, 2024)

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation. - Shivalika Singh; Angelika Romanou; Clémentine Fourrier; David Ifeoluwa Adelani; Jian Gang Ngui; Daniel Vila-Suero; Peerat Limkonchotiwat; Kelly Marchisio; Wei Qi Leong; Yosephine Susanto; Raymond Ng; Shayne Longpre; et al. (CoRR, 2024)

Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier. - John Dang; Shivalika Singh; Daniel D'souza; Arash Ahmadian; Alejandro Salamanca; Madeline Smith; Aidan Peppin; Sungjin Hong; Manoj Govindassamy; Terrence Zhao; Sandra Kublik; Meor Amer; et al. (CoRR, 2024)

Bridging the Data Provenance Gap Across Text, Speech and Video. - Shayne Longpre; Nikhil Singh 003; Manuel Cherep; Kushagra Tiwary; Joanna Materzynska; William Brannon; Robert Mahari; Manan Dey; Mohammed Hamdy; Nayan Saxena; Ahmad Mustafa Anis; Emad A. Alghamdi; et al. (CoRR, 2024)

A large-scale audit of dataset licensing and attribution in AI. - Shayne Longpre; Robert Mahari; Anthony Chen; Naana Obeng-Marnu; Damien Sileo; William Brannon; Niklas Muennighoff; Nathan Khazam; Jad Kabbara; Kartik Perisetla; Xinyi Wu; Enrico Shippole; et al. (Nat. Mac. Intell., 2024)

M-RewardBench: Evaluating Reward Models in Multilingual Settings. - Srishti Gureja; Lester James Validad Miranda; Shayekh Bin Islam; Rishabh Maheshwary; Drishti Sharma; Gusti Triandi Winata; Nathan Lambert 001; Sebastian Ruder; Sara Hooker; Marzieh Fadaee (ACL (1), 2025)

Multilingual Arbitration: Optimizing Data Pools to Accelerate Multilingual Progress. - Ayomide Odumakinde; Daniel D'souza; Pat Verga; Beyza Ermis; Sara Hooker (ACL (1), 2025)

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation. - Shivalika Singh; Angelika Romanou; Clémentine Fourrier; David Ifeoluwa Adelani; Jian Gang Ngui; Daniel Vila-Suero; Peerat Limkonchotiwat; Kelly Marchisio; Wei Qi Leong; Yosephine Susanto; Raymond Ng; Shayne Longpre; et al. (ACL (1), 2025)

Fairness of Deep Ensembles: On the interplay between per-group task difficulty and under-representation. - Estanislao Claucich; Sara Hooker; Diego H. Milone; Enzo Ferrante; Rodrigo Echeveste (FAccT, 2025)

The Reality of AI and Biorisk. - Aidan Peppin; Anka Reuel; Stephen Casper; Elliot Jones; Andrew Strait; Usman Anwar; Anurag Agrawal; Sayash Kapoor; Sanmi Koyejo; Marie Pellat; Rishi Bommasani; Nick Frosst; et al. (FAccT, 2025)

To Code or Not To Code? Exploring Impact of Code in Pre-training. - Viraat Aryabumi; Yixuan Su; Raymond Ma; Adrien Morisot; Ivan Zhang; Acyr Locatelli; Marzieh Fadaee; Ahmet Üstün; Sara Hooker (ICLR, 2025)

INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge. - Angelika Romanou; Negar Foroutan; Anna Sotnikova; Zeming Chen 001; Sree Harsha Nelaturu; Shivalika Singh; Rishabh Maheshwary; Micol Altomare; Mohamed A. Haggag; Imanol Schlag; Marzieh Fadaee; Sara Hooker; et al. (ICLR, 2025)

IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models. - David Ifeoluwa Adelani; Jessica Ojo; Israel Abebe Azime; Jian Yun Zhuang; Jesujoba Oluwadara Alabi; Xuanli He; Millicent Ochieng; Sara Hooker; Andiswa Bukula; En-Shiun Annie Lee; Chiamaka Ijeoma Chukwuneke; Happy Buzaaba; et al. (NAACL (Long Papers), 2025)

Open Problems in Technical AI Governance. - Anka Reuel; Benjamin Bucknall; Stephen Casper; Timothy Fist; Lisa Soder; Onni Aarne; Lewis Hammond; Lujain Ibrahim; Alan Chan; Peter Wills; Markus Anderljung; Ben Garfinkel; et al. (Trans. Mach. Learn. Res., 2025)

MMTEB: Massive Multilingual Text Embedding Benchmark. - Kenneth C. Enevoldsen; Isaac Chung; Imene Kerboua; Márton Kardos; Ashwin Mathur; David Stap; Jay Gala; Wissam Siblini; Dominik Krzeminski; Genta Indra Winata; Saba Sturua; Saiteja Utpala; et al. (CoRR, 2025)

Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation. - Israfel Salazar; Manuel Fernández Burda; Shayekh Bin Islam; Arshia Soltani Moakhar; Shivalika Singh; Fabian Farestam; Angelika Romanou; Danylo Boiko; Dipika Khullar; Mike Zhang; Dominik Krzeminski; Jekaterina Novikova; et al. (CoRR, 2025)

The Leaderboard Illusion. - Shivalika Singh; Yiyang Nan; Alex Wang; Daniel D'souza; Sayash Kapoor; Ahmet Üstün; Sanmi Koyejo; Yuntian Deng; Shayne Longpre; Noah Smith; Beyza Ermis; Marzieh Fadaee; et al. (CoRR, 2025)

Aya Vision: Advancing the Frontier of Multilingual Multimodality. - Saurabh Dash; Yiyang Nan; John Dang; Arash Ahmadian; Shivalika Singh; Madeline Smith; Bharat Venkitesh; Vlad Shmyhlo; Viraat Aryabumi; Walter Beller-Morales; Jeremy Pekmez; Jason Ozuzu; et al. (CoRR, 2025)

The Multilingual Divide and Its Impact on Global AI Safety. - Aidan Peppin; Julia Kreutzer; Alice Schoenauer Sebag; Kelly Marchisio; Beyza Ermis; John Dang; Samuel Cahyawijaya; Shivalika Singh; Seraphina Goldfarb-Tarrant; Viraat Aryabumi; Aakanksha; Wei-Yin Ko; et al. (CoRR, 2025)

One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers. - Diana Abagyan; Alejandro Salamanca; Felipe Cruz-Salinas; Kris Cao; Hangyu Lin; Acyr Locatelli; Marzieh Fadaee; Ahmet Üstün; Sara Hooker (CoRR, 2025)

Treasure Hunt: Real-time Targeting of the Long Tail using Training-Time Markers. - Daniel D'souza; Julia Kreutzer; Adrien Morisot; Ahmet Üstün; Sara Hooker (CoRR, 2025)

Language Models can perform Single-Utterance Self-Correction of Perturbed Reasoning. - Sam Silver; Jimin Sun; Ivan Zhang; Sara Hooker; Eddie Kim (CoRR, 2025)

When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs. - Ammar Khairi; Daniel D'souza; Ye Shen; Julia Kreutzer; Sara Hooker (CoRR, 2025)