An Empirical Investigation of Learning from Biased Toxicity Labels. - Neel Nanda; Jonathan Uesato; Sven Gowal (CoRR, 2021)

Predictability and Surprise in Large Generative Models. - Deep Ganguli; Danny Hernandez; Liane Lovitt; Amanda Askell; Yuntao Bai; Anna Chen; Tom Conerly; Nova DasSarma; Dawn Drain; Nelson Elhage; Sheer El Showk; Stanislav Fort; et al. (FAccT, 2022)

Predictability and Surprise in Large Generative Models. - Deep Ganguli; Danny Hernandez; Liane Lovitt; Nova DasSarma; Tom Henighan; Andy Jones; Nicholas Joseph; Jackson Kernion; Benjamin Mann; Amanda Askell; Yuntao Bai; Anna Chen; et al. (CoRR, 2022)

Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. - Yuntao Bai; Andy Jones; Kamal Ndousse; Amanda Askell; Anna Chen; Nova DasSarma; Dawn Drain; Stanislav Fort; Deep Ganguli; Tom Henighan; Nicholas Joseph; Saurav Kadavath; et al. (CoRR, 2022)

In-context Learning and Induction Heads. - Catherine Olsson; Nelson Elhage; Neel Nanda; Nicholas Joseph; Nova DasSarma; Tom Henighan; Ben Mann; Amanda Askell; Yuntao Bai; Anna Chen; Tom Conerly; Dawn Drain; et al. (CoRR, 2022)

Fully General Online Imitation Learning. - Michael K. Cohen; Marcus Hutter; Neel Nanda (J. Mach. Learn. Res., 2022)

Emergent Linear Representations in World Models of Self-Supervised Sequence Models. - Neel Nanda; Andrew Lee 001; Martin Wattenberg (BlackboxNLP@EMNLP, 2023)

Progress measures for grokking via mechanistic interpretability. - Neel Nanda; Lawrence Chan; Tom Lieberum; Jess Smith; Jacob Steinhardt (ICLR, 2023)

A Toy Model of Universality: Reverse Engineering how Networks Learn Group Operations. - Bilal Chughtai; Lawrence Chan; Neel Nanda (ICML, 2023)

N2G: A Scalable Approach for Quantifying Interpretable Neuron Representations in Large Language Models. - Alex Foote; Neel Nanda; Esben Kran; Ioannis Konstas; Fazl Barez (CoRR, 2023)

Neuron to Graph: Interpreting Language Model Neurons at Scale. - Alex Foote; Neel Nanda; Esben Kran; Ioannis Konstas; Shay B. Cohen; Fazl Barez (CoRR, 2023)

Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla. - Tom Lieberum; Matthew Rahtz; János Kramár; Neel Nanda; Geoffrey Irving; Rohin Shah; Vladimir Mikulik (CoRR, 2023)

Copy Suppression: Comprehensively Understanding an Attention Head. - Callum McDougall; Arthur Conmy; Cody Rushing; Thomas McGrath; Neel Nanda (CoRR, 2023)

Linear Representations of Sentiment in Large Language Models. - Curt Tigges; Oskar John Hollinsworth; Atticus Geiger; Neel Nanda (CoRR, 2023)

Training Dynamics of Contextual N-Grams in Language Models. - Lucia Quirke; Lovis Heindrich; Wes Gurnee; Neel Nanda (CoRR, 2023)

Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching. - Aleksandar Makelov; Georg Lange; Neel Nanda (CoRR, 2023)

Finding Neurons in a Haystack: Case Studies with Sparse Probing. - Wes Gurnee; Neel Nanda; Matthew Pauly; Katherine Harvey; Dmitrii Troitskii; Dimitris Bertsimas (Trans. Mach. Learn. Res., 2023)

Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching. - Aleksandar Makelov; Georg Lange; Atticus Geiger; Neel Nanda (ICLR, 2024)

Towards Best Practices of Activation Patching in Language Models: Metrics and Methods. - Fred Zhang; Neel Nanda (ICLR, 2024)

Explorations of Self-Repair in Language Models. - Cody Rushing; Neel Nanda (ICML, 2024)

Refusal in Language Models Is Mediated by a Single Direction. - Andy Arditi; Oscar Obeso; Aaquib Syed; Daniel Paleka; Nina Panickssery; Wes Gurnee; Neel Nanda (NeurIPS, 2024)

Transcoders find interpretable LLM feature circuits. - Jacob Dunefsky; Philippe Chlenski; Neel Nanda (NeurIPS, 2024)

Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders. - Senthooran Rajamanoharan; Arthur Conmy; Lewis Smith; Tom Lieberum; Vikrant Varma; János Kramár; Rohin Shah; Neel Nanda (NeurIPS, 2024)

Confidence Regulation Neurons in Language Models. - Alessandro Stolfo; Ben Wu 001; Wes Gurnee; Yonatan Belinkov; Xingyi Song; Mrinmaya Sachan; Neel Nanda (NeurIPS, 2024)

Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs. - Bilal Chughtai; Alan Cooney; Neel Nanda (CoRR, 2024)

AtP*: An efficient and scalable method for localizing LLM behaviour to components. - János Kramár; Tom Lieberum; Rohin Shah; Neel Nanda (CoRR, 2024)

How to use and interpret activation patching. - Stefan Heimersheim; Neel Nanda (CoRR, 2024)

Improving Dictionary Learning with Gated Sparse Autoencoders. - Senthooran Rajamanoharan; Arthur Conmy; Lewis Smith; Tom Lieberum; Vikrant Varma; János Kramár; Rohin Shah; Neel Nanda (CoRR, 2024)

Refusal in Language Models Is Mediated by a Single Direction. - Andy Arditi; Oscar Obeso; Aaquib Syed; Daniel Paleka; Nina Rimsky; Wes Gurnee; Neel Nanda (CoRR, 2024)

Interpreting Attention Layer Outputs with Sparse Autoencoders. - Connor Kissane; Robert Krzyzanowski; Joseph Isaac Bloom; Arthur Conmy; Neel Nanda (CoRR, 2024)

Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders. - Senthooran Rajamanoharan; Tom Lieberum; Nicolas Sonnerat; Arthur Conmy; Vikrant Varma; János Kramár; Neel Nanda (CoRR, 2024)

Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2. - Tom Lieberum; Senthooran Rajamanoharan; Arthur Conmy; Lewis Smith; Nicolas Sonnerat; Vikrant Varma; János Kramár; Anca D. Dragan; Rohin Shah; Neel Nanda (CoRR, 2024)

Evaluating Sparse Autoencoders on Targeted Concept Erasure Tasks. - Adam Karvonen; Can Rager; Samuel Marks; Neel Nanda (CoRR, 2024)

BatchTopK Sparse Autoencoders. - Bart Bussmann; Patrick Leask; Neel Nanda (CoRR, 2024)

Universal Neurons in GPT2 Language Models. - Wes Gurnee; Theo Horsley; Zifan Carl Guo; Tara Rezaei Kheirkhah; Qinyi Sun; Will Hathaway; Neel Nanda; Dimitris Bertsimas (Trans. Mach. Learn. Res., 2024)

Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models. - Javier Ferrando; Oscar Balcells Obeso; Senthooran Rajamanoharan; Neel Nanda (ICLR, 2025)

Sparse Autoencoders Do Not Find Canonical Units of Analysis. - Patrick Leask; Bart Bussmann; Michael T. Pearce; Joseph Isaac Bloom; Curt Tigges; Noura Al Moubayed; Lee Sharkey; Neel Nanda (ICLR, 2025)

Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control. - Aleksandar Makelov; Georg Lange; Neel Nanda (ICLR, 2025)

Open Problems in Mechanistic Interpretability. - Lee Sharkey; Bilal Chughtai; Joshua Batson; Jack Lindsey; Jeff Wu 003; Lucius Bushnaq; Nicholas Goldowsky-Dill; Stefan Heimersheim; Alejandro Ortega; Joseph Isaac Bloom; Stella Biderman; Adrià Garriga-Alonso; et al. (CoRR, 2025)

Are Sparse Autoencoders Useful? A Case Study in Sparse Probing. - Subhash Kantamneni; Joshua Engels; Senthooran Rajamanoharan; Max Tegmark; Neel Nanda (CoRR, 2025)

Chain-of-Thought Reasoning In The Wild Is Not Always Faithful. - Iván Arcuschin; Jett Janiak; Robert Krzyzanowski; Senthooran Rajamanoharan; Neel Nanda; Arthur Conmy (CoRR, 2025)

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability. - Adam Karvonen; Can Rager; Johnny Lin; Curt Tigges; Joseph Isaac Bloom; David Chanin; Yeu-Tong Lau; Eoin Farrell; Callum McDougall; Kola Ayonrinde; Matthew Wearden; Arthur Conmy; et al. (CoRR, 2025)

Learning Multi-Level Features with Matryoshka Sparse Autoencoders. - Bart Bussmann; Noa Nabeshima; Adam Karvonen; Neel Nanda (CoRR, 2025)

An Approach to Technical AGI Safety and Security. - Rohin Shah; Alex Irpan; Alexander Matt Turner; Anna Wang; Arthur Conmy; David Lindner; Jonah Brown-Cohen; Lewis Ho; Neel Nanda; Raluca Ada Popa; Rishub Jain; Rory Greig; et al. (CoRR, 2025)

Robustly identifying concepts introduced during chat fine-tuning using crosscoders. - Julian Minder; Clément Dumas; Caden Juang; Bilal Chugtai; Neel Nanda (CoRR, 2025)

Scaling sparse feature circuit finding for in-context learning. - Dmitrii Kharlapenko; Stepan Shabalin; Fazl Barez; Arthur Conmy; Neel Nanda (CoRR, 2025)

Towards eliciting latent knowledge from LLMs with mechanistic interpretability. - Bartosz Cywinski; Emil Ryd; Senthooran Rajamanoharan; Neel Nanda (CoRR, 2025)

Inference-Time Decomposition of Activations (ITDA): A Scalable Approach to Interpreting Large Language Models. - Patrick Leask; Neel Nanda; Noura Al Moubayed (CoRR, 2025)

Model Organisms for Emergent Misalignment. - Edward Turner; Anna Soligo; Mia Taylor; Senthooran Rajamanoharan; Neel Nanda (CoRR, 2025)

Convergent Linear Representations of Emergent Misalignment. - Anna Soligo; Edward Turner; Senthooran Rajamanoharan; Neel Nanda (CoRR, 2025)

How Visual Representations Map to Language Feature Space in Multimodal LLMs. - Constantin Venhoff; Ashkan Khakzar; Sonia Joseph; Philip Torr 001; Neel Nanda (CoRR, 2025)

Because we have LLMs, we Can and Should Pursue Agentic Interpretability. - Been Kim; John Hewitt; Neel Nanda; Noah Fiedel; Oyvind Tafjord (CoRR, 2025)

Understanding Reasoning in Thinking Language Models via Steering Vectors. - Constantin Venhoff; Iván Arcuschin; Philip Torr 001; Arthur Conmy; Neel Nanda (CoRR, 2025)

Thought Anchors: Which LLM Reasoning Steps Matter? - Paul C. Bogdan; Uzay Macar; Neel Nanda; Arthur Conmy (CoRR, 2025)

Simple Mechanistic Explanations for Out-Of-Context Reasoning. - Atticus Wang; Joshua Engels; Oliver Clive-Griffin; Senthooran Rajamanoharan; Neel Nanda (CoRR, 2025)

Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. - Tomek Korbak; Mikita Balesni; Elizabeth Barnes; Yoshua Bengio; Joe Benton; Joseph Bloom; Mark Chen 003; Alan Cooney; Allan Dafoe; Anca D. Dragan; Scott Emmons; Owain Evans; et al. (CoRR, 2025)

Reasoning-Finetuning Repurposes Latent Representations in Base Models. - Jake Ward; Chuqiao Lin; Constantin Venhoff; Neel Nanda (CoRR, 2025)

Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning. - Helena Casademunt; Caden Juang; Adam Karvonen; Samuel Marks; Senthooran Rajamanoharan; Neel Nanda (CoRR, 2025)