What can a Single Attention Layer Learn? A Study Through the Random Features Lens. - Hengyu Fu; Tianyu Guo 004; Yu Bai 017; Song Mei (NeurIPS, 2023)

How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations. - Tianyu Guo 004; Wei Hu; Song Mei; Huan Wang 014; Caiming Xiong; Silvio Savarese; Yu Bai 017 (ICLR, 2024)

Collaborative Heterogeneous Causal Inference Beyond Meta-analysis. - Tianyu Guo 004; Sai Praneeth Karimireddy; Michael I. Jordan (ICML, 2024)

Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs. - Tianyu Guo 004; Druv Pai; Yu Bai 017; Jiantao Jiao; Michael I. Jordan; Song Mei (CoRR, 2024)

How Do LLMs Perform Two-Hop Reasoning in Context? - Tianyu Guo 004; Hanlin Zhu; Ruiqi Zhang; Jiantao Jiao; Song Mei; Michael I. Jordan; Stuart Russell 001 (CoRR, 2025)

IDA-Bench: Evaluating LLMs on Interactive Guided Data Analysis. - Hanyu Li; Haoyu Liu; Tingyu Zhu; Tianyu Guo 004; Zeyu Zheng; Xiaotie Deng; Michael I. Jordan (CoRR, 2025)

Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformers. - Yixiao Huang 004; Hanlin Zhu; Tianyu Guo 004; Jiantao Jiao; Somayeh Sojoudi; Michael I. Jordan; Stuart Russell 001; Song Mei (CoRR, 2025)