Diminishing Returns Shape Constraints for Interpretability and Regularization. - Maya R. Gupta; Dara Bahri; Andrew Cotter; Kevin Robert Canini (NeurIPS, 2018)

Reverse Engineering Configurations of Neural Text Generation Models. - Yi Tay; Dara Bahri; Che Zheng; Clifford Brunk; Donald Metzler; Andrew Tomkins (ACL, 2020)

Deep k-NN for Noisy Labels. - Dara Bahri; Heinrich Jiang; Maya R. Gupta (ICML, 2020)

Sparse Sinkhorn Attention. - Yi Tay; Dara Bahri; Liu Yang; Donald Metzler; Da-Cheng Juan (ICML, 2020)

Choppy: Cut Transformer for Ranked List Truncation. - Dara Bahri; Yi Tay; Che Zheng; Donald Metzler; Andrew Tomkins (SIGIR, 2020)

HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections. - Yi Tay; Zhe Zhao 001; Dara Bahri; Donald Metzler; Da-Cheng Juan (CoRR, 2020)

StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling. - Yikang Shen; Yi Tay; Che Zheng; Dara Bahri; Donald Metzler; Aaron C. Courville (ACL/IJCNLP (1), 2021)

Are Pretrained Convolutions Better than Pretrained Transformers? - Yi Tay; Mostafa Dehghani 001; Jai Prakash Gupta 001; Vamsi Aribandi; Dara Bahri; Zhen Qin 001; Donald Metzler (ACL/IJCNLP (1), 2021)

Long Range Arena : A Benchmark for Efficient Transformers. - Yi Tay; Mostafa Dehghani 001; Samira Abnar; Yikang Shen; Dara Bahri; Philip Pham; Jinfeng Rao; Liu Yang; Sebastian Ruder; Donald Metzler (ICLR, 2021)

HyperGrid Transformers: Towards A Single Model for Multiple Tasks. - Yi Tay; Zhe Zhao 001; Dara Bahri; Donald Metzler; Da-Cheng Juan (ICLR, 2021)

Locally Adaptive Label Smoothing Improves Predictive Churn. - Dara Bahri; Heinrich Jiang (ICML, 2021)

OmniNet: Omnidirectional Representations from Transformers. - Yi Tay; Mostafa Dehghani 001; Vamsi Aribandi; Jai Prakash Gupta 001; Philip Pham; Zhen Qin 001; Dara Bahri; Da-Cheng Juan; Donald Metzler (ICML, 2021)

Synthesizer: Rethinking Self-Attention for Transformer Models. - Yi Tay; Dara Bahri; Donald Metzler; Da-Cheng Juan; Zhe Zhao 001; Che Zheng (ICML, 2021)

Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study. - Dara Bahri; Yi Tay; Che Zheng; Cliff Brunk; Donald Metzler; Andrew Tomkins (WSDM, 2021)

Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection. - Dara Bahri; Heinrich Jiang; Yi Tay; Donald Metzler (CoRR, 2021)

Locally Adaptive Label Smoothing for Predictive Churn. - Dara Bahri; Heinrich Jiang (CoRR, 2021)

Rethinking Search: Making Experts out of Dilettantes. - Donald Metzler; Yi Tay; Dara Bahri; Marc Najork (CoRR, 2021)

Are Pre-trained Convolutions Better than Pre-trained Transformers? - Yi Tay; Mostafa Dehghani 001; Jai Prakash Gupta 001; Dara Bahri; Vamsi Aribandi; Zhen Qin 001; Donald Metzler (CoRR, 2021)

Rethinking search: making domain experts out of dilettantes. - Donald Metzler; Yi Tay; Dara Bahri; Marc Najork (SIGIR Forum, 2021)

ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference. - Kai Hui 001; Honglei Zhuang; Tao Chen 008; Zhen Qin 001; Jing Lu; Dara Bahri; Ji Ma 004; Jai Prakash Gupta 001; Cícero Nogueira dos Santos; Yi Tay; Donald Metzler (ACL (Findings), 2022)

Sharpness-Aware Minimization Improves Language Model Generalization. - Dara Bahri; Hossein Mobahi; Yi Tay (ACL (1), 2022)

ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning. - Vamsi Aribandi; Yi Tay; Tal Schuster; Jinfeng Rao; Huaixiu Steven Zheng; Sanket Vaibhav Mehta; Honglei Zhuang; Vinh Q. Tran 002; Dara Bahri; Jianmo Ni; Jai Prakash Gupta 001; Kai Hui 001; et al. (ICLR, 2022)

Scarf: Self-Supervised Contrastive Learning using Random Feature Corruption. - Dara Bahri; Heinrich Jiang; Yi Tay; Donald Metzler (ICLR, 2022)

Churn Reduction via Distillation. - Heinrich Jiang; Harikrishna Narasimhan; Dara Bahri; Andrew Cotter; Afshin Rostamizadeh (ICLR, 2022)

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization. - Yi Tay; Vinh Q. Tran 002; Sebastian Ruder; Jai Prakash Gupta 001; Hyung Won Chung; Dara Bahri; Zhen Qin 001; Simon Baumgartner; Cong Yu 001; Donald Metzler (ICLR, 2022)

Confident Adaptive Language Modeling. - Tal Schuster; Adam Fisch; Jai Gupta 001; Mostafa Dehghani 001; Dara Bahri; Vinh Tran 002; Yi Tay; Donald Metzler (NeurIPS, 2022)

Transformer Memory as a Differentiable Search Index. - Yi Tay; Vinh Tran 002; Mostafa Dehghani 001; Jianmo Ni; Dara Bahri; Harsh Mehta; Zhen Qin 001; Kai Hui 001; Zhe Zhao 001; Jai Prakash Gupta 001; Tal Schuster; William W. Cohen; et al. (NeurIPS, 2022)

Unifying Language Learning Paradigms. - Yi Tay; Mostafa Dehghani 001; Vinh Q. Tran 002; Xavier Garcia; Dara Bahri; Tal Schuster; Huaixiu Steven Zheng; Neil Houlsby; Donald Metzler (CoRR, 2022)

Is margin all you need? An extensive empirical study of active learning on tabular data. - Dara Bahri; Heinrich Jiang; Tal Schuster; Afshin Rostamizadeh (CoRR, 2022)

UL2: Unifying Language Learning Paradigms. - Yi Tay; Mostafa Dehghani 001; Vinh Q. Tran 002; Xavier Garcia; Jason Wei; Xuezhi Wang 002; Hyung Won Chung; Dara Bahri; Tal Schuster; Huaixiu Steven Zheng; Denny Zhou; Neil Houlsby; et al. (ICLR, 2023)

Sharpness-Aware Minimization Leads to Low-Rank Features. - Maksym Andriushchenko; Dara Bahri; Hossein Mobahi; Nicolas Flammarion (NeurIPS, 2023)

Surprise: Result List Truncation via Extreme Value Theory. - Dara Bahri; Che Zheng; Yi Tay; Donald Metzler; Andrew Tomkins (SIGIR, 2023)

Efficient Transformers: A Survey. - Yi Tay; Mostafa Dehghani 001; Dara Bahri; Donald Metzler (ACM Comput. Surv., 2023)

A Universal Class of Sharpness-Aware Minimization Algorithms. - Behrooz Tahmasebi; Ashkan Soleymani; Dara Bahri; Stefanie Jegelka; Patrick Jaillet (ICML, 2024)

A Watermark for Black-Box Language Models. - Dara Bahri; John Wieting; Dana Alon; Donald Metzler (CoRR, 2024)

Decoding-based Regression. - Xingyou Song; Dara Bahri (Trans. Mach. Learn. Res., 2025)

Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities. - Gheorghe Comanici; Eric Bieber; Mike Schaekermann; Ice Pasupat; Noveen Sachdeva; Inderjit S. Dhillon; Marcel Blistein; Ori Ram; Dan Zhang; Evan Rosen; Luke Marris; Sam Petulla; et al. (CoRR, 2025)

Improving Detection of Watermarked Language Models. - Dara Bahri; John Wieting (CoRR, 2025)