ETC: Encoding Long and Structured Inputs in Transformers. - Joshua Ainslie; Santiago Ontañón; Chris Alberti; Vaclav Cvicek; Zachary Fisher; Philip Pham; Anirudh Ravula; Sumit Sanghai; Qifan Wang 001; Li Yang (EMNLP (1), 2020)

Big Bird: Transformers for Longer Sequences. - Manzil Zaheer; Guru Guruganesh; Kumar Avinava Dubey; Joshua Ainslie; Chris Alberti; Santiago Ontañón; Philip Pham; Anirudh Ravula; Qifan Wang 001; Li Yang; Amr Ahmed 001 (NeurIPS, 2020)

ETC: Encoding Long and Structured Data in Transformers. - Joshua Ainslie; Santiago Ontañón; Chris Alberti; Philip Pham; Anirudh Ravula; Sumit Sanghai (CoRR, 2020)

RealFormer: Transformer Likes Residual Attention. - Ruining He; Anirudh Ravula; Bhargav Kanagal; Joshua Ainslie (ACL/IJCNLP (Findings), 2021)

DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections. - Yury Zemlyanskiy; Sudeep Gandhe; Ruining He; Bhargav Kanagal; Anirudh Ravula; Juraj Gottweis; Fei Sha; Ilya Eckstein (EACL, 2021)

WebFormer: The Web-page Transformer for Structure Information Extraction. - Qifan Wang 001; Yi Fang 008; Anirudh Ravula; Fuli Feng; Xiaojun Quan; Dongfang Liu (WWW, 2022)

Deep Partial Multiplex Network Embedding. - Qifan Wang 001; Yi Fang 008; Anirudh Ravula; Ruining He; Bin Shen 002; Jingang Wang; Xiaojun Quan; Dongfang Liu (WWW (Companion Volume), 2022)