On Extractive and Abstractive Neural Document Summarization with Transformer Language Models. - Jonathan Pilault; Raymond Li; Sandeep Subramanian; Chris Pal (EMNLP (1), 2020)

On the impressive performance of randomly weighted encoders in summarization tasks. - Jonathan Pilault; Jaehong Park; Christopher J. Pal (CoRR, 2020)

Learning to Summarize Long Texts with Memory Compression and Transfer. - Jaehong Park; Jonathan Pilault; Christopher Joseph Pal (CoRR, 2020)

Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data. - Jonathan Pilault; Amine Elhattami; Christopher J. Pal (ICLR, 2021)

Towards Neural Functional Program Evaluation. - Torsten Scholak; Jonathan Pilault; Joey Velez-Ginorio (CoRR, 2021)

Using Graph Algorithms to Pretrain Graph Completion Transformers. - Jonathan Pilault; Michael Galkin; Bahare Fatemi; Perouz Taslakian; David Vázquez 001; Christopher Pal (CoRR, 2022)

On Conditional and Compositional Language Model Differentiable Prompting. - Jonathan Pilault; Can Liu; Mohit Bansal; Markus Dreyer (IJCAI, 2023)

Interactive-Chain-Prompting: Ambiguity Resolution for Crosslingual Conditional Generation with Interaction. - Jonathan Pilault; Xavier Garcia; Arthur Brazinskas; Orhan Firat (IJCNLP (1), 2023)

Block-State Transformers. - Jonathan Pilault; Mahan Fathi; Orhan Firat; Chris Pal; Pierre-Luc Bacon; Ross Goroshin (NeurIPS, 2023)

JaxPruner: A concise library for sparsity research. - Joo Hyung Lee; Wonpyo Park; Nicole Mitchell; Jonathan Pilault; Johan S. Obando-Ceron; Han-Byul Kim; Namhoon Lee; Elias Frantar; Yun Long; Amir Yazdanbakhsh; Shivani Agrawal; Suvinay Subramanian; et al. (CoRR, 2023)

Block-State Transformer. - Mahan Fathi; Jonathan Pilault; Pierre-Luc Bacon; Christopher Pal; Orhan Firat; Ross Goroshin (CoRR, 2023)

Jaxpruner: A Concise Library for Sparsity Research. - Joo Hyung Lee; Wonpyo Park; Nicole Mitchell; Jonathan Pilault; Johan Samir Obando Ceron; Han-Byul Kim; Namhoon Lee; Elias Frantar; Yun Long; Amir Yazdanbakhsh; Woohyun Han; Shivani Agrawal; et al. (CPAL, 2024)

Course Correcting Koopman Representations. - Mahan Fathi; Clement Gehring; Jonathan Pilault; David Kanaa; Pierre-Luc Bacon; Ross Goroshin (ICLR, 2024)

Zamba: A Compact 7B SSM Hybrid Model. - Paolo Glorioso; Quentin Anthony; Yury Tokpanov; James Whittington; Jonathan Pilault; Adam Ibrahim; Beren Millidge (CoRR, 2024)

Zyda: A 1.3T Dataset for Open Language Modeling. - Yury Tokpanov; Beren Millidge; Paolo Glorioso; Jonathan Pilault; Adam Ibrahim; James Whittington; Quentin Anthony (CoRR, 2024)

Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters. - Vasudev Shyam; Jonathan Pilault; Emily Shepperd; Quentin Anthony; Beren Millidge (CoRR, 2024)

The Zamba2 Suite: Technical Report. - Paolo Glorioso; Quentin Anthony; Yury Tokpanov; Anna Golubeva; Vasudev Shyam; James Whittington; Jonathan Pilault; Beren Millidge (CoRR, 2024)