Hierarchical Transformers Are More Efficient Language Models. - Piotr Nawrot; Szymon Tworkowski; Michal Tyrolski; Lukasz Kaiser; Yuhuai Wu; Christian Szegedy; Henryk Michalewski (NAACL-HLT (Findings), 2022)

Efficient Transformers with Dynamic Token Pooling. - Piotr Nawrot; Jan Chorowski; Adrian Lancucki; Edoardo Maria Ponti (ACL (1), 2023)

No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models. - Jean Kaddour; Oscar Key; Piotr Nawrot; Pasquale Minervini; Matt J. Kusner (NeurIPS, 2023)

nanoT5: A PyTorch Framework for Pre-training and Fine-tuning T5-style Models with Limited Resources. - Piotr Nawrot (CoRR, 2023)

Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference. - Piotr Nawrot; Adrian Lancucki; Marcin Chochowski; David Tarjan; Edoardo M. Ponti (ICML, 2024)

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs. - Piotr Nawrot; Robert Li; Renjie Huang; Sebastian Ruder; Kelly Marchisio; Edoardo M. Ponti (CoRR, 2025)

Inference-Time Hyper-Scaling with KV Cache Compression. - Adrian Lancucki; Konrad Staniszewski; Piotr Nawrot; Edoardo M. Ponti (CoRR, 2025)