Adaptive Estimators Show Information Compression in Deep Neural Networks. - Ivan Chelombiev; Conor J. Houghton; Cian O'Donnell (ICLR (Poster), 2019)

GroupBERT: Enhanced Transformer Architecture with Efficient Grouped Structures. - Ivan Chelombiev; Daniel Justus; Douglas Orr; Anastasia Dietrich; Frithjof Gressmann; Alexandros Koliousis; Carlo Luschi (CoRR, 2021)

Towards Structured Dynamic Sparse Pre-Training of BERT. - Anastasia Dietrich; Frithjof Gressmann; Douglas Orr; Ivan Chelombiev; Daniel Justus; Carlo Luschi (CoRR, 2021)

SparQ Attention: Bandwidth-Efficient LLM Inference. - Luka Ribar; Ivan Chelombiev; Luke Hudlass-Galley; Charlie Blake; Carlo Luschi; Douglas Orr (ICML, 2024)