Hierarchical Transformers Are More Efficient Language Models. - Piotr Nawrot; Szymon Tworkowski; Michal Tyrolski; Lukasz Kaiser; Yuhuai Wu; Christian Szegedy; Henryk Michalewski (NAACL-HLT (Findings), 2022)

Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Provers. - Albert Qiaochu Jiang; Wenda Li; Szymon Tworkowski; Konrad Czechowski; Tomasz Odrzygózdz; Piotr Milos; Yuhuai Wu; Mateja Jamnik (NeurIPS, 2022)

Focused Transformer: Contrastive Training for Context Scaling. - Szymon Tworkowski; Konrad Staniszewski; Mikolaj Pacek; Yuhuai Wu; Henryk Michalewski; Piotr Milos (NeurIPS, 2023)

Magnushammer: A Transformer-based Approach to Premise Selection. - Maciej Mikula; Szymon Antoniak; Szymon Tworkowski; Albert Qiaochu Jiang; Jin Peng Zhou; Christian Szegedy; Lukasz Kucinski; Piotr Milos; Yuhuai Wu (CoRR, 2023)

Explaining Competitive-Level Programming Solutions using LLMs. - Jierui Li; Szymon Tworkowski; Yingying Wu; Raymond J. Mooney (CoRR, 2023)

Structured Packing in LLM Training Improves Long Context Utilization. - Konrad Staniszewski; Szymon Tworkowski; Sebastian Jaszczur; Henryk Michalewski; Lukasz Kucinski; Piotr Milos (CoRR, 2023)

Analysing The Impact of Sequence Composition on Language Model Pre-Training. - Yu Zhao 043; Yuanbin Qu; Konrad Staniszewski; Szymon Tworkowski; Wei Liu; Piotr Milos; Yuxiang Wu; Pasquale Minervini (ACL (1), 2024)

Magnushammer: A Transformer-Based Approach to Premise Selection. - Maciej Mikula; Szymon Tworkowski; Szymon Antoniak; Bartosz Piotrowski; Albert Q. Jiang; Jin Peng Zhou; Christian Szegedy; Lukasz Kucinski; Piotr Milos; Yuhuai Wu (ICLR, 2024)

Structured Packing in LLM Training Improves Long Context Utilization. - Konrad Staniszewski; Szymon Tworkowski; Sebastian Jaszczur; Yu Zhao 043; Henryk Michalewski; Lukasz Kucinski; Piotr Milos (AAAI, 2025)