PGDB: A Debugger for MPI Applications. - Nikoli Dryden (XSEDE, 2014)

PPL: an abstract runtime system for hybrid parallel programming. - Alex Brooks; Hoang-Vu Dang; Nikoli Dryden; Marc Snir (ESPM@SC, 2015)

Communication Quantization for Data-Parallel Training of Deep Neural Networks. - Nikoli Dryden; Tim Moon; Sam Ade Jacobs; Brian Van Essen (MLHPC@SC, 2016)

Towards Scalable Parallel Training of Deep Neural Networks. - Sam Ade Jacobs; Nikoli Dryden; Roger A. Pearce; Brian Van Essen (MLHPC@SC, 2017)

Neural Network Based Silent Error Detector. - Chen Wang 004; Nikoli Dryden; Franck Cappello; Marc Snir (CLUSTER, 2018)

A Lightweight Communication Runtime for Distributed Graph Analytics. - Hoang-Vu Dang; Roshan Dathathri; Gurbinder Gill; Alex Brooks; Nikoli Dryden; Andrew Lenharth; Loc Hoang; Keshav Pingali; Marc Snir (IPDPS, 2018)

Gluon: a communication-optimizing substrate for distributed heterogeneous graph analytics. - Roshan Dathathri; Gurbinder Gill; Loc Hoang; Hoang-Vu Dang; Alex Brooks; Nikoli Dryden; Marc Snir; Keshav Pingali (PLDI, 2018)

Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism. - Nikoli Dryden; Naoya Maruyama; Tom Benson; Tim Moon; Marc Snir; Brian Van Essen (IPDPS, 2019)

Channel and filter parallelism for large-scale CNN training. - Nikoli Dryden; Naoya Maruyama; Tim Moon; Tom Benson; Marc Snir; Brian Van Essen (SC, 2019)

Predicting Weather Uncertainty with Deep Convnets. - Peter Grönquist; Tal Ben-Nun; Nikoli Dryden; Peter Dueben; Luca Lavarini; Shigang Li 002; Torsten Hoefler (CoRR, 2019)

Large-scale training of deep neural networks - Nikoli Dryden (2019)

Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging. - Shigang Li 002; Tal Ben-Nun; Dan Alistarh; Salvatore Di Girolamo; Nikoli Dryden; Torsten Hoefler (CoRR, 2020)

Deep Learning for Post-Processing Ensemble Weather Forecasts. - Peter Grönquist; Chengyuan Yao; Tal Ben-Nun; Nikoli Dryden; Peter Dueben; Shigang Li 002; Torsten Hoefler (CoRR, 2020)

Shapeshifter Networks: Cross-layer Parameter Sharing for Scalable and Effective Deep Learning. - Bryan A. Plummer; Nikoli Dryden; Julius Frost; Torsten Hoefler; Kate Saenko (CoRR, 2020)

Data Movement Is All You Need: A Case Study on Optimizing Transformers. - Andrei Ivanov; Nikoli Dryden; Tal Ben-Nun; Shigang Li 002; Torsten Hoefler (MLSys, 2021)

Clairvoyant prefetching for distributed machine learning I/O. - Nikoli Dryden; Roman Böhringer; Tal Ben-Nun; Torsten Hoefler (SC, 2021)

Learning Combinatorial Node Labeling Algorithms. - Lukas Gianinazzi; Maximilian Fries; Nikoli Dryden; Tal Ben-Nun; Maciej Besta; Torsten Hoefler (CoRR, 2021)

Co-design Center for Exascale Machine Learning Technologies (ExaLearn). - Francis J. Alexander; James A. Ang; Jenna A. Bilbrey; Jan Balewski; Tiernan Casey; Ryan Chard; Jong Choi 001; Sutanay Choudhury; Bert J. Debusschere; Anthony M. DeGennaro; Nikoli Dryden; J. Austin Ellis; et al. (Int. J. High Perform. Comput. Appl., 2021)

Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks. - Torsten Hoefler; Dan Alistarh; Tal Ben-Nun; Nikoli Dryden; Alexandra Peste (J. Mach. Learn. Res., 2021)

Breaking (Global) Barriers in Parallel Stochastic Optimization With Wait-Avoiding Group Averaging. - Shigang Li 002; Tal Ben-Nun; Giorgi Nadiradze; Salvatore Di Girolamo; Nikoli Dryden; Dan Alistarh; Torsten Hoefler (IEEE Trans. Parallel Distributed Syst., 2021)

The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs With Hybrid Parallelism. - Yosuke Oyama; Naoya Maruyama; Nikoli Dryden; Erin McCarthy; Peter Harrington; Jan Balewski; Satoshi Matsuoka; Peter Nugent; Brian Van Essen (IEEE Trans. Parallel Distributed Syst., 2021)

Neural Parameter Allocation Search. - Bryan A. Plummer; Nikoli Dryden; Julius Frost; Torsten Hoefler; Kate Saenko (ICLR, 2022)

A data-centric optimization framework for machine learning. - Oliver Rausch; Tal Ben-Nun; Nikoli Dryden; Andrei Ivanov; Shigang Li 002; Torsten Hoefler (ICS, 2022)

Motif Prediction with Graph Neural Networks. - Maciej Besta; Raphael Grob; Cesare Miglioli; Nicola Bernold; Grzegorz Kwasniewski; Gabriel Gjini; Raghavendra Kanakagiri; Saleh Ashkboos; Lukas Gianinazzi; Nikoli Dryden; Torsten Hoefler (KDD, 2022)

Neural Graph Databases. - Maciej Besta; Patrick Iff; Florian Scheidl; Kazuki Osawa; Nikoli Dryden; Michal Podstawski; Tiancheng Chen; Torsten Hoefler (LoG, 2022)

ENS-10: A Dataset For Post-Processing Ensemble Weather Forecasts. - Saleh Ashkboos; Langwen Huang; Nikoli Dryden; Tal Ben-Nun; Peter Dueben; Lukas Gianinazzi; Luca Kummer; Torsten Hoefler (NeurIPS, 2022)

Spatial Mixture-of-Experts. - Nikoli Dryden; Torsten Hoefler (NeurIPS, 2022)

ENS-10: A Dataset For Post-Processing Ensemble Weather Forecast. - Saleh Ashkboos; Langwen Huang; Nikoli Dryden; Tal Ben-Nun; Peter Dueben; Lukas Gianinazzi; Luca Kummer; Torsten Hoefler (CoRR, 2022)

STen: Productive and Efficient Sparsity in PyTorch. - Andrei Ivanov; Nikoli Dryden; Tal Ben-Nun; Saleh Ashkboos; Torsten Hoefler (CoRR, 2023)

Cached Operator Reordering: A Unified View for Fast GNN Training. - Julia Bazinska; Andrei Ivanov; Tal Ben-Nun; Nikoli Dryden; Maciej Besta; Siyuan Shen; Torsten Hoefler (CoRR, 2023)

Learning to Compose SuperWeights for Neural Parameter Allocation Search. - Piotr Teterwak; Soren Nelson; Nikoli Dryden; Dina Bashkirova; Kate Saenko; Bryan A. Plummer (WACV, 2024)

SuperGCN: General and Scalable Framework for GCN Training on CPU-powered Supercomputers. - Chen Zhuang; Peng Chen 035; Xin Liu 020; Rio Yokota; Nikoli Dryden; Toshio Endo; Satoshi Matsuoka; Mohamed Wahib (CoRR, 2024)

Lion Cub: Minimizing Communication Overhead in Distributed Lion. - Satoki Ishikawa; Tal Ben-Nun; Brian Van Essen; Rio Yokota; Nikoli Dryden (CoRR, 2024)

A General and Scalable GCN Training Framework on CPU Supercomputers. - Chen Zhuang; Peng Chen 035; Xin Liu 020; Rio Yokota; Nikoli Dryden; Lingqi Zhang 001; Toshio Endo; Satoshi Matsuoka; Mohamed Wahib (PPoPP, 2025)

Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity. - Susav Shrestha; Brad Settlemyer; Nikoli Dryden; Narasimha Reddy (CoRR, 2025)