A Practical Dynamic Buffer Overflow Detector. - Olatunji Ruwase; Monica S. Lam (NDSS, 2004)

Flexible Hardware Acceleration for Instruction-Grain Program Monitoring. - Shimin Chen; Michael Kozuch; Theodoros Strigkos; Babak Falsafi; Phillip B. Gibbons; Todd C. Mowry; Vijaya Ramachandran; Olatunji Ruwase; Michael P. Ryan; Evangelos Vlachos (ISCA, 2008)

Ditto: a system for opportunistic caching in multi-hop wireless networks. - Fahad R. Dogar; Amar Phanishayee; Himabindu Pucha; Olatunji Ruwase; David G. Andersen (MobiCom, 2008)

Parallelizing dynamic information flow tracking. - Olatunji Ruwase; Phillip B. Gibbons; Todd C. Mowry; Vijaya Ramachandran; Shimin Chen; Michael Kozuch; Michael P. Ryan (SPAA, 2008)

Flexible Hardware Acceleration for Instruction-Grain Lifeguards. - Shimin Chen; Michael Kozuch; Phillip B. Gibbons; Michael P. Ryan; Theodoros Strigkos; Todd C. Mowry; Olatunji Ruwase; Evangelos Vlachos; Babak Falsafi; Vijaya Ramachandran (IEEE Micro, 2009)

Decoupled lifeguards: enabling path optimizations for dynamic correctness checking tools. - Olatunji Ruwase; Shimin Chen; Phillip B. Gibbons; Todd C. Mowry (PLDI, 2010)

Improving Device Driver Reliability through Decoupled Dynamic Binary Analyses. - Olatunji Ruwase (2013)

Guardrail: a high fidelity approach to protecting hardware devices from buggy drivers. - Olatunji Ruwase; Michael A. Kozuch; Phillip B. Gibbons; Todd C. Mowry (ASPLOS, 2014)

Toward accelerating deep learning at scale using specialized hardware in the datacenter. - Kalin Ovtcharov; Olatunji Ruwase; Joo-Young Kim 001; Jeremy Fowers; Karin Strauss; Eric S. Chung (Hot Chips Symposium, 2015)

Page overlays: an enhanced virtual memory framework to enable fine-grained memory management. - Vivek Seshadri; Gennady Pekhimenko; Olatunji Ruwase; Onur Mutlu; Phillip B. Gibbons; Michael A. Kozuch; Todd C. Mowry; Trishul M. Chilimbi (ISCA, 2015)

Performance Modeling and Scalability Optimization of Distributed Deep Learning Systems. - Feng Yan 001; Olatunji Ruwase; Yuxiong He; Trishul M. Chilimbi (KDD, 2015)

SERF: efficient scheduling for fast deep neural network serving via judicious parallelism. - Feng Yan 001; Yuxiong He; Olatunji Ruwase; Evgenia Smirni (SC, 2016)

Optimizing CNNs on Multicores for Scalability, Performance and Goodput. - Samyam Rajbhandari; Yuxiong He; Olatunji Ruwase; Michael Carbin; Trishul M. Chilimbi (ASPLOS, 2017)

HyperDrive: exploring hyperparameters with POP scheduling. - Jeff Rasley; Yuxiong He; Feng Yan 001; Olatunji Ruwase; Rodrigo Fonseca (Middleware, 2017)

Efficient Deep Neural Network Serving: Fast and Furious. - Feng Yan 001; Yuxiong He; Olatunji Ruwase; Evgenia Smirni (IEEE Trans. Netw. Serv. Manag., 2018)

Accelerating Large Scale Deep Learning Inference through DeepCPU at Microsoft. - Minjia Zhang; Samyam Rajbhandari; Wenhan Wang; Elton Zheng; Olatunji Ruwase; Jeff Rasley; Jason Li; Junhua Wang; Yuxiong He (OpML, 2019)

ZeRO: Memory Optimization Towards Training A Trillion Parameter Models. - Samyam Rajbhandari; Jeff Rasley; Olatunji Ruwase; Yuxiong He (CoRR, 2019)

LSTM-Sharp: An Adaptable, Energy-Efficient Hardware Accelerator for Long Short-Term Memory. - Reza Yazdani; Olatunji Ruwase; Minjia Zhang; Yuxiong He; José-María Arnau; Antonio González 001 (CoRR, 2019)

DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters. - Jeff Rasley; Samyam Rajbhandari; Olatunji Ruwase; Yuxiong He (KDD, 2020)

ZeRO: memory optimizations toward training trillion parameter models. - Samyam Rajbhandari; Jeff Rasley; Olatunji Ruwase; Yuxiong He (SC, 2020)

SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement. - Heyang Qin; Samyam Rajbhandari; Olatunji Ruwase; Feng Yan 001; Lei Yang 001; Yuxiong He (NeurIPS, 2021)

ZeRO-infinity: breaking the GPU memory wall for extreme scale deep learning. - Samyam Rajbhandari; Olatunji Ruwase; Jeff Rasley; Shaden Smith; Yuxiong He (SC, 2021)

ZeRO-Offload: Democratizing Billion-Scale Model Training. - Jie Ren 015; Samyam Rajbhandari; Reza Yazdani Aminabadi; Olatunji Ruwase; Shuangyan Yang; Minjia Zhang; Dong Li 001; Yuxiong He (USENIX ATC, 2021)

DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale. - Reza Yazdani Aminabadi; Samyam Rajbhandari; Ammar Ahmad Awan; Cheng Li 001; Du Li; Elton Zheng; Olatunji Ruwase; Shaden Smith; Minjia Zhang; Jeff Rasley; Yuxiong He (SC, 2022)

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. - Teven Le Scao; Angela Fan; Christopher Akiki; Ellie Pavlick; Suzana Ilic; Daniel Hesslow; Roman Castagné; Alexandra Sasha Luccioni; François Yvon; Matthias Gallé; Jonathan Tow; Alexander M. Rush; et al. (CoRR, 2022)

A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training. - Siddharth Singh; Olatunji Ruwase; Ammar Ahmad Awan; Samyam Rajbhandari; Yuxiong He; Abhinav Bhatele (ICS, 2023)

A Novel Tensor-Expert Hybrid Parallelism Approach to Scale Mixture-of-Experts Training. - Siddharth Singh; Olatunji Ruwase; Ammar Ahmad Awan; Samyam Rajbhandari; Yuxiong He; Abhinav Bhatele (CoRR, 2023)

ZeRO++: Extremely Efficient Collective Communication for Giant Model Training. - Guanhua Wang; Heyang Qin; Sam Ade Jacobs; Connor Holmes; Samyam Rajbhandari; Olatunji Ruwase; Feng Yan 001; Lei Yang 001; Yuxiong He (CoRR, 2023)

DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales. - Zhewei Yao; Reza Yazdani Aminabadi; Olatunji Ruwase; Samyam Rajbhandari; Xiaoxia Wu; Ammar Ahmad Awan; Jeff Rasley; Minjia Zhang; Conglong Li; Connor Holmes; Zhongzhu Zhou; Michael Wyatt; et al. (CoRR, 2023)

DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention. - Zhewei Yao; Xiaoxia Wu; Conglong Li; Minjia Zhang; Heyang Qin; Olatunji Ruwase; Ammar Ahmad Awan; Samyam Rajbhandari; Yuxiong He (CoRR, 2023)

ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks. - Xiaoxia Wu; Haojun Xia; Stephen Youn; Zhen Zheng; Shiyang Chen 004; Arash Bakhtiari; Michael Wyatt; Reza Yazdani Aminabadi; Yuxiong He; Olatunji Ruwase; Leon Song; Zhewei Yao (CoRR, 2023)

SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks. - Reza Yazdani Aminabadi; Olatunji Ruwase; Minjia Zhang; Yuxiong He; José-María Arnau; Antonio González 001 (ACM Trans. Embed. Comput. Syst., 2023)

ZeRO++: Extremely Efficient Collective Communication for Large Model Training. - Guanhua Wang; Heyang Qin; Sam Ade Jacobs; Xiaoxia Wu; Connor Holmes; Zhewei Yao; Samyam Rajbhandari; Olatunji Ruwase; Feng Yan 001; Lei Yang 001; Yuxiong He (ICLR, 2024)

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding. - Zhenyu Zhang 015; Runjin Chen; Shiwei Liu 003; Zhewei Yao; Olatunji Ruwase; Beidi Chen; Xiaoxia Wu; Zhangyang Wang (NeurIPS, 2024)

RecFlex: Enabling Feature Heterogeneity-Aware Optimization for Deep Recommendation Models with Flexible Schedules. - Zaifeng Pan; Zhen Zheng; Feng Zhang 007; Bing Xie; Ruofan Wu; Shaden Smith; Chuanjie Liu; Olatunji Ruwase; Xiaoyong Du 001; Yufei Ding 001 (SC, 2024)

Quant-LLM: Accelerating the Serving of Large Language Models via FP6-Centric Algorithm-System Co-Design on Modern GPUs. - Haojun Xia; Zhen Zheng; Xiaoxia Wu; Shiyang Chen 004; Zhewei Yao; Stephen Youn; Arash Bakhtiari; Michael Wyatt; Donglin Zhuang; Zhongzhu Zhou; Olatunji Ruwase; Yuxiong He; et al. (USENIX ATC, 2024)

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design. - Haojun Xia; Zhen Zheng; Xiaoxia Wu; Shiyang Chen 004; Zhewei Yao; Stephen Youn; Arash Bakhtiari; Michael Wyatt; Donglin Zhuang; Zhongzhu Zhou; Olatunji Ruwase; Yuxiong He; et al. (CoRR, 2024)

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. - Marah I Abdin; Sam Ade Jacobs; Ammar Ahmad Awan; Jyoti Aneja; Ahmed Awadallah 001; Hany Awadalla; Nguyen Bach; Amit Bahree; Arash Bakhtiari; Harkirat S. Behl; Alon Benhaim; Misha Bilenko; et al. (CoRR, 2024)

FastPersist: Accelerating Model Checkpointing in Deep Learning. - Guanhua Wang; Olatunji Ruwase; Bing Xie; Yuxiong He (CoRR, 2024)

Universal Checkpointing: Efficient and Flexible Checkpointing for Large Scale Distributed Training. - Xinyu Lian; Sam Ade Jacobs; Lev Kurilenko; Masahiro Tanaka; Stas Bekman; Olatunji Ruwase; Minjia Zhang (CoRR, 2024)

Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer. - Jinghan Yao; Sam Ade Jacobs; Masahiro Tanaka; Olatunji Ruwase; Aamir Shafi; Hari Subramoni; Dhabaleswar K. Panda 001 (CoRR, 2024)

Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping. - Guanhua Wang; Chengming Zhang; Zheyu Shen; Ang Li; Olatunji Ruwase (CoRR, 2024)

Mojito: Motion Trajectory and Intensity Control for Video Generation. - Xuehai He; Shuohang Wang; Jianwei Yang; Xiaoxia Wu; Yiping Wang; Kuan Wang; Zheng Zhan 004; Olatunji Ruwase; Yelong Shen; Xin Eric Wang (CoRR, 2024)

Universal Checkpointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Parallelism. - Xinyu Lian; Sam Ade Jacobs; Lev Kurilenko; Masahiro Tanaka; Stas Bekman; Olatunji Ruwase; Minjia Zhang (USENIX ATC, 2025)

DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training. - Masahiro Tanaka; Du Li; Umesh Chand; Ali Zafar; Haiying Shen; Olatunji Ruwase (CoRR, 2025)