Secure communications between bandwidth brokers. - Bu-Sung Lee; Wing-Keong Woo; Chai Kiat Yeo; Teck Meng Lim; Bee-Hwa Lim; Yuxiong He; Jie Song 005 (ACM SIGOPS Oper. Syst. Rev., 2004)

An Empirical Evaluation ofWork Stealing with Parallelism Feedback. - Kunal Agrawal 001; Yuxiong He; Charles E. Leiserson (ICDCS, 2006)

Provably Efficient Two-Level Adaptive Scheduling. - Yuxiong He; Wen-Jing Hsu; Charles E. Leiserson (JSSPP, 2006)

Adaptive Scheduling of Parallel Jobs on Functionally Heterogeneous Resources. - Yuxiong He; Hongyang Sun 001; Wen-Jing Hsu (ICPP, 2007)

Adaptive Scheduling with Parallelism Feedback. - Kunal Agrawal 001; Yuxiong He; Wen-Jing Hsu; Charles E. Leiserson (IPDPS, 2007)

Provably Efficient Online Non-clairvoyant Adaptive Scheduling. - Yuxiong He; Wen-Jing Hsu; Charles E. Leiserson (IPDPS, 2007)

Adaptive work stealing with parallelism feedback. - Kunal Agrawal 001; Yuxiong He; Charles E. Leiserson (PPoPP, 2007)

Adaptive work-stealing with parallelism feedback. - Kunal Agrawal 001; Charles E. Leiserson; Yuxiong He; Wen-Jing Hsu (ACM Trans. Comput. Syst., 2008)

Provably Efficient Online Nonclairvoyant Adaptive Scheduling. - Yuxiong He; Wen-Jing Hsu; Charles E. Leiserson (IEEE Trans. Parallel Distributed Syst., 2008)

The Cilkview scalability analyzer. - Yuxiong He; Charles E. Leiserson; William M. Leiserson (SPAA, 2010)

Energy-Efficient Multiprocessor Scheduling for Flow Time and Makespan - Hongyang Sun 001; Yuxiong He; Wen-Jing Hsu (CoRR, 2010)

Improved results for scheduling batched parallel jobs by using a generalized analysis framework. - Yuxiong He; Hongyang Sun 001; Wen-Jing Hsu (J. Parallel Distributed Comput., 2010)

Position Paper: Embracing Heterogeneity - Improving Energy Efficiency for Interactive Services on Heterogeneous Data Center Hardware. - Yuxiong He; Sameh Elnikety (AI for Data Center Management and Cloud Computing, 2011)

Scheduling for data center interactive services. - Yuxiong He; Sameh Elnikety (Allerton, 2011)

Tians Scheduling: Using Partial Processing in Best-Effort Applications. - Yuxiong He; Sameh Elnikety; Hongyang Sun 001 (ICDCS, 2011)

Scheduling Functionally Heterogeneous Systems with Utilization Balancing. - Yuxiong He; Jie Liu; Hongyang Sun 001 (IPDPS, 2011)

Speed Scaling for Energy and Performance with Instantaneous Parallelism. - Hongyang Sun 001; Yuxiong He; Wen-Jing Hsu (TAPAS, 2011)

G-SPARQL: a hybrid engine for querying large attributed graphs. - Sherif Sakr; Sameh Elnikety; Yuxiong He (CIKM, 2012)

Zeta: scheduling interactive services with partial execution. - Yuxiong He; Sameh Elnikety; James R. Larus; Chenyu Yan (SoCC, 2012)

Budget-based control for interactive services with adaptive execution. - Yuxiong He; Zihao Ye; Qiang Fu 015; Sameh Elnikety (ICAC, 2012)

Provably-Efficient Job Scheduling for Energy and Fairness in Geographically Distributed Data Centers. - Shaolei Ren; Yuxiong He; Fei Xu (ICDCS, 2012)

Horton: Online Query Execution Engine for Large Distributed Graphs. - Mohamed Sarwat; Sameh Elnikety; Yuxiong He; Gabriel Kliot (ICDE, 2012)

QACO: exploiting partial execution in web servers. - Jinhan Kim; Sameh Elnikety; Yuxiong He; Seung-won Hwang; Shaolei Ren (CAC, 2013)

Topic 3: Scheduling and Load Balancing - (Introduction). - Zhihui Du; Ramin Yahyapour; Yuxiong He; Nectarios Koziris; Bilha Mendelson; Veronika Sonigo; Achim Streit; Andrei Tchernykh (Euro-Par, 2013)

Adaptive parallelism for web search. - Myeongjae Jeon; Yuxiong He; Sameh Elnikety; Alan L. Cox; Scott Rixner (EuroSys, 2013)

Exploiting Processor Heterogeneity in Interactive Services. - Shaolei Ren; Yuxiong He; Sameh Elnikety; Kathryn S. McKinley (ICAC, 2013)

Performance Inconsistency in Large Scale Data Processing Clusters. - Mingyuan Xia 001; Nan Zhu; Sameh Elnikety; Xue Liu 001; Yuxiong He (ICAC, 2013)

Power-effiicent resource allocation in MapReduce clusters. - Kaiqi Xiong; Yuxiong He (IM, 2013)

Energy-Efficient Scheduling for Best-Effort Interactive Services to Achieve High Response Quality. - Zhihui Du; Hongyang Sun 001; Yuxiong He; Yu He; David A. Bader; Huazhe Zhang (IPDPS, 2013)

COCA: online distributed resource management for cost minimization and carbon neutrality in data centers. - Shaolei Ren; Yuxiong He (SC, 2013)

Solving Graph Isomorphism Using Parameterized Matching. - Juan Mendivelso; Sunghwan Kim; Sameh Elnikety; Yuxiong He; Seung-won Hwang; Yoan J. Pinzón (SPIRE, 2013)

Horton+: A Distributed System for Processing Declarative Reachability Queries over Partitioned Graphs. - Mohamed Sarwat; Sameh Elnikety; Yuxiong He; Mohamed F. Mokbel (Proc. VLDB Endow., 2013)

Mars: Real-time spatio-temporal queries on microblogs. - Amr Magdy 001; Ahmed M. Aly; Mohamed F. Mokbel; Sameh Elnikety; Yuxiong He; Suman Nath (ICDE, 2014)

Mercury: A memory-constrained spatio-temporal real-time search on microblogs. - Amr Magdy 001; Mohamed F. Mokbel; Sameh Elnikety; Suman Nath; Yuxiong He (ICDE, 2014)

Predictive parallelization: taming tail latencies in web search. - Myeongjae Jeon; Saehoon Kim; Seung-won Hwang; Yuxiong He; Sameh Elnikety; Alan L. Cox; Scott Rixner (SIGIR, 2014)

A Theoretical Foundation for Scheduling and Designing Heterogeneous Processors for Interactive Applications. - Shaolei Ren; Yuxiong He; Kathryn S. McKinley (DISC, 2014)

Hybrid query execution engine for large attributed graphs. - Sherif Sakr; Sameh Elnikety; Yuxiong He (Inf. Syst., 2014)

Energy-efficient multiprocessor scheduling for flow time and makespan. - Hongyang Sun 001; Yuxiong He; Wen-Jing Hsu; Rui Fan 004 (Theor. Comput. Sci., 2014)

Few-to-Many: Incremental Parallelism for Reducing Tail Latency in Interactive Services. - Md. Enamul Haque; Yong Hun Eom; Yuxiong He; Sameh Elnikety; Ricardo Bianchini; Kathryn S. McKinley (ASPLOS, 2015)

Measuring and Managing Answer Quality for Online Data-Intensive Services. - Jaimie Kelley; Christopher Stewart; Nathaniel Morris; Devesh Tiwari; Yuxiong He; Sameh Elnikety (ICAC, 2015)

Performance Modeling and Scalability Optimization of Distributed Deep Learning Systems. - Feng Yan 001; Olatunji Ruwase; Yuxiong He; Trishul M. Chilimbi (KDD, 2015)

BATS: Budget-Constrained Autoscaling for Cloud Performance Optimization. - A. Hasan Mahmud; Yuxiong He; Shaolei Ren (MASCOTS, 2015)

Optimal Aggregation Policy for Reducing Tail Latency of Web Search. - Jeong-Min Yun; Yuxiong He; Sameh Elnikety; Shaolei Ren (SIGIR, 2015)

Delayed-Dynamic-Selective (DDS) Prediction for Reducing Extreme Tail Latency in Web Search. - Saehoon Kim; Yuxiong He; Seung-won Hwang; Sameh Elnikety; Seungjin Choi (WSDM, 2015)

Processing and Optimizing Main Memory Spatial-Keyword Queries. - Taesung Lee; Jin-Woo Park; Sanghoon Lee; Seung-won Hwang; Sameh Elnikety; Yuxiong He (Proc. VLDB Endow., 2015)

Online Resource Management for Carbon-Neutral Cloud Computing. - Kishwar Ahmed; Shaolei Ren; Yuxiong He; Athanasios V. Vasilakos (Handbook on Data Centers, 2015)

TPC: Target-Driven Parallelism Combining Prediction and Correction to Reduce Tail Latency in Interactive Services. - Myeongjae Jeon; Yuxiong He; Hwanju Kim; Sameh Elnikety; Scott Rixner; Alan L. Cox (ASPLOS, 2016)

GeoTrend: spatial trending queries on real-time microblogs. - Amr Magdy 001; Ahmed M. Aly; Mohamed F. Mokbel; Sameh Elnikety; Yuxiong He; Suman Nath; Walid G. Aref (SIGSPATIAL/GIS, 2016)

Work stealing for interactive services to meet target latency. - Jing Li 025; Kunal Agrawal 001; Sameh Elnikety; Yuxiong He; I-Ting Angelina Lee; Chenyang Lu 001; Kathryn S. McKinley (PPoPP, 2016)

SERF: efficient scheduling for fast deep neural network serving via judicious parallelism. - Feng Yan 001; Yuxiong He; Olatunji Ruwase; Evgenia Smirni (SC, 2016)

Venus: Scalable Real-Time Spatial Queries on Microblogs with Adaptive Load Shedding. - Amr Magdy 001; Mohamed F. Mokbel; Sameh Elnikety; Suman Nath; Yuxiong He (IEEE Trans. Knowl. Data Eng., 2016)

Prediction and Predictability for Search Query Acceleration. - Seung-won Hwang; Saehoon Kim; Yuxiong He; Sameh Elnikety; Seungjin Choi (ACM Trans. Web, 2016)

Optimizing CNNs on Multicores for Scalability, Performance and Goodput. - Samyam Rajbhandari; Yuxiong He; Olatunji Ruwase; Michael Carbin; Trishul M. Chilimbi (ASPLOS, 2017)

Workload analysis and caching strategies for search advertising systems. - Conglong Li; David G. Andersen; Qiang Fu; Sameh Elnikety; Yuxiong He (SoCC, 2017)

When Good Enough Is Better: Energy-Aware Scheduling for Multicore Servers. - Xinning Hui; Zhihui Du; Jason Liu 001; Hongyang Sun 001; Yuxiong He; David A. Bader (IPDPS Workshops, 2017)

Exploiting heterogeneity for tail latency and energy efficiency. - Md. Enamul Haque; Yuxiong He; Sameh Elnikety; Thu D. Nguyen; Ricardo Bianchini; Kathryn S. McKinley (MICRO, 2017)

Swayam: distributed autoscaling to meet SLAs of machine learning inference services with resource efficiency. - Arpan Gujarati; Sameh Elnikety; Yuxiong He; Kathryn S. McKinley; Björn B. Brandenburg (Middleware, 2017)

HyperDrive: exploring hyperparameters with POP scheduling. - Jeff Rasley; Yuxiong He; Feng Yan 001; Olatunji Ruwase; Rodrigo Fonseca (Middleware, 2017)

BitFunnel: Revisiting Signatures for Search. - Bob Goodwin; Michael Hopcroft; Dan Luu; Alex Clemmer; Mihaela Curmei; Sameh Elnikety; Yuxiong He (SIGIR, 2017)

Optimal Reissue Policies for Reducing Tail Latency. - Tim Kaler; Yuxiong He; Sameh Elnikety (SPAA, 2017)

Learning Intrinsic Sparse Structures within Long Short-term Memory. - Wei Wen; Yuxiong He; Samyam Rajbhandari; Wenhan Wang; Fang Liu; Bin Hu; Yiran Chen 001; Hai Li 001 (CoRR, 2017)

Obtaining and Managing Answer Quality for Online Data-Intensive Services. - Jaimie Kelley; Christopher Stewart; Nathaniel Morris; Devesh Tiwari; Yuxiong He; Sameh Elnikety (ACM Trans. Model. Perform. Evaluation Comput. Syst., 2017)

Learning Intrinsic Sparse Structures within Long Short-Term Memory. - Wei Wen; Yuxiong He; Samyam Rajbhandari; Minjia Zhang; Wenhan Wang; Fang Liu; Bin Hu; Yiran Chen 001; Hai Li 001 (ICLR (Poster), 2018)

Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models. - Minjia Zhang; Wenhan Wang; Xiaodong Liu 003; Jianfeng Gao 001; Yuxiong He (NeurIPS, 2018)

DeepCPU: Serving RNN-based Deep Learning Models 10x Faster. - Minjia Zhang; Samyam Rajbhandari; Wenhan Wang; Yuxiong He (USENIX ATC, 2018)

Better Caching in Search Advertising Systems with Rapid Refresh Predictions. - Conglong Li; David G. Andersen; Qiang Fu; Sameh Elnikety; Yuxiong He (WWW, 2018)

Zoom: SSD-based Vector Search for Optimizing Accuracy, Latency and Memory. - Minjia Zhang; Yuxiong He (CoRR, 2018)

Stochastic Modeling and Optimization of Stragglers. - Farshid Farhat; Diman Zad Tootaghaj; Yuxiong He; Anand Sivasubramaniam; Mahmut T. Kandemir; Chita R. Das (IEEE Trans. Cloud Comput., 2018)

Efficient Deep Neural Network Serving: Fast and Furious. - Feng Yan 001; Yuxiong He; Olatunji Ruwase; Evgenia Smirni (IEEE Trans. Netw. Serv. Manag., 2018)

GRIP: Multi-Store Capacity-Optimized High-Performance Nearest Neighbor Search for Vector Search Engine. - Minjia Zhang; Yuxiong He (CIKM, 2019)

GRNN: Low-Latency and Scalable RNN Inference on GPUs. - Connor Holmes; Daniel Mawhirter; Yuxiong He; Feng Yan 001; Bo Wu 002 (EuroSys, 2019)

Fast LSTM Inference by Dynamic Decomposition on Cloud Systems. - Yang You 001; Yuxiong He; Samyam Rajbhandari; Wenhan Wang; Cho-Jui Hsieh; Kurt Keutzer; James Demmel (ICDM, 2019)

Deep Learning Inference Service at Microsoft. - Jonathan Soifer; Jason Li; Mingqin Li; Jeffrey Zhu; Yingnan Li; Yuxiong He; Elton Zheng; Adi Oltean; Maya Mosyak; Chris Barnes; Thomas Liu; Junhua Wang (OpML, 2019)

Accelerating Large Scale Deep Learning Inference through DeepCPU at Microsoft. - Minjia Zhang; Samyam Rajbhandari; Wenhan Wang; Elton Zheng; Olatunji Ruwase; Jeff Rasley; Jason Li; Junhua Wang; Yuxiong He (OpML, 2019)

AntMan: Sparse Low-Rank Compression to Accelerate RNN inference. - Samyam Rajbhandari; Harsh Shrivastava 001; Yuxiong He (CoRR, 2019)

ZeRO: Memory Optimization Towards Training A Trillion Parameter Models. - Samyam Rajbhandari; Jeff Rasley; Olatunji Ruwase; Yuxiong He (CoRR, 2019)

LSTM-Sharp: An Adaptable, Energy-Efficient Hardware Accelerator for Long Short-Term Memory. - Reza Yazdani; Olatunji Ruwase; Minjia Zhang; Yuxiong He; José-María Arnau; Antonio González 001 (CoRR, 2019)

Communication-Aware Scheduling of Precedence-Constrained Tasks. - Yu Su 013; Xiaoqi Ren; Shai Vardi; Adam Wierman; Yuxiong He (SIGMETRICS Perform. Evaluation Rev., 2019)

DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters. - Jeff Rasley; Samyam Rajbhandari; Olatunji Ruwase; Yuxiong He (KDD, 2020)

Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping. - Minjia Zhang; Yuxiong He (NeurIPS, 2020)

ZeRO: memory optimizations toward training trillion parameter models. - Samyam Rajbhandari; Jeff Rasley; Olatunji Ruwase; Yuxiong He (SC, 2020)

Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination. - Conglong Li; Minjia Zhang; David G. Andersen; Yuxiong He (SIGMOD Conference, 2020)

APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm. - Hanlin Tang; Shaoduo Gan; Samyam Rajbhandari; Xiangru Lian; Ji Liu 002; Yuxiong He; Ce Zhang 001 (CoRR, 2020)

Local trend discovery on real-time microblogs with uncertain locations in tight memory environments. - Abdulaziz Almaslukh; Amr Magdy 001; Ahmed M. Aly; Mohamed F. Mokbel; Sameh Elnikety; Yuxiong He; Suman Nath; Walid G. Aref (GeoInformatica, 2020)

Fast LSTM by dynamic decomposition on cloud and distributed systems. - Yang You 001; Yuxiong He; Samyam Rajbhandari; Wenhan Wang; Cho-Jui Hsieh; Kurt Keutzer; James Demmel (Knowl. Inf. Syst., 2020)

1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed. - Hanlin Tang; Shaoduo Gan; Ammar Ahmad Awan; Samyam Rajbhandari; Conglong Li; Xiangru Lian; Ji Liu 002; Ce Zhang 001; Yuxiong He (ICML, 2021)

NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM. - Connor Holmes; Minjia Zhang; Yuxiong He; Bo Wu 002 (NeurIPS, 2021)

SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement. - Heyang Qin; Samyam Rajbhandari; Olatunji Ruwase; Feng Yan 001; Lei Yang 001; Yuxiong He (NeurIPS, 2021)

ZeRO-infinity: breaking the GPU memory wall for extreme scale deep learning. - Samyam Rajbhandari; Olatunji Ruwase; Jeff Rasley; Shaden Smith; Yuxiong He (SC, 2021)

ZeRO-Offload: Democratizing Billion-Scale Model Training. - Jie Ren 015; Samyam Rajbhandari; Reza Yazdani Aminabadi; Olatunji Ruwase; Shuangyan Yang; Minjia Zhang; Dong Li 001; Yuxiong He (USENIX ATC, 2021)

Curriculum Learning: A Regularization Method for Efficient and Stable Billion-Scale GPT Model Pre-Training. - Conglong Li; Minjia Zhang; Yuxiong He (CoRR, 2021)

Scalable and Efficient MoE Training for Multitask Multilingual Models. - Young Jin Kim 001; Ammar Ahmad Awan; Alexandre Muzio; Andrés Felipe Cruz-Salinas; Liyang Lu; Amr Hendy; Samyam Rajbhandari; Yuxiong He; Hany Hassan Awadalla (CoRR, 2021)

Adversarial Data Augmentation for Task-Specific Knowledge Distillation of Pre-trained Transformers. - Minjia Zhang; Uma-Naresh Niranjan; Yuxiong He (AAAI, 2022)

Task Offloading Based on GRU Model in IoT. - Xiao Zhang; Yuxiong He; Youhuai Wang; Xiaoming Chen 005; Shi Jin 005; Ye Liang 004 (EITCE, 2022)

1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed. - Conglong Li; Ammar Ahmad Awan; Hanlin Tang; Samyam Rajbhandari; Yuxiong He (HIPC, 2022)

DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale. - Samyam Rajbhandari; Conglong Li; Zhewei Yao; Minjia Zhang; Reza Yazdani Aminabadi; Ammar Ahmad Awan; Jeff Rasley; Yuxiong He (ICML, 2022)

The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models. - Conglong Li; Minjia Zhang; Yuxiong He (NeurIPS, 2022)

XTC: Extreme Compression for Pre-trained Transformers Made Simple and Efficient. - Xiaoxia Wu; Zhewei Yao; Minjia Zhang; Conglong Li; Yuxiong He (NeurIPS, 2022)

ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers. - Zhewei Yao; Reza Yazdani Aminabadi; Minjia Zhang; Xiaoxia Wu; Conglong Li; Yuxiong He (NeurIPS, 2022)

DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale. - Reza Yazdani Aminabadi; Samyam Rajbhandari; Ammar Ahmad Awan; Cheng Li 001; Du Li; Elton Zheng; Olatunji Ruwase; Shaden Smith; Minjia Zhang; Jeff Rasley; Yuxiong He (SC, 2022)

GraSP: Optimizing Graph-based Nearest Neighbor Search with Subgraph Sampling and Pruning. - Minjia Zhang; Wenhan Wang; Yuxiong He (WSDM, 2022)

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model. - Shaden Smith; Mostofa Patwary; Brandon Norick; Patrick LeGresley; Samyam Rajbhandari; Jared Casper; Zhun Liu; Shrimai Prabhumoye; George Zerveas; Vijay Korthikanti; Elton Zheng; Rewon Child; et al. (CoRR, 2022)

ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noise. - Minjia Zhang; Uma-Naresh Niranjan; Yuxiong He (CoRR, 2022)

Extreme Compression for Pre-trained Transformers Made Simple and Efficient. - Xiaoxia Wu; Zhewei Yao; Minjia Zhang; Conglong Li; Yuxiong He (CoRR, 2022)

Compressing Pre-trained Transformers via Low-Bit NxM Sparsity for Natural Language Understanding. - Connor Holmes; Minjia Zhang; Yuxiong He; Bo Wu 002 (CoRR, 2022)

BiFeat: Supercharge GNN Training via Graph Feature Quantization. - Yuxin Ma; Ping Gong 009; Jun Yi; Zhewei Yao; Minjie Wang; Cheng Li 001; Yuxiong He; Feng Yan 001 (CoRR, 2022)

Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformers. - Zhewei Yao; Xiaoxia Wu; Conglong Li; Connor Holmes; Minjia Zhang; Cheng Li 001; Yuxiong He (CoRR, 2022)

Revisiting the Efficiency-Accuracy Tradeoff in Adapting Transformer Models via Adversarial Fine-Tuning. - Minjia Zhang; Uma-Naresh Niranjan; Yuxiong He (ECAI, 2023)

Scaling Vision-Language Models with Sparse Mixture of Experts. - Sheng Shen 001; Zhewei Yao; Chunyuan Li; Trevor Darrell; Kurt Keutzer; Yuxiong He (EMNLP (Findings), 2023)

Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam. - Yucheng Lu; Conglong Li; Minjia Zhang; Christopher De Sa; Yuxiong He (ICLR, 2023)

DySR: Adaptive Super-Resolution via Algorithm and System Co-design. - Syed Zawad; Cheng Li 001; Zhewei Yao; Elton Zheng; Yuxiong He; Feng Yan 001 (ICLR, 2023)

Understanding Int4 Quantization for Language Models: Latency Speedup, Composability, and Failure Cases. - Xiaoxia Wu; Cheng Li 001; Reza Yazdani Aminabadi; Zhewei Yao; Yuxiong He (ICML, 2023)

HEAT: A Highly Efficient and Affordable Training System for Collaborative Filtering Based Recommendation on CPUs. - Chengming Zhang 006; Shaden Smith; Baixi Sun; Jiannan Tian; Jonathan Soifer; Xiaodong Yu 001; Shuaiwen Leon Song; Yuxiong He; Dingwen Tao (ICS, 2023)

A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training. - Siddharth Singh; Olatunji Ruwase; Ammar Ahmad Awan; Samyam Rajbhandari; Yuxiong He; Abhinav Bhatele (ICS, 2023)

MCR-DL: Mix-and-Match Communication Runtime for Deep Learning. - Quentin Anthony; Ammar Ahmad Awan; Jeff Rasley; Yuxiong He; Aamir Shafi; Mustafa Abduljabbar; Hari Subramoni; Dhabaleswar K. Panda 001 (IPDPS, 2023)

Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases. - Xiaoxia Wu; Cheng Li 001; Reza Yazdani Aminabadi; Zhewei Yao; Yuxiong He (CoRR, 2023)

A Novel Tensor-Expert Hybrid Parallelism Approach to Scale Mixture-of-Experts Training. - Siddharth Singh; Olatunji Ruwase; Ammar Ahmad Awan; Samyam Rajbhandari; Yuxiong He; Abhinav Bhatele (CoRR, 2023)

A Comprehensive Study on Post-Training Quantization for Large Language Models. - Zhewei Yao; Cheng Li 001; Xiaoxia Wu; Stephen Youn; Yuxiong He (CoRR, 2023)

Selective Guidance: Are All the Denoising Steps of Guided Diffusion Important? - Pareesa Ameneh Golnari; Zhewei Yao; Yuxiong He (CoRR, 2023)

ZeRO++: Extremely Efficient Collective Communication for Giant Model Training. - Guanhua Wang; Heyang Qin; Sam Ade Jacobs; Connor Holmes; Samyam Rajbhandari; Olatunji Ruwase; Feng Yan 001; Lei Yang 001; Yuxiong He (CoRR, 2023)

ZeroQuant-FP: A Leap Forward in LLMs Post-Training W4A8 Quantization Using Floating-Point Formats. - Xiaoxia Wu; Zhewei Yao; Yuxiong He (CoRR, 2023)

DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales. - Zhewei Yao; Reza Yazdani Aminabadi; Olatunji Ruwase; Samyam Rajbhandari; Xiaoxia Wu; Ammar Ahmad Awan; Jeff Rasley; Minjia Zhang; Conglong Li; Connor Holmes; Zhongzhu Zhou; Michael Wyatt; et al. (CoRR, 2023)

DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention. - Zhewei Yao; Xiaoxia Wu; Conglong Li; Minjia Zhang; Heyang Qin; Olatunji Ruwase; Ammar Ahmad Awan; Samyam Rajbhandari; Yuxiong He (CoRR, 2023)

DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models. - Sam Ade Jacobs; Masahiro Tanaka; Chengming Zhang 006; Minjia Zhang; Shuaiwen Leon Song; Samyam Rajbhandari; Yuxiong He (CoRR, 2023)

DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies. - Shuaiwen Leon Song; Bonnie Kruft; Minjia Zhang; Conglong Li; Shiyang Chen 004; Chengming Zhang 006; Masahiro Tanaka; Xiaoxia Wu; Jeff Rasley; Ammar Ahmad Awan; Connor Holmes; Martin Cai; et al. (CoRR, 2023)

ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers. - Zhewei Yao; Reza Yazdani Aminabadi; Stephen Youn; Xiaoxia Wu; Elton Zheng; Yuxiong He (CoRR, 2023)

ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks. - Xiaoxia Wu; Haojun Xia; Stephen Youn; Zhen Zheng; Shiyang Chen 004; Arash Bakhtiari; Michael Wyatt; Reza Yazdani Aminabadi; Yuxiong He; Olatunji Ruwase; Leon Song; Zhewei Yao (CoRR, 2023)

SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks. - Reza Yazdani Aminabadi; Olatunji Ruwase; Minjia Zhang; Yuxiong He; José-María Arnau; Antonio González 001 (ACM Trans. Embed. Comput. Syst., 2023)

DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing. - Conglong Li; Zhewei Yao; Xiaoxia Wu; Minjia Zhang; Connor Holmes; Cheng Li 001; Yuxiong He (AAAI, 2024)

Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation. - Zhewei Yao; Xiaoxia Wu; Cheng Li 001; Stephen Youn; Yuxiong He (AAAI, 2024)

ZeRO++: Extremely Efficient Collective Communication for Large Model Training. - Guanhua Wang; Heyang Qin; Sam Ade Jacobs; Xiaoxia Wu; Connor Holmes; Zhewei Yao; Samyam Rajbhandari; Olatunji Ruwase; Feng Yan 001; Lei Yang 001; Yuxiong He (ICLR, 2024)

System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models. - Sam Ade Jacobs; Masahiro Tanaka; Chengming Zhang 006; Minjia Zhang; Reza Yazdani Aminabadi; Shuaiwen Leon Song; Samyam Rajbhandari; Yuxiong He (IPDPS (Workshops), 2024)

System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models. - Sam Ade Jacobs; Masahiro Tanaka; Chengming Zhang 006; Minjia Zhang; Reza Yazdani Aminadabi; Shuaiwen Leon Song; Samyam Rajbhandari; Yuxiong He (PODC, 2024)

Quant-LLM: Accelerating the Serving of Large Language Models via FP6-Centric Algorithm-System Co-Design on Modern GPUs. - Haojun Xia; Zhen Zheng; Xiaoxia Wu; Shiyang Chen 004; Zhewei Yao; Stephen Youn; Arash Bakhtiari; Michael Wyatt; Donglin Zhuang; Zhongzhu Zhou; Olatunji Ruwase; Yuxiong He; et al. (USENIX ATC, 2024)

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference. - Connor Holmes; Masahiro Tanaka; Michael Wyatt; Ammar Ahmad Awan; Jeff Rasley; Samyam Rajbhandari; Reza Yazdani Aminabadi; Heyang Qin; Arash Bakhtiari; Lev Kurilenko; Yuxiong He (CoRR, 2024)

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design. - Haojun Xia; Zhen Zheng; Xiaoxia Wu; Shiyang Chen 004; Zhewei Yao; Stephen Youn; Arash Bakhtiari; Michael Wyatt; Donglin Zhuang; Zhongzhu Zhou; Olatunji Ruwase; Yuxiong He; et al. (CoRR, 2024)

FastPersist: Accelerating Model Checkpointing in Deep Learning. - Guanhua Wang; Olatunji Ruwase; Bing Xie; Yuxiong He (CoRR, 2024)

SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation. - Aurick Qiao; Zhewei Yao; Samyam Rajbhandari; Yuxiong He (CoRR, 2024)

STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning. - Jaeseong Lee 002; Seung-won Hwang; Aurick Qiao; Daniel F. Campos; Zhewei Yao; Yuxiong He (ACL (1), 2025)

Optimizing Reasoning for Text-to-SQL with Execution Feedback. - Bohan Zhai; Canwen Xu; Yuxiong He; Zhewei Yao (ACL (Findings), 2025)

ConvCodeWorld: Benchmarking Conversational Code Generation in Reproducible Feedback Environments. - Hojae Han; Seung-won Hwang; Rajhans Samdani; Yuxiong He (ICLR, 2025)

Inference Scaling for Bridging Retrieval and Augmented Generation. - Youngwon Lee 003; Seung-won Hwang; Daniel F. Campos; Filip Gralinski; Zhewei Yao; Yuxiong He (NAACL (Findings), 2025)

CORD: Balancing COnsistency and Rank Distillation for Robust Retrieval-Augmented Generation. - Youngwon Lee 003; Seung-won Hwang; Daniel F. Campos; Filip Gralinski; Zhewei Yao; Yuxiong He (NAACL (Short Papers), 2025)

FedCust: Offloading hyperparameter customization for federated learning. - Syed Zawad; Xiaolong Ma; Jun Yi; Cheng Li 001; Minjia Zhang; Lei Yang 001; Feng Yan 001; Yuxiong He (Perform. Evaluation, 2025)

Agentic Verification for Ambiguous Query Disambiguation. - Youngwon Lee 003; Seung-won Hwang; Ruofan Wu; Feng Yan 001; Danmei Xu; Moutasem Akkad; Zhewei Yao; Yuxiong He (CoRR, 2025)

ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedback. - Bohan Zhai; Canwen Xu; Yuxiong He; Zhewei Yao (CoRR, 2025)

RenAIssance: A Survey Into AI Text-to-Image Generation in the Era of Large Model. - Fengxiang Bie; Yibo Yang; Zhongzhu Zhou; Adam Ghanem; Minjia Zhang; Zhewei Yao; Xiaoxia Wu; Connor Holmes; Pareesa Ameneh Golnari; David A. Clifton; Yuxiong He; Dacheng Tao; et al. (IEEE Trans. Pattern Anal. Mach. Intell., 2025)

Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL. - Zhewei Yao; Guoheng Sun; Lukasz Borchmann; Zheyu Shen; Minghang Deng; Bohan Zhai; Hao Zhang; Ang Li; Yuxiong He (CoRR, 2025)

ComposeRAG: A Modular and Composable RAG for Corpus-Grounded Multi-Hop Question Answering. - Ruofan Wu; Youngwon Lee 003; Fan Shu; Danmei Xu; Seung-won Hwang; Zhewei Yao; Yuxiong He; Feng Yan 001 (CoRR, 2025)

MedOrch: Medical Diagnosis with Tool-Augmented Reasoning Agents for Flexible Extensibility. - Yexiao He; Ang Li; Boyi Liu; Zhewei Yao; Yuxiong He (CoRR, 2025)

Arctic Long Sequence Training: Scalable And Efficient Training For Multi-Million Token Sequences. - Stas Bekman; Samyam Rajbhandari; Michael Wyatt; Jeff Rasley; Tunji Ruwase; Zhewei Yao; Aurick Qiao; Yuxiong He (CoRR, 2025)

Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AI. - Samyam Rajbhandari; Mert Hidayetoglu; Aurick Qiao; Ye Wang; Juncheng Yang; Jeff Rasley; Michael Wyatt; Yuxiong He (CoRR, 2025)

Snowpark: Performant, Secure, User-Friendly Data Engineering and AI/ML Next To Your Data. - Brandon Baker; Elliott Brossard; Chenwei Xie; Zihao Ye; Deen Liu; Yijun Xie; Arthur Zwiegincew; Nitya Kumar Sharma; Gaurav Jain; Eugene Retunsky; Mike Halcrow; Derek Denny-Brown; et al. (CoRR, 2025)