Hao-Jun Michael Shi

According to our database, Hao-Jun Michael Shi authored at least 15 papers between 2016 and 2026.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2026

Clarifying Shampoo: Adapting Spectral Descent to Stochasticity and the Parameter Trajectory.

Runa Eschenhagen, Anna Cai, Tsung-Hsien Lee, Hao-Jun Michael Shi

CoRR, February, 2026

Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent.

Hiroki Naganuma, Shagun Gupta, Youssef Briki, Ioannis Mitliagkas, Irina Rish, Parameswaran Raman, Hao-Jun Michael Shi

CoRR, February, 2026

2025

Smoothing DiLoCo with Primal Averaging for Faster Training of LLMs.

Aaron Defazio, Konstantin Mishchenko, Parameswaran Raman, Hao-Jun Michael Shi, Lin Xiao

CoRR, December, 2025

SNOO: Step-K Nesterov Outer Optimizer - The Surprising Effectiveness of Nesterov Momentum Applied to Pseudo-Gradients.

Dominik Kallusky, Vinay Rao, Vishal Nandavanam, Hao-Jun Michael Shi

CoRR, October, 2025

Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditioner.

Runa Eschenhagen, Aaron Defazio, Tsung-Hsien Lee, Richard E. Turner, Hao-Jun Michael Shi

CoRR, June, 2025

2023

On the numerical performance of finite-difference-based methods for derivative-free optimization.

Hao-Jun Michael Shi, Melody Qiming Xuan, Figen Öztoprak, Jorge Nocedal

Optim. Methods Softw., March, 2023

A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale.

Hao-Jun Michael Shi, Tsung-Hsien Lee, Shintaro Iwasaki, Jose Gallego-Posada, Zhijing Li, Kaushik Rangadurai, Dheevatsa Mudigere, Michael Rabbat

CoRR, 2023

2022

Adaptive Finite-Difference Interval Estimation for Noisy Derivative-Free Optimization.

Hao-Jun Michael Shi, Yuchen Xie, Melody Qiming Xuan, Jorge Nocedal

SIAM J. Sci. Comput., August, 2022

A Noise-Tolerant Quasi-Newton Algorithm for Unconstrained Optimization.

Hao-Jun Michael Shi, Yuchen Xie, Richard H. Byrd, Jorge Nocedal

SIAM J. Optim., 2022

2020

Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems.

Hao-Jun Michael Shi, Dheevatsa Mudigere, Maxim Naumov, Jiyan Yang

Proceedings of the KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2020

2019

Deep Learning Recommendation Model for Personalization and Recommendation Systems.

Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherniavskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, Volodymyr Kondratenko, Stephanie Pereira, Xianjie Chen, Wenlin Chen, Vijay Rao, Bill Jia, Liang Xiong, Misha Smelyanskiy

CoRR, 2019

2018

Optimizing Quantization for Lasso Recovery.

Xiaoyi Gu, Shenyinying Tu, Hao-Jun Michael Shi, Mindy Case, Deanna Needell, Yaniv Plan

IEEE Signal Process. Lett., 2018

A Progressive Batching L-BFGS Method for Machine Learning.

Raghu Bollapragada, Dheevatsa Mudigere, Jorge Nocedal, Hao-Jun Michael Shi, Ping Tak Peter Tang

Proceedings of the 35th International Conference on Machine Learning, 2018

2016

Practical Algorithms for Learning Near-Isometric Linear Embeddings.

Jerry Luo, Kayla Shapiro, Hao-Jun Michael Shi, Qi Yang, Kan Zhu

CoRR, 2016

Methods for quantized compressed sensing.

Hao-Jun Michael Shi, Mindy Case, Xiaoyi Gu, Shenyinying Tu, Deanna Needell

Proceedings of the 2016 Information Theory and Applications Workshop, 2016