Neural gradients are lognormally distributed: understanding sparse and quantized training. - Brian Chmiel; Liad Ben-Uri; Moran Shkolnik; Elad Hoffer; Ron Banner; Daniel Soudry (CoRR, 2020)

Neural gradients are near-lognormal: improved quantized and sparse training. - Brian Chmiel; Liad Ben-Uri; Moran Shkolnik; Elad Hoffer; Ron Banner; Daniel Soudry (ICLR, 2021)