Daniel Soudry

According to our database, Daniel Soudry authored at least 76 papers between 2010 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers.

Gon Buzaglo, Itamar Harel, Mor Shpigel Nacson, Alon Brutzkus, Nathan Srebro, Daniel Soudry

CoRR, 2024

Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators.

Yaniv Blumenfeld, Itay Hubara, Daniel Soudry

CoRR, 2024

The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting - An Analytical Model.

Daniel Goldfarb, Itay Evron, Nir Weinberger, Daniel Soudry, Paul Hand

CoRR, 2024

2023

Explore to Generalize in Zero-Shot RL.

Ev Zisselman, Itai Lavie, Daniel Soudry, Aviv Tamar

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

How do Minimum-Norm Shallow Denoisers Look in Function Space?

Chen Zeno, Greg Ongie, Yaniv Blumenfeld, Nir Weinberger, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

DropCompute: simple and more robust distributed synchronous training via compute variance reduction.

Niv Giladi, Shahar Gottlieb, Moran Shkolnik, Asaf Karnieli, Ron Banner, Elad Hoffer, Kfir Y. Levy, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond.

Itai Kreisler, Mor Shpigel Nacson, Daniel Soudry, Yair Carmon

Proceedings of the International Conference on Machine Learning, 2023

Continual Learning in Linear Classification on Separable Data.

Itay Evron, Edward Moroshko, Gon Buzaglo, Maroun Khriesh, Badea Marjieh, Nathan Srebro, Daniel Soudry

Proceedings of the International Conference on Machine Learning, 2023

The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks.

Mor Shpigel Nacson, Rotem Mulayoff, Greg Ongie, Tomer Michaeli, Daniel Soudry

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Minimum Variance Unbiased N: M Sparsity for the Neural Gradients.

Brian Chmiel, Itay Hubara, Ron Banner, Daniel Soudry

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats.

Brian Chmiel, Ron Banner, Elad Hoffer, Hilla Ben-Yaacov, Daniel Soudry

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activations.

Hagay Michaeli, Tomer Michaeli, Daniel Soudry

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

The Role of Codeword-to-Class Assignments in Error-Correcting Codes: An Empirical Study.

Itay Evron, Ophir Onn, Tamar Weiss Orzech, Hai Azeroual, Daniel Soudry

Proceedings of the International Conference on Artificial Intelligence and Statistics, 2023

2022

Optimal Fine-Grained N: M sparsity for Activations and Neural Gradients.

Brian Chmiel, Itay Hubara, Ron Banner, Daniel Soudry

CoRR, 2022

Implicit Bias of the Step Size in Linear Diagonal Neural Networks.

Mor Shpigel Nacson, Kavya Ravichandran, Nathan Srebro, Daniel Soudry

Proceedings of the International Conference on Machine Learning, 2022

A Statistical Framework for Efficient Out of Distribution Detection in Deep Neural Networks.

Matan Haroush, Tzviel Frostig, Ruth Heller, Daniel Soudry

Proceedings of the Tenth International Conference on Learning Representations, 2022

How catastrophic can catastrophic forgetting be in linear regression?

Itay Evron, Edward Moroshko, Rachel A. Ward, Nathan Srebro, Daniel Soudry

Proceedings of the Conference on Learning Theory, 2-5 July 2022, London, UK., 2022

Regularization Guarantees Generalization in Bayesian Reinforcement Learning through Algorithmic Stability.

Aviv Tamar, Daniel Soudry, Ev Zisselman

Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence, 2022

2021

Task-Agnostic Continual Learning Using Online Variational Bayes With Fixed-Point Updates.

Chen Zeno, Itay Golan, Elad Hoffer, Daniel Soudry

Neural Comput., 2021

Logarithmic Unbiased Quantization: Practical 4-bit Training in Deep Learning.

Brian Chmiel, Ron Banner, Elad Hoffer, Hilla Ben-Yaacov, Daniel Soudry

CoRR, 2021

Statistical Testing for Efficient Out of Distribution Detection in Deep Neural Networks.

Matan Haroush, Tzviel Frostig, Ruth Heller, Daniel Soudry

CoRR, 2021

The Implicit Bias of Minima Stability: A View from Function Space.

Rotem Mulayoff, Tomer Michaeli, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N: M Transposable Masks.

Itay Hubara, Brian Chmiel, Moshe Island, Ron Banner, Joseph Naor, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Physics-Aware Downsampling with Deep Learning for Scalable Flood Modeling.

Niv Giladi, Zvika Ben-Haim, Sella Nevo, Yossi Matias, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Accurate Post Training Quantization With Small Calibration Sets.

Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, Daniel Soudry

Proceedings of the 38th International Conference on Machine Learning, 2021

On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent.

Shahar Azulay, Edward Moroshko, Mor Shpigel Nacson, Blake E. Woodworth, Nathan Srebro, Amir Globerson, Daniel Soudry

Proceedings of the 38th International Conference on Machine Learning, 2021

Neural gradients are near-lognormal: improved quantized and sparse training.

Brian Chmiel, Liad Ben-Uri, Moran Shkolnik, Elad Hoffer, Ron Banner, Daniel Soudry

Proceedings of the 9th International Conference on Learning Representations, 2021

2020

The Global Optimization Geometry of Shallow Linear Neural Networks.

Zhihui Zhu, Daniel Soudry, Yonina C. Eldar, Michael B. Wakin

J. Math. Imaging Vis., 2020

Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming.

Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, Daniel Soudry

CoRR, 2020

Neural gradients are lognormally distributed: understanding sparse and quantized training.

Brian Chmiel, Liad Ben-Uri, Moran Shkolnik, Elad Hoffer, Ron Banner, Daniel Soudry

CoRR, 2020

Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy.

Edward Moroshko, Blake E. Woodworth, Suriya Gunasekar, Jason D. Lee, Nati Srebro, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?

Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry

Proceedings of the 37th International Conference on Machine Learning, 2020

A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case.

Greg Ongie, Rebecca Willett, Daniel Soudry, Nathan Srebro

Proceedings of the 8th International Conference on Learning Representations, 2020

At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?

Niv Giladi, Mor Shpigel Nacson, Elad Hoffer, Daniel Soudry

Proceedings of the 8th International Conference on Learning Representations, 2020

Augment Your Batch: Improving Generalization Through Instance Repetition.

Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, Daniel Soudry

Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

The Knowledge Within: Methods for Data-Free Model Compression.

Matan Haroush, Itay Hubara, Elad Hoffer, Daniel Soudry

Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

Kernel and Rich Regimes in Overparametrized Models.

Blake E. Woodworth, Suriya Gunasekar, Jason D. Lee, Edward Moroshko, Pedro Savarese, Itay Golan, Daniel Soudry, Nathan Srebro

Proceedings of the Conference on Learning Theory, 2020

2019

MTJ-Based Hardware Synapse Design for Quantized Deep Neural Networks.

Tzofnat Greenberg-Toledo, Ben Perach, Daniel Soudry, Shahar Kvatinsky

CoRR, 2019

Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency.

Elad Hoffer, Berry Weinstein, Itay Hubara, Tal Ben-Nun, Torsten Hoefler, Daniel Soudry

CoRR, 2019

Augment your batch: better training with larger batches.

Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, Daniel Soudry

CoRR, 2019

A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off.

Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Post training 4-bit quantization of convolutional networks for rapid-deployment.

Ron Banner, Yury Nahshan, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models.

Mor Shpigel Nacson, Suriya Gunasekar, Jason D. Lee, Nathan Srebro, Daniel Soudry

Proceedings of the 36th International Conference on Machine Learning, 2019

How do infinite width bounded norm networks look in function space?

Pedro Savarese, Itay Evron, Daniel Soudry, Nathan Srebro

Proceedings of the Conference on Learning Theory, 2019

Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate.

Mor Shpigel Nacson, Nathan Srebro, Daniel Soudry

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, 2019

Convergence of Gradient Descent on Separable Data.

Mor Shpigel Nacson, Jason D. Lee, Suriya Gunasekar, Pedro Henrique Pamplona Savarese, Nathan Srebro, Daniel Soudry

Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, 2019

2018

Seizure pathways: A model-based investigation.

Philippa J. Karoly, Levin Kuhlmann, Daniel Soudry, David B. Grayden, Mark J. Cook, Dean R. Freestone

PLoS Comput. Biol., 2018

The Implicit Bias of Gradient Descent on Separable Data.

Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, Nathan Srebro

J. Mach. Learn. Res., 2018

ACIQ: Analytical Clipping for Integer Quantization of neural networks.

Ron Banner, Yury Nahshan, Elad Hoffer, Daniel Soudry

CoRR, 2018

Bayesian Gradient Descent: Online Variational Bayes Learning with Increased Robustness to Catastrophic Forgetting and Weight Pruning.

Chen Zeno, Itay Golan, Elad Hoffer, Daniel Soudry

CoRR, 2018

Convergence of Gradient Descent on Separable Data.

Mor Shpigel Nacson, Jason D. Lee, Suriya Gunasekar, Nathan Srebro, Daniel Soudry

CoRR, 2018

On the Blindspots of Convolutional Networks.

Elad Hoffer, Shai Fine, Daniel Soudry

CoRR, 2018

Norm matters: efficient and accurate normalization schemes in deep networks.

Elad Hoffer, Ron Banner, Itay Golan, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

Implicit Bias of Gradient Descent on Linear Convolutional Networks.

Suriya Gunasekar, Jason D. Lee, Daniel Soudry, Nati Srebro

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

Scalable methods for 8-bit training of neural networks.

Ron Banner, Itay Hubara, Elad Hoffer, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

Characterizing Implicit Bias in Terms of Optimization Geometry.

Suriya Gunasekar, Jason D. Lee, Daniel Soudry, Nathan Srebro

Proceedings of the 35th International Conference on Machine Learning, 2018

The Implicit Bias of Gradient Descent on Separable Data.

Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Nathan Srebro

Proceedings of the 6th International Conference on Learning Representations, 2018

Exponentially vanishing sub-optimal local minima in multilayer neural networks.

Daniel Soudry, Elad Hoffer

Proceedings of the 6th International Conference on Learning Representations, 2018

Fix your classifier: the marginal value of training the last weight layer.

Elad Hoffer, Itay Hubara, Daniel Soudry

Proceedings of the 6th International Conference on Learning Representations, 2018

2017

Multi-scale approaches for high-speed imaging and analysis of large neural populations.

Johannes Friedrich, Weijian Yang, Daniel Soudry, Yu Mu, Misha B. Ahrens, Rafael Yuste, Darcy S. Peterka, Liam Paninski

PLoS Comput. Biol., 2017

Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations.

Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, Yoshua Bengio

J. Mach. Learn. Res., 2017

The Implicit Bias of Gradient Descent on Separable Data.

Daniel Soudry, Elad Hoffer, Nathan Srebro

CoRR, 2017

Train longer, generalize better: closing the generalization gap in large batch training of neural networks.

Elad Hoffer, Itay Hubara, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

2016

No bad local minima: Data independent training error guarantees for multilayer neural networks.

Daniel Soudry, Yair Carmon

CoRR, 2016

Binarized Neural Networks.

Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, Yoshua Bengio

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

A fully analog memristor-based neural network with online gradient training.

Eyal Rosenthal, Sergey Greshnikov, Daniel Soudry, Shahar Kvatinsky

Proceedings of the IEEE International Symposium on Circuits and Systems, 2016

2015

Memristor-Based Multilayer Neural Networks With Online Gradient Descent Training.

Daniel Soudry, Dotan Di Castro, Asaf Gal, Avinoam Kolodny, Shahar Kvatinsky

IEEE Trans. Neural Networks Learn. Syst., 2015

Efficient "Shotgun" Inference of Neural Connectivity from Highly Sub-sampled Activity Data.

Daniel Soudry, Suraj Keshri, Patrick Stinson, Min-hwan Oh, Garud Iyengar, Liam Paninski

PLoS Comput. Biol., 2015

Training Binary Multilayer Neural Networks for Image Classification using Expectation Backpropagation.

Zhiyong Cheng, Daniel Soudry, Zexi Mao, Zhen-zhong Lan

CoRR, 2015

2014

The neuronal response at extended timescales: a linearized spiking input-output relation.

Daniel Soudry, Ron Meir

Frontiers Comput. Neurosci., 2014

The neuronal response at extended timescales: long-term correlations without long-term memory.

Daniel Soudry, Ron Meir

Frontiers Comput. Neurosci., 2014

Diffusion approximation-based simulation of stochastic ion channels: which method to use?

Danilo Pezo, Daniel Soudry, Patricio Orio

Frontiers Comput. Neurosci., 2014

Expectation Backpropagation: Parameter-Free Training of Multilayer Neural Networks with Continuous or Discrete Weights.

Daniel Soudry, Itay Hubara, Ron Meir

Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 2014

2012

Conductance-Based Neuron Models and the Slow Dynamics of Excitability.

Daniel Soudry, Ron Meir

Frontiers Comput. Neurosci., 2012

"Neuronal spike generation mechanism as an oversampling, noise-shaping A-to-D converter".

Dmitri B. Chklovskii, Daniel Soudry

Proceedings of the Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3-6, 2012

2010

History-Dependent Dynamics in a Generic Model of Ion Channels - An Analytic Study.

Daniel Soudry, Ron Meir

Frontiers Comput. Neurosci., 2010