Jonathan Frankle

David Rolnick

CoRR, 2024

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws.

[BibT_eX]

[DOI]

Nikhil Sardana

CoRR, 2024

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws.

[BibT_eX]

[DOI]

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Dynamic Masking Rate Schedules for MLM Pretraining.

[BibT_eX]

[DOI]

Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, 2024

2023

The Lottery Ticket Hypothesis: On Sparse, Trainable Neural Networks

[BibT_eX]

[DOI]

PhD thesis, 2023

Report of the 1st Workshop on Generative AI and Law.

[BibT_eX]

[DOI]

CoRR, 2023

CommonCanvas: An Open Diffusion Model Trained with Creative-Commons Images.

[BibT_eX]

[DOI]

CoRR, 2023

Knowledge Distillation for Efficient Sequences of Training Runs.

[BibT_eX]

[DOI]

CoRR, 2023

MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?

[BibT_eX]

[DOI]

Proceedings of the Eleventh International Conference on Learning Representations, 2023

2022

The Effect of Data Dimensionality on Neural Network Prunability.

[BibT_eX]

[DOI]

Zachary Ankner

Alex Renda

Tian Jin

CoRR, 2022

Reduce, Reuse, Recycle: Improving Training Efficiency with Distillation.

[BibT_eX]

[DOI]

CoRR, 2022

Non-Determinism and the Lawlessness of ML Code.

[BibT_eX]

[DOI]

A. Feder Cooper

Christopher De Sa

CoRR, 2022

Fast Benchmarking of Accuracy vs. Training Time with Cyclic Learning Rates.

[BibT_eX]

[DOI]

CoRR, 2022

Strengthening Subcommunities: Towards Sustainable Growth in AI Research.

[BibT_eX]

[DOI]

CoRR, 2022

Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Pruning's Effect on Generalization Through the Lens of Training and Regularization.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

What Can Linear Interpolation of Neural Network Loss Landscapes Tell Us?

[BibT_eX]

[DOI]

Tiffany J. Vlaar

Proceedings of the International Conference on Machine Learning, 2022

Non-Determinism and the Lawlessness of Machine Learning Code.

[BibT_eX]

[DOI]

A. Feder Cooper

Jose Javier Gonzalez Ortiz

Christopher De Sa

Proceedings of the 2022 Symposium on Computer Science and Law, 2022

2021

Trade-offs of Local SGD at Scale: An Empirical Study.

[BibT_eX]

[DOI]

CoRR, 2021

Studying the Consistency and Composability of Lottery Ticket Pruning Masks.

[BibT_eX]

[DOI]

Rajiv Movva

CoRR, 2021

On the Predictability of Pruning Across Scales.

[BibT_eX]

[DOI]

Jonathan S. Rosenfeld

Nir Shavit

Proceedings of the 38th International Conference on Machine Learning, 2021

Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs.

[BibT_eX]

[DOI]

David J. Schwab

Ari S. Morcos

Proceedings of the 9th International Conference on Learning Representations, 2021

Pruning Neural Networks at Initialization: Why Are We Missing the Mark?

[BibT_eX]

[DOI]

Daniel M. Roy

Proceedings of the 9th International Conference on Learning Representations, 2021

The Lottery Tickets Hypothesis for Supervised and Self-Supervised Pre-Training in Computer Vision Models.

[BibT_eX]

[DOI]

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021

2020

Revisiting "Qualitatively Characterizing Neural Network Optimization Problems".

[BibT_eX]

[DOI]

CoRR, 2020

Are all negatives created equal in contrastive instance discrimination?

[BibT_eX]

[DOI]

CoRR, 2020

TIRAMISU: A Polyhedral Compiler for Dense and Sparse Deep Learning.

[BibT_eX]

[DOI]

Riyadh Baghdadi

Abdelkader Nadir Debbagh

Kamel Abdous

Fatima-Zohra Benhamida

Alex Renda

Jonathan Elliott Frankle

Jose Javier Gonzalez Ortiz

Saman P. Amarasinghe

CoRR, 2020

The Lottery Ticket Hypothesis for Pre-trained BERT Networks.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

What is the State of Neural Network Pruning?

[BibT_eX]

[DOI]

Davis W. Blalock

John V. Guttag

Proceedings of the Third Conference on Machine Learning and Systems, 2020

Linear Mode Connectivity and the Lottery Ticket Hypothesis.

[BibT_eX]

[DOI]

Daniel M. Roy

Proceedings of the 37th International Conference on Machine Learning, 2020

Comparing Rewinding and Fine-tuning in Neural Network Pruning.

[BibT_eX]

[DOI]

Alex Renda

Proceedings of the 8th International Conference on Learning Representations, 2020

The Early Phase of Neural Network Training.

[BibT_eX]

[DOI]

David J. Schwab

Ari S. Morcos

Proceedings of the 8th International Conference on Learning Representations, 2020

2019

Dissecting Pruned Neural Networks.

[BibT_eX]

[DOI]

David Bau

CoRR, 2019

The Lottery Ticket Hypothesis at Scale.

[BibT_eX]

[DOI]

Daniel M. Roy

CoRR, 2019

The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks.

[BibT_eX]

[DOI]

Proceedings of the 7th International Conference on Learning Representations, 2019

2018

Practical Accountability of Secret Processes.

[BibT_eX]

[DOI]

IACR Cryptol. ePrint Arch., 2018

The Lottery Ticket Hypothesis: Training Pruned Neural Networks.

[BibT_eX]

[DOI]

CoRR, 2018

2016

Example-directed synthesis: a type-theoretic interpretation.

[BibT_eX]

[DOI]

Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, 2016

2015

Type-Directed Synthesis of Products.

[BibT_eX]

[DOI]