Justin Gilmer

According to our database, Justin Gilmer authored at least 40 papers between 2015 and 2023.

Collaborative distances:

• Dijkstra number of three.
• Erdős number of two.

2023

Small-scale proxies for large-scale Transformer training instabilities.

Mitchell Wortsman, Peter J. Liu, Lechao Xiao, Katie Everett, Alex Alemi, Ben Adlam, John D. Co-Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein, Kelvin Xu, Jaehoon Lee, Justin Gilmer, Simon Kornblith

CoRR, 2023

Replacing softmax with ReLU in Vision Transformers.

Mitchell Wortsman, Jaehoon Lee, Justin Gilmer, Simon Kornblith

CoRR, 2023

Benchmarking Neural Network Training Algorithms.

George E. Dahl, Frank Schneider, Zachary Nado, Naman Agarwal, Chandramouli Shama Sastry, Philipp Hennig, Sourabh Medapati, Runa Eschenhagen, Priya Kasimbeg, Daniel Suo, Juhan Bae, Justin Gilmer, Abel L. Peirson, Bilal Khan, Rohan Anil, Mike Rabbat, Shankar Krishnan, Daniel Snider, Ehsan Amid, Kongtao Chen, Chris J. Maddison, Rakshith Vasudev, Michal Badura, Ankush Garg, Peter Mattson

CoRR, 2023

Scaling Vision Transformers to 22 Billion Parameters.

Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, Rodolphe Jenatton, Lucas Beyer, Michael Tschannen, Anurag Arnab, Xiao Wang, Carlos Riquelme, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin F. Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Patrick Collier, Alexey A. Gritsenko, Vighnesh Birodkar, Cristina N. Vasconcelos, Yi Tay, Thomas Mensink, Alexander Kolesnikov, Filip Pavetic, Dustin Tran, Thomas Kipf, Mario Lucic, Xiaohua Zhai, Daniel Keysers, Jeremiah Harmsen, Neil Houlsby

CoRR, 2023

Order Matters in the Presence of Dataset Imbalance for Multilingual Learning.

Dami Choi, Derrick Xin, Hamid Dadkhahi, Justin Gilmer, Ankush Garg, Orhan Firat, Chih-Kuan Yeh, Andrew M. Dai, Behrooz Ghorbani

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Improving Training Stability for Multitask Ranking Models in Recommender Systems.

Jiaxi Tang, Yoel Drori, Daryl Chang, Maheswaran Sathiamoorthy, Justin Gilmer, Li Wei, Xinyang Yi, Lichan Hong, Ed H. Chi

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023

Scaling Vision Transformers to 22 Billion Parameters.

Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, Rodolphe Jenatton, Lucas Beyer, Michael Tschannen, Anurag Arnab, Xiao Wang, Carlos Riquelme Ruiz, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin Fathy Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Collier, Alexey A. Gritsenko, Vighnesh Birodkar, Cristina Nader Vasconcelos, Yi Tay, Thomas Mensink, Alexander Kolesnikov, Filip Pavetic, Dustin Tran, Thomas Kipf, Mario Lucic, Xiaohua Zhai, Daniel Keysers, Jeremiah J. Harmsen, Neil Houlsby

Proceedings of the International Conference on Machine Learning, 2023

2022

Adaptive Gradient Methods at the Edge of Stability.

Jeremy Cohen, Behrooz Ghorbani, Shankar Krishnan, Naman Agarwal, Sourabh Medapati, Michal Badura, Daniel Suo, David Cardoze, Zachary Nado, George E. Dahl, Justin Gilmer

CoRR, 2022

Pre-training helps Bayesian optimization too.

Zi Wang, George E. Dahl, Kevin Swersky, Chansoo Lee, Zelda Mariet, Zachary Nado, Justin Gilmer, Jasper Snoek, Zoubin Ghahramani

CoRR, 2022

AI system for fetal ultrasound in low-resource settings.

Ryan G. Gomes, Bellington Vwalika, Chace Lee, Angelica Willis, Marcin Sieniek, Joan T. Price, Christina Chen, Margaret P. Kasaro, James A. Taylor, Elizabeth M. Stringer, Scott Mayer McKinney, Ntazana Sindano, George E. Dahl, William Goodnight III, Justin Gilmer, Benjamin H. Chi, Charles Lau, Terry Spitz, T. Saensuksopa, Kris Liu, Jonny Wong, Rory Pilgrim, Akib Uddin, Greg Corrado, Lily Peng, Katherine Chou, Daniel Tse, Jeffrey S. A. Stringer, Shravya Shetty

CoRR, 2022

Do Current Multi-Task Optimization Methods in Deep Learning Even Help?

Derrick Xin, Behrooz Ghorbani, Justin Gilmer, Ankush Garg, Orhan Firat

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

A Loss Curvature Perspective on Training Instabilities of Deep Learning Models.

Justin Gilmer, Behrooz Ghorbani, Ankush Garg, Sneha Kudugunta, Behnam Neyshabur, David Cardoze, George Edward Dahl, Zachary Nado, Orhan Firat

Proceedings of the Tenth International Conference on Learning Representations, 2022

Predicting the utility of search spaces for black-box optimization: a simple, budget-aware approach.

Setareh Ariafar, Justin Gilmer, Zachary Nado, Jasper Snoek, Rodolphe Jenatton, George E. Dahl

Proceedings of the International Conference on Artificial Intelligence and Statistics, 2022

2021

A Loss Curvature Perspective on Training Instability in Deep Learning.

Justin Gilmer, Behrooz Ghorbani, Ankush Garg, Sneha Kudugunta, Behnam Neyshabur, David Cardoze, George E. Dahl, Zachary Nado, Orhan Firat

CoRR, 2021

Automatic prior selection for meta Bayesian optimization with a case study on tuning deep neural network optimizers.

Zi Wang, George E. Dahl, Kevin Swersky, Chansoo Lee, Zelda Mariet, Zachary Nado, Justin Gilmer, Jasper Snoek, Zoubin Ghahramani

CoRR, 2021

A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes.

Zachary Nado, Justin Gilmer, Christopher J. Shallue, Rohan Anil, George E. Dahl

CoRR, 2021

The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization.

Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, Justin Gilmer

Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, 2021

2020

AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty.

Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, Balaji Lakshminarayanan

Proceedings of the 8th International Conference on Learning Representations, 2020

2019

Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation.

Raphael Gontijo Lopes, Dong Yin, Ben Poole, Justin Gilmer, Ekin D. Cubuk

CoRR, 2019

MNIST-C: A Robustness Benchmark for Computer Vision.

Norman Mu, Justin Gilmer

CoRR, 2019

A Fourier Perspective on Model Robustness in Computer Vision.

Dong Yin, Raphael Gontijo Lopes, Jonathon Shlens, Ekin Dogus Cubuk, Justin Gilmer

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Adversarial Examples Are a Natural Consequence of Test Error in Noise.

Justin Gilmer, Nicolas Ford, Nicholas Carlini, Ekin D. Cubuk

Proceedings of the 36th International Conference on Machine Learning, 2019

2018

Motivating the Rules of the Game for Adversarial Example Research.

Justin Gilmer, Ryan P. Adams, Ian J. Goodfellow, David G. Andersen, George E. Dahl

CoRR, 2018

Relational inductive biases, deep learning, and graph networks.

Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinícius Flores Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Çaglar Gülçehre, H. Francis Song, Andrew J. Ballard, Justin Gilmer, George E. Dahl, Ashish Vaswani, Kelsey R. Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matthew M. Botvinick, Oriol Vinyals, Yujia Li, Razvan Pascanu

CoRR, 2018

Sanity Checks for Saliency Maps.

Julius Adebayo, Justin Gilmer, Michael Muelly, Ian J. Goodfellow, Moritz Hardt, Been Kim

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV).

Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Viégas, Rory Sayres

Proceedings of the 35th International Conference on Machine Learning, 2018

Adversarial Spheres.

Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S. Schoenholz, Maithra Raghu, Martin Wattenberg, Ian J. Goodfellow

Proceedings of the 6th International Conference on Learning Representations, 2018

Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values.

Julius Adebayo, Justin Gilmer, Ian J. Goodfellow, Been Kim

Proceedings of the 6th International Conference on Learning Representations, 2018

2017

A Communication Game Related to the Sensitivity Conjecture.

Justin Gilmer, Michal Koucký, Michael E. Saks

Theory Comput., 2017

Adversarial Patch.

Tom B. Brown, Dandelion Mané, Aurko Roy, Martín Abadi, Justin Gilmer

CoRR, 2017

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Understanding and Improvement.

Maithra Raghu, Justin Gilmer, Jason Yosinski, Jascha Sohl-Dickstein

CoRR, 2017

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability.

Maithra Raghu, Justin Gilmer, Jason Yosinski, Jascha Sohl-Dickstein

Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

Neural Message Passing for Quantum Chemistry.

Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, George E. Dahl

Proceedings of the 34th International Conference on Machine Learning, 2017

Input Switched Affine Networks: An RNN Architecture Designed for Interpretability.

Jakob N. Foerster, Justin Gilmer, Jascha Sohl-Dickstein, Jan Chorowski, David Sussillo

Proceedings of the 34th International Conference on Machine Learning, 2017

Deep Information Propagation.

Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, Jascha Sohl-Dickstein

Proceedings of the 5th International Conference on Learning Representations, 2017

Explaining the Learning Dynamics of Direct Feedback Alignment.

Justin Gilmer, Colin Raffel, Samuel S. Schoenholz, Maithra Raghu, Jascha Sohl-Dickstein

Proceedings of the 5th International Conference on Learning Representations, 2017

2016

A local central limit theorem for triangles in a random graph.

Justin Gilmer, Swastik Kopparty

Random Struct. Algorithms, 2016

Intelligible Language Modeling with Input Switched Affine Networks.

Jakob N. Foerster, Justin Gilmer, Jan Chorowski, Jascha Sohl-Dickstein, David Sussillo

CoRR, 2016

Composition limits and separating examples for some boolean function complexity measures.

Justin Gilmer, Michael E. Saks, Srikanth Srinivasan

Comb., 2016

2015

A New Approach to the Sensitivity Conjecture.

Justin Gilmer, Michal Koucký, Michael E. Saks

Proceedings of the 2015 Conference on Innovations in Theoretical Computer Science, 2015