An Unsupervised Algorithm For Learning Lie Group Transformations - Jascha Sohl-Dickstein; Jimmy C. Wang; Bruno A. Olshausen (CoRR, 2010)

Lie Group Transformation Models for Predictive Video Coding. - Ching Ming Wang; Jascha Sohl-Dickstein; Ivana Tosic; Bruno A. Olshausen (DCC, 2011)

Building a better probabilistic model of images by factorization. - Benjamin J. Culpepper; Jascha Sohl-Dickstein; Bruno A. Olshausen (ICCV, 2011)

Minimum Probability Flow Learning. - Jascha Sohl-Dickstein; Peter Battaglino; Michael Robert DeWeese (ICML, 2011)

Training sparse natural image models with a fast Gibbs sampler of an extended state space. - Lucas Theis; Jascha Sohl-Dickstein; Matthias Bethge (NIPS, 2012)

The Natural Gradient by Analogy to Signal Whitening, and Recipes and Tricks for its Use - Jascha Sohl-Dickstein (CoRR, 2012)

Hamiltonian Annealed Importance Sampling for partition function estimation - Jascha Sohl-Dickstein; Benjamin J. Culpepper (CoRR, 2012)

Hamiltonian Monte Carlo with Reduced Momentum Flips - Jascha Sohl-Dickstein (CoRR, 2012)

Efficient Methods for Unsupervised Learning of Probabilistic Models - Jascha Sohl-Dickstein (CoRR, 2012)

Efficient Methods for Unsupervised Learning of Probabilistic Models. - Jascha Sohl-Dickstein (2012)

Controlled experiments on millions of students to personalize learning. - Eliana Feasley; Chris Klaiber; James Irwin; Jace Kohlmeier; Jascha Sohl-Dickstein (AIED Workshops, 2013)

Measurably Increasing Motivation in MOOCs. - Joseph Jay Williams; Dave Paunesku; Benjamin Heley; Jascha Sohl-Dickstein (AIED Workshops, 2013)

An adaptive low dimensional quasi-Newton sum of functions optimizer. - Jascha Sohl-Dickstein; Ben Poole; Surya Ganguli (CoRR, 2013)

Hamiltonian Monte Carlo Without Detailed Balance. - Jascha Sohl-Dickstein; Mayur Mudigonda; Michael Robert DeWeese (ICML, 2014)

Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods. - Jascha Sohl-Dickstein; Ben Poole; Surya Ganguli (ICML, 2014)

Analyzing noise in autoencoders and deep networks. - Ben Poole; Jascha Sohl-Dickstein; Surya Ganguli (CoRR, 2014)

Modeling Higher-Order Correlations within Cortical Microcolumns. - Urs Köster; Jascha Sohl-Dickstein; Charles M. Gray; Bruno A. Olshausen (PLoS Comput. Biol., 2014)

Deep Unsupervised Learning using Nonequilibrium Thermodynamics. - Jascha Sohl-Dickstein; Eric A. Weiss; Niru Maheswaranathan; Surya Ganguli (ICML, 2015)

Deep Knowledge Tracing. - Chris Piech; Jonathan Bassen; Jonathan Huang; Surya Ganguli; Mehran Sahami; Leonidas J. Guibas; Jascha Sohl-Dickstein (NIPS, 2015)

Technical Note on Equivalence Between Recurrent Neural Network Time Series Models and Variational Bayesian Models. - Jascha Sohl-Dickstein; Diederik P. Kingma (CoRR, 2015)

A Device for Human Ultrasonic Echolocation. - Jascha Sohl-Dickstein; Santani Teng; Benjamin M. Gaub; Chris C. Rodgers; Crystal Li; Michael Robert DeWeese; Nicol S. Harper (IEEE Trans. Biomed. Eng., 2015)

Exponential expressivity in deep neural networks through transient chaos. - Ben Poole; Subhaneil Lahiri; Maithra Raghu; Jascha Sohl-Dickstein; Surya Ganguli (NIPS, 2016)

Intelligible Language Modeling with Input Switched Affine Networks. - Jakob N. Foerster; Justin Gilmer; Jan Chorowski; Jascha Sohl-Dickstein; David Sussillo (CoRR, 2016)

A universal tradeoff between power, precision and speed in physical communication. - Subhaneil Lahiri; Jascha Sohl-Dickstein; Surya Ganguli (CoRR, 2016)

Improved generator objectives for GANs. - Ben Poole; Alexander A. Alemi; Jascha Sohl-Dickstein; Anelia Angelova (CoRR, 2016)

Survey of Expressivity in Deep Neural Networks. - Maithra Raghu; Ben Poole; Jon M. Kleinberg; Surya Ganguli; Jascha Sohl-Dickstein (CoRR, 2016)

Capacity and Trainability in Recurrent Neural Networks. - Jasmine Collins; Jascha Sohl-Dickstein; David Sussillo (ICLR (Poster), 2017)

Density estimation using Real NVP. - Laurent Dinh; Jascha Sohl-Dickstein; Samy Bengio (ICLR (Poster), 2017)

Explaining the Learning Dynamics of Direct Feedback Alignment. - Justin Gilmer; Colin Raffel; Samuel S. Schoenholz; Maithra Raghu; Jascha Sohl-Dickstein (ICLR (Workshop), 2017)

Unrolled Generative Adversarial Networks. - Luke Metz; Ben Poole; David Pfau; Jascha Sohl-Dickstein (ICLR (Poster), 2017)

Deep Information Propagation. - Samuel S. Schoenholz; Justin Gilmer; Surya Ganguli; Jascha Sohl-Dickstein (ICLR (Poster), 2017)

REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models. - George Tucker; Andriy Mnih; Chris J. Maddison; Jascha Sohl-Dickstein (ICLR (Workshop), 2017)

Input Switched Affine Networks: An RNN Architecture Designed for Interpretability. - Jakob N. Foerster; Justin Gilmer; Jascha Sohl-Dickstein; Jan Chorowski; David Sussillo (ICML, 2017)

On the Expressive Power of Deep Neural Networks. - Maithra Raghu; Ben Poole; Jon M. Kleinberg; Surya Ganguli; Jascha Sohl-Dickstein (ICML, 2017)

Learned Optimizers that Scale and Generalize. - Olga Wichrowska; Niru Maheswaranathan; Matthew W. Hoffman; Sergio Gomez Colmenarejo; Misha Denil; Nando de Freitas; Jascha Sohl-Dickstein (ICML, 2017)

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability. - Maithra Raghu; Justin Gilmer; Jason Yosinski; Jascha Sohl-Dickstein (NIPS, 2017)

REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models. - George Tucker; Andriy Mnih; Chris J. Maddison; Dieterich Lawson; Jascha Sohl-Dickstein (NIPS, 2017)

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Understanding and Improvement. - Maithra Raghu; Justin Gilmer; Jason Yosinski; Jascha Sohl-Dickstein (CoRR, 2017)

A Correspondence Between Random Neural Networks and Statistical Field Theory. - Samuel S. Schoenholz; Jeffrey Pennington; Jascha Sohl-Dickstein (CoRR, 2017)

Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations. - Badr F. Albanna; Christopher Hillar; Jascha Sohl-Dickstein; Michael Robert DeWeese (Entropy, 2017)

Deep Neural Networks as Gaussian Processes. - Jaehoon Lee 001; Yasaman Bahri; Roman Novak; Samuel S. Schoenholz; Jeffrey Pennington; Jascha Sohl-Dickstein (ICLR (Poster), 2018)

Generalizing Hamiltonian Monte Carlo with Neural Networks. - Daniel Levy; Matthew D. Hoffman; Jascha Sohl-Dickstein (ICLR (Poster), 2018)

Learning to Learn Without Labels. - Luke Metz; Niru Maheswaranathan; Brian Cheung; Jascha Sohl-Dickstein (ICLR (Workshop), 2018)

Sensitivity and Generalization in Neural Networks: an Empirical Study. - Roman Novak; Yasaman Bahri; Daniel A. Abolafia; Jeffrey Pennington; Jascha Sohl-Dickstein (ICLR (Poster), 2018)

Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10, 000-Layer Vanilla Convolutional Neural Networks. - Lechao Xiao; Yasaman Bahri; Jascha Sohl-Dickstein; Samuel S. Schoenholz; Jeffrey Pennington (ICML, 2018)

PCA of high dimensional random walks with comparison to neural network training. - Joseph M. Antognini; Jascha Sohl-Dickstein (NeurIPS, 2018)

Adversarial Examples that Fool both Computer Vision and Time-Limited Humans. - Gamaleldin F. Elsayed; Shreya Shankar; Brian Cheung; Nicolas Papernot; Alexey Kurakin; Ian J. Goodfellow; Jascha Sohl-Dickstein (NeurIPS, 2018)

Adversarial Examples that Fool both Human and Computer Vision. - Gamaleldin F. Elsayed; Shreya Shankar; Brian Cheung; Nicolas Papernot; Alex Kurakin; Ian J. Goodfellow; Jascha Sohl-Dickstein (CoRR, 2018)

Learning Unsupervised Learning Rules. - Luke Metz; Niru Maheswaranathan; Brian Cheung; Jascha Sohl-Dickstein (CoRR, 2018)

Stochastic natural gradient descent draws posterior samples in function space. - Samuel L. Smith; Daniel Duckworth; Quoc V. Le; Jascha Sohl-Dickstein (CoRR, 2018)

Guided evolutionary strategies: escaping the curse of dimensionality in random search. - Niru Maheswaranathan; Luke Metz; George Tucker; Jascha Sohl-Dickstein (CoRR, 2018)

Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes. - Roman Novak; Lechao Xiao; Jaehoon Lee 001; Yasaman Bahri; Daniel A. Abolafia; Jeffrey Pennington; Jascha Sohl-Dickstein (CoRR, 2018)

Learned optimizers that outperform SGD on wall-clock and test loss. - Luke Metz; Niru Maheswaranathan; Jeremy Nixon; C. Daniel Freeman; Jascha Sohl-Dickstein (CoRR, 2018)

A RAD approach to deep mixture models. - Laurent Dinh; Jascha Sohl-Dickstein; Razvan Pascanu; Hugo Larochelle (DGS@ICLR, 2019)

Adversarial Reprogramming of Neural Networks. - Gamaleldin F. Elsayed; Ian J. Goodfellow; Jascha Sohl-Dickstein (ICLR (Poster), 2019)

Meta-Learning Update Rules for Unsupervised Representation Learning. - Luke Metz; Niru Maheswaranathan; Brian Cheung; Jascha Sohl-Dickstein (ICLR, 2019)

Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes. - Roman Novak; Lechao Xiao; Yasaman Bahri; Jaehoon Lee 001; Greg Yang; Jiri Hron; Daniel A. Abolafia; Jeffrey Pennington; Jascha Sohl-Dickstein (ICLR (Poster), 2019)

A Mean Field Theory of Batch Normalization. - Greg Yang; Jeffrey Pennington; Vinay Rao; Jascha Sohl-Dickstein; Samuel S. Schoenholz (ICLR (Poster), 2019)

Guided evolutionary strategies: augmenting random search with surrogate gradients. - Niru Maheswaranathan; Luke Metz; George Tucker; Dami Choi; Jascha Sohl-Dickstein (ICML, 2019)

Understanding and correcting pathologies in the training of learned optimizers. - Luke Metz; Niru Maheswaranathan; Jeremy Nixon; C. Daniel Freeman; Jascha Sohl-Dickstein (ICML, 2019)

The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study. - Daniel S. Park; Jascha Sohl-Dickstein; Quoc V. Le; Samuel L. Smith (ICML, 2019)

Invertible Convolutional Flow. - Mahdi Karami; Dale Schuurmans; Jascha Sohl-Dickstein; Laurent Dinh; Daniel Duckworth (NeurIPS, 2019)

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. - Jaehoon Lee 001; Lechao Xiao; Samuel S. Schoenholz; Yasaman Bahri; Roman Novak; Jascha Sohl-Dickstein; Jeffrey Pennington (NeurIPS, 2019)

Eliminating all bad Local Minima from Loss Landscapes without even adding an Extra Unit. - Jascha Sohl-Dickstein; Kenji Kawaguchi (CoRR, 2019)

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent. - Jaehoon Lee 001; Lechao Xiao; Samuel S. Schoenholz; Yasaman Bahri; Jascha Sohl-Dickstein; Jeffrey Pennington (CoRR, 2019)

Using learned optimizers to make models robust to input noise. - Luke Metz; Niru Maheswaranathan; Jonathon Shlens; Jascha Sohl-Dickstein; Ekin D. Cubuk (CoRR, 2019)

Neural reparameterization improves structural optimization. - Stephan Hoyer; Jascha Sohl-Dickstein; Sam Greydanus (CoRR, 2019)

Measuring the Effects of Data Parallelism on Neural Network Training. - Christopher J. Shallue; Jaehoon Lee 001; Joseph M. Antognini; Jascha Sohl-Dickstein; Roy Frostig; George E. Dahl (J. Mach. Learn. Res., 2019)

Neural Tangents: Fast and Easy Infinite Neural Networks in Python. - Roman Novak; Lechao Xiao; Jiri Hron; Jaehoon Lee 001; Alexander A. Alemi; Jascha Sohl-Dickstein; Samuel S. Schoenholz (ICLR, 2020)

Infinite attention: NNGP and NTK for deep attention networks. - Jiri Hron; Yasaman Bahri; Jascha Sohl-Dickstein; Roman Novak (ICML, 2020)

Your GAN is Secretly an Energy-based Model and You Should Use Discriminator Driven Latent Sampling. - Tong Che; Ruixiang Zhang; Jascha Sohl-Dickstein; Hugo Larochelle; Liam Paull; Yuan Cao; Yoshua Bengio (NeurIPS, 2020)

Finite Versus Infinite Neural Networks: an Empirical Study. - Jaehoon Lee 001; Samuel S. Schoenholz; Jeffrey Pennington; Ben Adlam; Lechao Xiao; Roman Novak; Jascha Sohl-Dickstein (NeurIPS, 2020)

On the infinite width limit of neural networks with a standard parameterization. - Jascha Sohl-Dickstein; Roman Novak; Samuel S. Schoenholz; Jaehoon Lee 001 (CoRR, 2020)

Using a thousand optimization tasks to learn hyperparameter search strategies. - Luke Metz; Niru Maheswaranathan; Ruoxi Sun 002; C. Daniel Freeman; Ben Poole; Jascha Sohl-Dickstein (CoRR, 2020)

The large learning rate phase of deep learning: the catapult mechanism. - Aitor Lewkowycz; Yasaman Bahri; Ethan Dyer; Jascha Sohl-Dickstein; Guy Gur-Ari (CoRR, 2020)

Exact posterior distributions of wide Bayesian neural networks. - Jiri Hron; Yasaman Bahri; Roman Novak; Jeffrey Pennington; Jascha Sohl-Dickstein (CoRR, 2020)

A new method for parameter estimation in probabilistic models: Minimum probability flow. - Jascha Sohl-Dickstein; Peter Battaglino; Michael Robert DeWeese (CoRR, 2020)

Whitening and second order optimization both destroy information about the dataset, and can make generalization impossible. - Neha S. Wadia; Daniel Duckworth; Samuel S. Schoenholz; Ethan Dyer; Jascha Sohl-Dickstein (CoRR, 2020)

Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves. - Luke Metz; Niru Maheswaranathan; C. Daniel Freeman; Ben Poole; Jascha Sohl-Dickstein (CoRR, 2020)

Is Batch Norm unique? An empirical investigation and prescription to emulate the best properties of common normalizers without batch dependence. - Vinay Rao; Jascha Sohl-Dickstein (CoRR, 2020)

Towards NNGP-guided Neural Architecture Search. - Daniel S. Park; Jaehoon Lee 001; Daiyi Peng; Yuan Cao; Jascha Sohl-Dickstein (CoRR, 2020)

Parallel Training of Deep Networks with Local Updates. - Michael Laskin; Luke Metz; Seth Nabarrao; Mark Saroufim; Badreddine Noune; Carlo Luschi; Jascha Sohl-Dickstein; Pieter Abbeel (CoRR, 2020)

Score-Based Generative Modeling through Stochastic Differential Equations. - Yang Song 011; Jascha Sohl-Dickstein; Diederik P. Kingma; Abhishek Kumar; Stefano Ermon; Ben Poole (ICLR, 2021)

Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies. - Paul Vicol; Luke Metz; Jascha Sohl-Dickstein (ICML, 2021)

Whitening and Second Order Optimization Both Make Information in the Dataset Unusable During Training, and Can Reduce or Prevent Generalization. - Neha S. Wadia; Daniel Duckworth; Samuel S. Schoenholz; Ethan Dyer; Jascha Sohl-Dickstein (ICML, 2021)

Reverse engineering learned optimizers reveals known and novel mechanisms. - Niru Maheswaranathan; David Sussillo; Luke Metz; Ruoxi Sun 002; Jascha Sohl-Dickstein (NeurIPS, 2021)

Training Learned Optimizers with Randomly Initialized Learned Optimizers. - Luke Metz; C. Daniel Freeman; Niru Maheswaranathan; Jascha Sohl-Dickstein (CoRR, 2021)

Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping. - James Martens; Andy Ballard; Guillaume Desjardins; Grzegorz Swirszcz; Valentin Dalibard; Jascha Sohl-Dickstein; Samuel S. Schoenholz (CoRR, 2021)

NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation. - Kaustubh D. Dhole; Varun Gangal; Sebastian Gehrmann; Aadesh Gupta; Zhenhao Li; Saad Mahamood; Abinaya Mahendiran; Simon Mille; Ashish Srivastava; Samson Tan; Tongshuang Wu; Jascha Sohl-Dickstein; et al. (CoRR, 2021)

Practical Tradeoffs between Memory, Compute, and Performance in Learned Optimizers. - Luke Metz; C. Daniel Freeman; James Harrison; Niru Maheswaranathan; Jascha Sohl-Dickstein (CoLLAs, 2022)

Wide Bayesian neural networks have a simple weight posterior: theory and accelerated sampling. - Jiri Hron; Roman Novak; Jeffrey Pennington; Jascha Sohl-Dickstein (ICML, 2022)

Fast Finite Width Neural Tangent Kernel. - Roman Novak; Jascha Sohl-Dickstein; Samuel S. Schoenholz (ICML, 2022)

Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies (Extended Abstract). - Paul Vicol; Luke Metz; Jascha Sohl-Dickstein (IJCAI, 2022)

A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases. - James Harrison; Luke Metz; Jascha Sohl-Dickstein (NeurIPS, 2022)

Language Model Cascades. - David Dohan; Winnie Xu; Aitor Lewkowycz; Jacob Austin; David Bieber; Raphael Gontijo Lopes; Yuhuai Wu; Henryk Michalewski; Rif A. Saurous; Jascha Sohl-Dickstein; Kevin Murphy 002; Charles Sutton (CoRR, 2022)

VeLO: Training Versatile Learned Optimizers by Scaling Up. - Luke Metz; James Harrison; C. Daniel Freeman; Amil Merchant; Lucas Beyer; James Bradbury; Naman Agrawal; Ben Poole; Igor Mordatch; Adam Roberts; Jascha Sohl-Dickstein (CoRR, 2022)

General-Purpose In-Context Learning by Meta-Learning Transformers. - Louis Kirsch; James Harrison; Jascha Sohl-Dickstein; Luke Metz (CoRR, 2022)

Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC. - Yilun Du; Conor Durkan; Robin Strudel; Joshua B. Tenenbaum; Sander Dieleman; Rob Fergus; Jascha Sohl-Dickstein; Arnaud Doucet; Will Sussman Grathwohl (ICML, 2023)

Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies. - Oscar Li; James Harrison; Jascha Sohl-Dickstein; Virginia Smith; Luke Metz (NeurIPS, 2023)

Noise-Reuse in Online Evolution Strategies. - Oscar Li; James Harrison; Jascha Sohl-Dickstein; Virginia Smith; Luke Metz (CoRR, 2023)

Levels of AGI: Operationalizing Progress on the Path to AGI. - Meredith Ringel Morris; Jascha Sohl-Dickstein; Noah Fiedel; Tris Warkentin; Allan Dafoe; Aleksandra Faust; Clément Farabet; Shane Legg (CoRR, 2023)

Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5? - C. Daniel Freeman; Laura Culp; Aaron Parisi; Maxwell L. Bileschi; Gamaleldin F. Elsayed; Alex Rizkowsky; Isabelle Simpson; Alex Alemi; Azade Nova; Ben Adlam; Bernd Bohnet; Gaurav Mishra; et al. (CoRR, 2023)

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. - Aarohi Srivastava; Abhinav Rastogi; Abhishek Rao; Abu Awal Md Shoeb; Abubakar Abid; Adam Fisch; Adam R. Brown; Adam Santoro; Aditya Gupta; Adrià Garriga-Alonso; Agnieszka Kluska; Aitor Lewkowycz; et al. (Trans. Mach. Learn. Res., 2023)

Small-scale proxies for large-scale Transformer training instabilities. - Mitchell Wortsman; Peter J. Liu; Lechao Xiao; Katie E. Everett; Alexander A. Alemi; Ben Adlam; John D. Co-Reyes; Izzeddin Gur; Abhishek Kumar; Roman Novak; Jeffrey Pennington; Jascha Sohl-Dickstein; et al. (ICLR, 2024)

Scaling Exponents Across Parameterizations and Optimizers. - Katie E. Everett; Lechao Xiao; Mitchell Wortsman; Alexander A. Alemi; Roman Novak; Peter J. Liu; Izzeddin Gur; Jascha Sohl-Dickstein; Leslie Pack Kaelbling; Jaehoon Lee 001; Jeffrey Pennington (ICML, 2024)

Position: Levels of AGI for Operationalizing Progress on the Path to AGI. - Meredith Ringel Morris; Jascha Sohl-Dickstein; Noah Fiedel; Tris Warkentin; Allan Dafoe; Aleksandra Faust; Clément Farabet; Shane Legg (ICML, 2024)

The boundary of neural network trainability is fractal. - Jascha Sohl-Dickstein (CoRR, 2024)

Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability. - Jiri Hron; Laura Culp; Gamaleldin F. Elsayed; Rosanne Liu; Ben Adlam; Maxwell L. Bileschi; Bernd Bohnet; JD Co-Reyes; Noah Fiedel; C. Daniel Freeman; Izzeddin Gur; Kathleen Kenealy; et al. (CoRR, 2024)

Training LLMs over Neurally Compressed Text. - Brian Lester; Jaehoon Lee 001; Alexander A. Alemi; Jeffrey Pennington; Adam Roberts; Jascha Sohl-Dickstein; Noah Constant (Trans. Mach. Learn. Res., 2024)

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models. - Avi Singh; John D. Co-Reyes; Rishabh Agarwal; Ankesh Anand; Piyush Patil; Xavier Garcia; Peter J. Liu; James Harrison; Jaehoon Lee 001; Kelvin Xu; Aaron T. Parisi; Abhishek Kumar; et al. (Trans. Mach. Learn. Res., 2024)