Tri Dao

According to our database1, Tri Dao authored at least 45 papers between 2017 and 2024.

Collaborative distances:

Timeline

Legend:

Book 
In proceedings 
Article 
PhD thesis 
Dataset
Other 

Links

On csauthors.net:

Bibliography

2024
The Mamba in the Llama: Distilling and Accelerating Hybrid Models.
CoRR, 2024

Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers.
CoRR, 2024

FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision.
CoRR, 2024

An Empirical Study of Mamba-based Language Models.
CoRR, 2024

StarCoder 2 and The Stack v2: The Next Generation.
CoRR, 2024

BitDelta: Your Fine-Tune May Only Be Worth One Bit.
CoRR, 2024

Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling.
Proceedings of the Forty-first International Conference on Machine Learning, 2024

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality.
Proceedings of the Forty-first International Conference on Machine Learning, 2024

Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads.
Proceedings of the Forty-first International Conference on Machine Learning, 2024

FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning.
Proceedings of the Twelfth International Conference on Learning Representations, 2024

2023
StarCoder: may the source be with you!
Trans. Mach. Learn. Res., 2023

Mamba: Linear-Time Sequence Modeling with Selective State Spaces.
CoRR, 2023

Hyena Hierarchy: Towards Larger Convolutional Language Models.
Proceedings of the International Conference on Machine Learning, 2023

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time.
Proceedings of the International Conference on Machine Learning, 2023

Simple Hardware-Efficient Long Convolutions for Sequence Modeling.
Proceedings of the International Conference on Machine Learning, 2023

Effectively Modeling Time Series with Simple Discrete State Spaces.
Proceedings of the Eleventh International Conference on Learning Representations, 2023

Hungry Hungry Hippos: Towards Language Modeling with State Space Models.
Proceedings of the Eleventh International Conference on Learning Representations, 2023

2022
S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces.
CoRR, 2022

Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees.
CoRR, 2022

Decentralized Training of Foundation Models in Heterogeneous Environments.
Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees.
Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Transform Once: Efficient Operator Learning in Frequency Domain.
Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

S4ND: Modeling Images and Videos as Multidimensional Signals with State Spaces.
Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness.
Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

ButterflyFlow: Building Invertible Layers with Butterfly Matrices.
Proceedings of the International Conference on Machine Learning, 2022

Monarch: Expressive Structured Matrices for Efficient and Accurate Training.
Proceedings of the International Conference on Machine Learning, 2022

Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models.
Proceedings of the Tenth International Conference on Learning Representations, 2022

2021
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation.
CoRR, 2021

Rethinking Neural Operations for Diverse Tasks.
Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space Layers.
Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Scatterbrain: Unifying Sparse and Low-rank Attention.
Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Catformer: Designing Stable Transformers via Sensitivity Analysis.
Proceedings of the 38th International Conference on Machine Learning, 2021

Knowledge Distillation as Semiparametric Inference.
Proceedings of the 9th International Conference on Learning Representations, 2021

MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training.
Proceedings of the 9th International Conference on Learning Representations, 2021

2020
HiPPO: Recurrent Memory with Optimal Polynomial Projections.
Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps.
Proceedings of the 8th International Conference on Learning Representations, 2020

2019
Adaptive Hashing for Model Counting.
Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, 2019

On the Downstream Performance of Compressed Word Embeddings.
Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Approximating the Permanent by Sampling from Adaptive Partitions.
Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

A Kernel Theory of Modern Data Augmentation.
Proceedings of the 36th International Conference on Machine Learning, 2019

Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations.
Proceedings of the 36th International Conference on Machine Learning, 2019

Low-Precision Random Fourier Features for Memory-constrained Kernel Approximation.
Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, 2019

2018
Learning Compressed Transforms with Low Displacement Rank.
Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

Learning Invariance with Compact Transforms.
Proceedings of the 6th International Conference on Learning Representations, 2018

2017
Gaussian Quadrature for Kernel Features.
Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017


  Loading...