The RWTH English lecture recognition system. - Simon Wiesler; Kazuki Irie; Zoltán Tüske; Ralf Schlüter; Hermann Ney (ICASSP, 2014)

On efficient training of word classes and their application to recurrent neural network language models. - Rami Botros; Kazuki Irie; Martin Sundermeyer; Hermann Ney (INTERSPEECH, 2015)

Bag-of-words input for long history representation in neural network-based language models for speech recognition. - Kazuki Irie; Ralf Schlüter; Hermann Ney (INTERSPEECH, 2015)

Investigation on log-linear interpolation of multi-domain neural network language model. - Zoltán Tüske; Kazuki Irie; Ralf Schlüter; Hermann Ney (ICASSP, 2016)

LSTM, GRU, Highway and a Bit of Attention: An Empirical Overview for Language Modeling in Speech Recognition. - Kazuki Irie; Zoltán Tüske; Tamer Alkhouli; Ralf Schlüter; Hermann Ney (INTERSPEECH, 2016)

Automatic Speech Recognition Based on Neural Networks. - Ralf Schlüter; Patrick Doetsch; Pavel Golik; Markus Kitza; Tobias Menne; Kazuki Irie; Zoltán Tüske; Albert Zeyer (SPECOM, 2016)

Investigations on byte-level convolutional neural networks for language modeling in low resource speech recognition. - Kazuki Irie; Pavel Golik; Ralf Schlüter; Hermann Ney (ICASSP, 2017)

The 2016 RWTH Keyword Search System for Low-Resource Languages. - Pavel Golik; Zoltán Tüske; Kazuki Irie; Eugen Beck; Ralf Schlüter; Hermann Ney (SPECOM, 2017)

RADMM: Recurrent Adaptive Mixture Model with Applications to Domain Robust Language Modeling. - Kazuki Irie; Shankar Kumar; Michael Nirschl; Hank Liao (ICASSP, 2018)

Prediction of LSTM-RNN Full Context States as a Subtask for N-Gram Feedforward Language Models. - Kazuki Irie; Zhihong Lei; Ralf Schlüter; Hermann Ney (ICASSP, 2018)

Investigation on Estimation of Sentence Probability by Combining Forward, Backward and Bi-directional LSTM-RNNs. - Kazuki Irie; Zhihong Lei; Liuhui Deng; Ralf Schlüter; Hermann Ney (INTERSPEECH, 2018)

Improved Training of End-to-end Attention Models for Speech Recognition. - Albert Zeyer; Kazuki Irie; Ralf Schlüter; Hermann Ney (INTERSPEECH, 2018)

Training Language Models for Long-Span Cross-Sentence Evaluation. - Kazuki Irie; Albert Zeyer; Ralf Schlüter; Hermann Ney (ASRU, 2019)

A Comparison of Transformer and LSTM Encoder Decoder Models for ASR. - Albert Zeyer; Parnia Bahar; Kazuki Irie; Ralf Schlüter; Hermann Ney (ASRU, 2019)

On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition. - Kazuki Irie; Rohit Prabhavalkar; Anjuli Kannan; Antoine Bruguier; David Rybach; Patrick Nguyen (INTERSPEECH, 2019)

Language Modeling with Deep Transformers. - Kazuki Irie; Albert Zeyer; Ralf Schlüter; Hermann Ney (INTERSPEECH, 2019)

RWTH ASR Systems for LibriSpeech: Hybrid vs Attention. - Christoph Lüscher; Eugen Beck; Kazuki Irie; Markus Kitza; Wilfried Michel; Albert Zeyer; Ralf Schlüter; Hermann Ney (INTERSPEECH, 2019)

Model Unit Exploration for Sequence-to-Sequence Speech Recognition. - Kazuki Irie; Rohit Prabhavalkar; Anjuli Kannan; Antoine Bruguier; David Rybach; Patrick Nguyen (CoRR, 2019)

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling. - Jonathan Shen; Patrick Nguyen; Yonghui Wu; Zhifeng Chen; Mia Xu Chen; Ye Jia; Anjuli Kannan; Tara N. Sainath; Yuan Cao 007; Chung-Cheng Chiu; Yanzhang He; Jan Chorowski; et al. (CoRR, 2019)

RWTH ASR Systems for LibriSpeech: Hybrid vs Attention - w/o Data Augmentation. - Christoph Lüscher; Eugen Beck; Kazuki Irie; Markus Kitza; Wilfried Michel; Albert Zeyer; Ralf Schlüter; Hermann Ney (CoRR, 2019)

Domain Robust, Fast, and Compact Neural Language Models. - Alexander Gerstenberger; Kazuki Irie; Pavel Golik; Eugen Beck; Hermann Ney (ICASSP, 2020)

How Much Self-Attention Do We Need? Trading Attention for Feed-Forward Layers. - Kazuki Irie; Alexander Gerstenberger; Ralf Schlüter; Hermann Ney (ICASSP, 2020)

The Rwth Asr System for Ted-Lium Release 2: Improving Hybrid Hmm With Specaugment. - Wei Zhou 043; Wilfried Michel; Kazuki Irie; Markus Kitza; Ralf Schlüter; Hermann Ney (ICASSP, 2020)

Advancing neural language modeling in automatic speech recognition. - Kazuki Irie (2020)

The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers. - Róbert Csordás; Kazuki Irie; Jürgen Schmidhuber (EMNLP (1), 2021)

Linear Transformers Are Secretly Fast Weight Programmers. - Imanol Schlag; Kazuki Irie; Jürgen Schmidhuber (ICML, 2021)

Going Beyond Linear Transformers with Recurrent Fast Weight Programmers. - Kazuki Irie; Imanol Schlag; Róbert Csordás; Jürgen Schmidhuber (NeurIPS, 2021)

Linear Transformers Are Secretly Fast Weight Memory Systems. - Imanol Schlag; Kazuki Irie; Jürgen Schmidhuber (CoRR, 2021)

Training and Generating Neural Networks in Compressed Weight Space. - Kazuki Irie; Jürgen Schmidhuber (CoRR, 2021)

Improving Baselines in the Wild. - Kazuki Irie; Imanol Schlag; Róbert Csordás; Jürgen Schmidhuber (CoRR, 2021)

CTL++: Evaluating Generalization on Never-Seen Compositional Patterns of Known Functions, and Compatibility of Neural Representations. - Róbert Csordás; Kazuki Irie; Jürgen Schmidhuber (EMNLP, 2022)

The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization. - Róbert Csordás; Kazuki Irie; Jürgen Schmidhuber (ICLR, 2022)

The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention. - Kazuki Irie; Róbert Csordás; Jürgen Schmidhuber (ICML, 2022)

A Modern Self-Referential Weight Matrix That Learns to Modify Itself. - Kazuki Irie; Imanol Schlag; Róbert Csordás; Jürgen Schmidhuber (ICML, 2022)

Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules. - Kazuki Irie; Francesco Faccio; Jürgen Schmidhuber (NeurIPS, 2022)

Learning to Control Rapidly Changing Synaptic Connections: An Alternative Type of Memory in Sequence Processing Artificial Neural Networks. - Kazuki Irie; Jürgen Schmidhuber (CoRR, 2022)

Approximating Two-Layer Feedforward Networks for Efficient Transformers. - Róbert Csordás; Kazuki Irie; Jürgen Schmidhuber (EMNLP (Findings), 2023)

Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions. - Kazuki Irie; Róbert Csordás; Jürgen Schmidhuber (EMNLP, 2023)

Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules. - Kazuki Irie; Jürgen Schmidhuber (ICLR, 2023)

Contrastive Training of Complex-Valued Autoencoders for Object Discovery. - Aleksandar Stanic; Anand Gopalakrishnan; Kazuki Irie; Jürgen Schmidhuber (NeurIPS, 2023)

Topological Neural Discrete Representation Learning à la Kohonen. - Kazuki Irie; Róbert Csordás; Jürgen Schmidhuber (CoRR, 2023)

Accelerating Neural Self-Improvement via Bootstrapping. - Kazuki Irie; Jürgen Schmidhuber (CoRR, 2023)

Automating Continual Learning. - Kazuki Irie; Róbert Csordás; Jürgen Schmidhuber (CoRR, 2023)

Unsupervised Learning of Temporal Abstractions With Slot-Based Transformers. - Anand Gopalakrishnan; Kazuki Irie; Jürgen Schmidhuber; Sjoerd van Steenkiste (Neural Comput., 2023)

Self-organising Neural Discrete Representation Learning à la Kohonen. - Kazuki Irie; Róbert Csordás; Jürgen Schmidhuber (ICANN (1), 2024)

Exploring the Promise and Limits of Real-Time Recurrent Learning. - Kazuki Irie; Anand Gopalakrishnan; Jürgen Schmidhuber (ICLR, 2024)

MoEUT: Mixture-of-Experts Universal Transformers. - Róbert Csordás; Kazuki Irie; Jürgen Schmidhuber; Christopher Potts; Christopher D. Manning (NeurIPS, 2024)

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention. - Róbert Csordás; Piotr Piekos; Kazuki Irie; Jürgen Schmidhuber (NeurIPS, 2024)

Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers. - Lorenzo Tiberi; Francesca Mignacco; Kazuki Irie; Haim Sompolinsky (NeurIPS, 2024)

Neural networks that overcome classic challenges through practice. - Kazuki Irie; Brenden M. Lake (CoRR, 2024)

Why Are Positional Encodings Nonessential for Deep Autoregressive Transformers? A Petroglyph Revisited. - Kazuki Irie (ACL (Findings), 2025)

Mindstorms in Natural Language-Based Societies of Mind. - Mingchen Zhuge; Haozhe Liu; Francesco Faccio; Dylan R. Ashley; Róbert Csordás; Anand Gopalakrishnan; Abdullah Hamdi; Hasan Abed Al Kader Hammoud; Vincent Herrmann; Kazuki Irie; Louis Kirsch; Bing Li 024; et al. (Comput. Vis. Media, 2025)

Metalearning Continual Learning Algorithms. - Kazuki Irie; Róbert Csordás; Jürgen Schmidhuber (Trans. Mach. Learn. Res., 2025)

Why Are Positional Encodings Nonessential for Deep Autoregressive Transformers? Revisiting a Petroglyph. - Kazuki Irie (CoRR, 2025)

Key-value memory in the brain. - Samuel J. Gershman; Ila Fiete; Kazuki Irie (CoRR, 2025)

Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers. - Kazuki Irie; Morris Yau; Samuel J. Gershman (CoRR, 2025)

Sequential-Parallel Duality in Prefix Scannable Models. - Morris Yau; Sharut Gupta; Valerie Engelmayer; Kazuki Irie; Stefanie Jegelka; Jacob Andreas (CoRR, 2025)

Fast weight programming and linear transformers: from machine learning to neurobiology. - Kazuki Irie; Samuel J. Gershman (CoRR, 2025)