The Devil in Linear Transformer. - Zhen Qin 003; Xiaodong Han; Weixuan Sun; Dongxu Li; Lingpeng Kong; Nick Barnes; Yiran Zhong (EMNLP, 2022)

cosFormer: Rethinking Softmax In Attention. - Zhen Qin 003; Weixuan Sun; Hui Deng; Dongxu Li; Yunshen Wei; Baohong Lv; Junjie Yan; Lingpeng Kong; Yiran Zhong (ICLR, 2022)

Neural Architecture Search on Efficient Transformers and Beyond. - Zexiang Liu; Dong Li 033; Kaiyue Lu; Zhen Qin 003; Weixuan Sun; Jiacheng Xu; Yiran Zhong (CoRR, 2022)

Linear Video Transformer with Feature Fixation. - Kaiyue Lu; Zexiang Liu; Jianyuan Wang; Weixuan Sun; Zhen Qin 003; Dong Li 033; Xuyang Shen; Hui Deng; Xiaodong Han; Yuchao Dai; Yiran Zhong (CoRR, 2022)

Fine-grained Audible Video Description. - Xuyang Shen; Dong Li 033; Jinxing Zhou; Zhen Qin 003; Bowen He; Xiaodong Han; Aixuan Li; Yuchao Dai; Lingpeng Kong; Meng Wang 001; Yu Qiao 001; Yiran Zhong (CVPR, 2023)

Accelerating Toeplitz Neural Network with Constant-time Inference Complexity. - Zhen Qin 003; Yiran Zhong (EMNLP, 2023)

All-pairs Consistency Learning for Weakly Supervised Semantic Segmentation. - Weixuan Sun; Yanhao Zhang; Zhen Qin 003; Zheyuan Liu 002; Lin Cheng; Fanyi Wang; Yiran Zhong; Nick Barnes (ICCV (Workshops), 2023)

Toeplitz Neural Network for Sequence Modeling. - Zhen Qin 003; Xiaodong Han; Weixuan Sun; Bowen He; Dong Li 033; Dongxu Li; Yuchao Dai; Lingpeng Kong; Yiran Zhong (ICLR, 2023)

Hierarchically Gated Recurrent Neural Network for Sequence Modeling. - Zhen Qin 003; Songlin Yang; Yiran Zhong (NeurIPS, 2023)

Scaling TransNormer to 175 Billion Parameters. - Zhen Qin 003; Dong Li 033; Weigao Sun; Weixuan Sun; Xuyang Shen; Xiaodong Han; Yunshen Wei; Baohong Lv; Fei Yuan; Xiao Luo; Yu Qiao 001; Yiran Zhong (CoRR, 2023)

Linearized Relative Positional Encoding. - Zhen Qin 003; Weixuan Sun; Kaiyue Lu; Hui Deng; Dongxu Li; Xiaodong Han; Yuchao Dai; Lingpeng Kong; Yiran Zhong (Trans. Mach. Learn. Res., 2023)

Vicinity Vision Transformer. - Weixuan Sun; Zhen Qin 003; Hui Deng; Jianyuan Wang; Yi Zhang 137; Kaihao Zhang; Nick Barnes; Stan Birchfield; Lingpeng Kong; Yiran Zhong (IEEE Trans. Pattern Anal. Mach. Intell., 2023)

Exploring Transformer Extrapolation. - Zhen Qin 003; Yiran Zhong; Hui Deng (AAAI, 2024)

Scaling Laws for Linear Complexity Language Models. - Xuyang Shen; Dong Li 033; Ruitao Leng; Zhen Qin 003; Weigao Sun; Yiran Zhong (EMNLP, 2024)

CO2: Efficient Distributed Training with Full Communication-Computation Overlap. - Weigao Sun; Zhen Qin 003; Weixuan Sun; Shidi Li; Dong Li 033; Xuyang Shen; Yu Qiao 001; Yiran Zhong (ICLR, 2024)

Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention. - Zhen Qin 003; Weigao Sun; Dong Li 033; Xuyang Shen; Weixuan Sun; Yiran Zhong (ICML, 2024)

TAVGBench: Benchmarking Text to Audible-Video Generation. - Yuxin Mao; Xuyang Shen; Jing Zhang 052; Zhen Qin 003; Jinxing Zhou; Mochu Xiang; Yiran Zhong; Yuchao Dai (ACM Multimedia, 2024)

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models. - Zhen Qin 003; Weigao Sun; Dong Li 033; Xuyang Shen; Weixuan Sun; Yiran Zhong (CoRR, 2024)

Linear Attention Sequence Parallelism. - Weigao Sun; Zhen Qin 003; Dong Li 033; Xuyang Shen; Yu Qiao 001; Yiran Zhong (CoRR, 2024)

HGRN2: Gated Linear RNNs with State Expansion. - Zhen Qin 003; Songlin Yang; Weixuan Sun; Xuyang Shen; Dong Li 033; Weigao Sun; Yiran Zhong (CoRR, 2024)

Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective. - Zhen Qin 003; Xuyang Shen; Dong Li 033; Weigao Sun; Stan Birchfield; Richard I. Hartley; Yiran Zhong (CoRR, 2024)

You Only Scan Once: Efficient Multi-dimension Sequential Modeling with LightNet. - Zhen Qin 003; Yuxin Mao; Xuyang Shen; Dong Li 033; Jing Zhang 052; Yuchao Dai; Yiran Zhong (CoRR, 2024)

Deep Non-Rigid Structure-from-Motion Revisited: Canonicalization and Sequence Modeling. - Hui Deng; Jiawei Shi; Zhen Qin 003; Yiran Zhong; Yuchao Dai (AAAI, 2025)

Local enhanced toeplitz neural network for in-vehicle network intrusion detection. - Dongxian Shi; Ming Xu; Zhen Qin 003; Yiran Zhong (J. King Saud Univ. Comput. Inf. Sci., 2025)

LASP: Linear Attention Sequence Parallelism. - Weigao Sun; Zhen Qin 003; Dong Li 033; Xuyang Shen; Yu Qiao 001; Yiran Zhong (Trans. Mach. Learn. Res., 2025)

MiniMax-01: Scaling Foundation Models with Lightning Attention. - MiniMax; Aonian Li; Bangwei Gong; Bo Yang; Boji Shan; Chang Liu; Cheng Zhu; Chunhao Zhang; Congchao Guo; Da Chen; Dong Li 033; Enwei Jiao; et al. (CoRR, 2025)

Autoregressive Image Generation with Linear Complexity: A Spatial-Aware Decay Perspective. - Yuxin Mao; Zhen Qin 003; Jinxing Zhou; Hui Deng; Xuyang Shen; Bin Fan 002; Jing Zhang 052; Yiran Zhong; Yuchao Dai (CoRR, 2025)