Kaiyue Wen

According to our database, Kaiyue Wen authored at least 21 papers between 2022 and 2025.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2025

QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation.

Jiazheng Li, Hong Lu, Kaiyue Wen, Zaiwen Yang, Jiaxuan Gao, Hongzhou Lin, Yi Wu, Jingzhao Zhang

CoRR, July, 2025

PaTH Attention: Position Encoding via Accumulating Householder Transformations.

Songlin Yang, Yikang Shen, Kaiyue Wen, Shawn Tan, Mayank Mishra, Liliang Ren, Rameswar Panda, Yoon Kim

CoRR, May, 2025

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free.

Zihan Qiu, Zekun Wang, Bo Zheng, Zeyu Huang, Kaiyue Wen, Songlin Yang, Rui Men, Le Yu, Fei Huang, Suozhi Huang, Dayiheng Liu, Jingren Zhou, Junyang Lin

CoRR, May, 2025

Weight Ensembling Improves Reasoning in Language Models.

Xingyu Dang, Christina Baek, Kaiyue Wen, Zico Kolter, Aditi Raghunathan

CoRR, April, 2025

Overtrained Language Models Are Harder to Fine-Tune.

Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, Aditi Raghunathan

CoRR, March, 2025

Task Generalization With AutoRegressive Compositional Structure: Can Learning From <i>D</i> Tasks Generalize to <i>D</i>^T Tasks?

Amirhesam Abedsoltan, Huaqing Zhang, Kaiyue Wen, Hongzhou Lin, Jingzhao Zhang, Mikhail Belkin

CoRR, February, 2025

From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency.

Kaiyue Wen, Huaqing Zhang, Hongzhou Lin, Jingzhao Zhang

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

RNNs are not Transformers (Yet): The Key Bottleneck on In-Context Retrieval.

Kaiyue Wen, Xingyu Dang, Kaifeng Lyu

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View.

Kaiyue Wen, Zhiyuan Li, Jason S. Wang, David Leo Wright Hall, Percy Liang, Tengyu Ma

Proceedings of the Thirteenth International Conference on Learning Representations, 2025

Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images.

Shengguang Wu, Fan-Yun Sun, Kaiyue Wen, Nick Haber

Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models.

Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin

Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025

2024

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective.

Kaiyue Wen, Zhiyuan Li, Jason S. Wang, David Hall, Percy Liang, Tengyu Ma

CoRR, 2024

2023

Practically Solving LPN in High Noise Regimes Faster Using Neural Networks.

Haozhe Jiang, Kaiyue Wen, Yilei Chen

IACR Cryptol. ePrint Arch., 2023

Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars.

Kaiyue Wen, Yuchen Li, Bingbin Liu, Andrej Risteski

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization.

Kaiyue Wen, Zhiyuan Li, Tengyu Ma

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models.

Kaiyue Wen, Jiaye Teng, Jingzhao Zhang

Proceedings of the Eleventh International Conference on Learning Representations, 2023

How Sharpness-Aware Minimization Minimizes Sharpness?

Kaiyue Wen, Tengyu Ma, Zhiyuan Li

Proceedings of the Eleventh International Conference on Learning Representations, 2023

2022

How Does Sharpness-Aware Minimization Minimize Sharpness?

Kaiyue Wen, Tengyu Ma, Zhiyuan Li

CoRR, 2022

Realistic Deep Learning May Not Fit Benignly.

Kaiyue Wen, Jiaye Teng, Jingzhao Zhang

CoRR, 2022

On Transferability of Prompt Tuning for Natural Language Processing.

Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, Jie Zhou

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022

Finding Skill Neurons in Pre-trained Transformer-based Language Models.

Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, Juanzi Li

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022