A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embeddings. - Puyuan Peng; Herman Kamper; Karen Livescu (CoRR, 2020)

Fast-Slow Transformer for Visually Grounding Speech. - Puyuan Peng; David Harwath (ICASSP, 2022)

MAE-AST: Masked Autoencoding Audio Spectrogram Transformer. - Alan Baade; Puyuan Peng; David Harwath (INTERSPEECH, 2022)

Word Discovery in Visually Grounded, Self-Supervised Speech Models. - Puyuan Peng; David Harwath (INTERSPEECH, 2022)

Zero-shot Video Moment Retrieval With Off-the-Shelf Models. - Anuj Diwan; Puyuan Peng; Raymond J. Mooney (TL4NLP, 2022)

Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling. - Puyuan Peng; David Harwath (CoRR, 2022)

Audio-Visual Neural Syntax Acquisition. - Cheng-I Jeff Lai; Freda Shi; Puyuan Peng; Yoon Kim; Kevin Gimpel; Shiyu Chang; Yung-Sung Chuang; Saurabhchand Bhati; David D. Cox; David Harwath; Yang Zhang 001; Karen Livescu; et al. (ASRU, 2023)

Style-transfer based Speech and Audio-visual Scene understanding for Robot Action Sequence Acquisition from Videos. - Chiori Hori; Puyuan Peng; David Harwath; Xinyu Liu; Kei Ota; Siddarth Jain; Radu Corcodel; Devesh K. Jha; Diego Romeres; Jonathan Le Roux (INTERSPEECH, 2023)

Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model. - Puyuan Peng; Shang-Wen Li 001; Okko Räsänen; Abdelrahman Mohamed; David Harwath (INTERSPEECH, 2023)

Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalization. - Puyuan Peng; Brian Yan; Shinji Watanabe 001; David Harwath (INTERSPEECH, 2023)

Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Mode. - Puyuan Peng; Shang-Wen Li 001; Okko Räsänen; Abdelrahman Mohamed; David Harwath (CoRR, 2023)

AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models. - Yuan Tseng; Layne Berry; Yi-Ting Chen; I-Hsiang Chiu; Hsuan-Hao Lin; Max Liu; Puyuan Peng; Yi-Jen Shih; Hung-Yu Wang; Haibin Wu; Po-Yao Huang 001; Chun-Mao Lai; et al. (CoRR, 2023)

VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild. - Puyuan Peng; Po-Yao Huang 001; Shang-Wen Li 001; Abdelrahman Mohamed; David Harwath (ACL (1), 2024)

Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos. - Changan Chen; Puyuan Peng; Ami Baid; Zihui Xue; Wei-Ning Hsu; David Harwath; Kristen Grauman (ECCV (70), 2024)

Integrating Self-Supervised Speech Model with Pseudo Word-Level Targets from Visually-Grounded Speech Model. - Hung-Chieh Fang; Nai-Xuan Ye; Yi-Jen Shih; Puyuan Peng; Hsuan-Fu Wang; Layne Berry; Hung-Yi Lee; David Harwath (ICASSP Workshops, 2024)

AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models. - Yuan Tseng; Layne Berry; Yiting Chen; I-Hsiang Chiu; Hsuan-Hao Lin; Max Liu; Puyuan Peng; Yi-Jen Shih; Hung-Yu Wang; Haibin Wu; Poyao Huang 001; Chun-Mao Lai; et al. (ICASSP, 2024)

SpeechCLIP+: Self-Supervised Multi-Task Representation Learning for Speech Via Clip and Speech-Image Data. - Hsuan-Fu Wang; Yi-Jen Shih; Heng-Jui Chang; Layne Berry; Puyuan Peng; Hung-Yi Lee; Hsin-Min Wang; David Harwath (ICASSP Workshops, 2024)

BAT: Learning to Reason about Spatial Sounds with Large Language Models. - Zhisheng Zheng; Puyuan Peng; Ziyang Ma 001; Xie Chen 001; Eunsol Choi; David Harwath (ICML, 2024)

Neural Codec Language Models for Disentangled and Textless Voice Conversion. - Alan Baade; Puyuan Peng; David Harwath (INTERSPEECH, 2024)

VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild. - Puyuan Peng; Po-Yao Huang 001; Daniel Li 006; Abdelrahman Mohamed; David Harwath (CoRR, 2024)

Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks. - Chien-yu Huang; Wei-Chih Chen; Shu-Wen Yang; Andy T. Liu; Chen-An Li; Yu-Xiang Lin; Wei-Cheng Tseng; Anuj Diwan; Yi-Jen Shih; Jiatong Shi; William Chen; Xuanjun Chen; et al. (CoRR, 2024)

SyllableLM: Learning Coarse Semantic Units for Speech Language Models. - Alan Baade; Puyuan Peng; David Harwath (ICLR, 2025)

Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks. - Chien-yu Huang; Wei-Chih Chen; Shu-Wen Yang; Andy T. Liu; Chen-An Li; Yu-Xiang Lin; Wei-Cheng Tseng; Anuj Diwan; Yi-Jen Shih; Jiatong Shi; William Chen; Chih-Kai Yang; et al. (ICLR, 2025)

Temporally Streaming Audio-Visual Synchronization for Real-World Videos. - Jordan Voas; Wei-Cheng Tseng; Layne Berry; Xixi Hu 001; Puyuan Peng; James Stuedemann; David Harwath (WACV, 2025)

VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models. - Kim Sung-Bin; Jeongsoo Choi; Puyuan Peng; Joon Son Chung; Tae-Hyun Oh; David Harwath (CoRR, 2025)

ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models. - Liyan Tang; Grace Kim; Xinyu Zhao; Thom Lake; Wenxuan Ding 003; Fangcong Yin; Prasann Singhal; Manya Wadhwa; Zeyu Leo Liu; Zayne Sprague; Ramya Namuduri; Bodun Hu; et al. (CoRR, 2025)

VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation. - Puyuan Peng; Shang-Wen Li 001; Abdelrahman Mohamed; David Harwath (CoRR, 2025)

TalkLess: Blending Extractive and Abstractive Speech Summarization for Editing Speech to Preserve Content and Style. - Karim Benharrak; Puyuan Peng; Amy Pavel (CoRR, 2025)