Topic identification based extrinsic evaluation of summarization techniques applied to conversational speech. - David F. Harwath; Timothy J. Hazen (ICASSP, 2012)

Zero resource spoken audio corpus analysis. - David F. Harwath; Timothy J. Hazen; James R. Glass (ICASSP, 2013)

A summary of the 2012 JHU CLSP workshop on zero resource speech technologies and models of early language acquisition. - Aren Jansen; Emmanuel Dupoux; Sharon Goldwater; Mark Johnson 001; Sanjeev Khudanpur; Kenneth Church 001; Naomi Feldman; Hynek Hermansky; Florian Metze; Richard C. Rose; Mike Seltzer; Pascal Clark; et al. (ICASSP, 2013)

Speech recognition without a lexicon - bridging the gap between graphemic and phonetic systems. - David F. Harwath; James R. Glass (INTERSPEECH, 2014)

Choosing useful word alternates for automatic speech recognition correction interfaces. - David Harwath; Alexander Gruenstein; Ian McGraw (INTERSPEECH, 2014)

Deep multimodal semantic embeddings for speech and images. - David F. Harwath; James R. Glass (ASRU, 2015)

Unsupervised Learning of Spoken Language with Visual Context. - David F. Harwath; Antonio Torralba 001; James R. Glass (NIPS, 2016)

Look, listen, and decode: Multimodal speech recognition with images. - Felix Sun; David F. Harwath; James R. Glass (SLT, 2016)

On the Use of Acoustic Unit Discovery for Language Recognition. - Stephen H. Shum; David F. Harwath; Najim Dehak; James R. Glass (IEEE ACM Trans. Audio Speech Lang. Process., 2016)

Learning Word-Like Units from Joint Audio-Visual Analysis. - David Harwath; James R. Glass (ACL (1), 2017)

Learning modality-invariant representations for speech and images. - Kenneth Leidal; David Harwath; James R. Glass (ASRU, 2017)

Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speech. - David Harwath; Galen Chuang; James R. Glass (ICASSP, 2018)

Learning spoken language through vision. - David Frank Harwath (2018)

Grounding Spoken Words in Unlabeled Video. - Angie W. Boggust; Kartik Audhkhasi; Dhiraj Joshi; David Harwath; Samuel Thomas 001; Rogério Schmidt Feris; Danny Gutfreund; Yang Zhang 001; Antonio Torralba 001; Michael Picheny; James R. Glass (CVPR Workshops, 2019)

Learning Words by Drawing Images. - Didac Suris; Adrià Recasens; David Bau; David Harwath; James R. Glass; Antonio Torralba 001 (CVPR, 2019)

Towards Visually Grounded Sub-word Speech Unit Discovery. - David Harwath; James R. Glass (ICASSP, 2019)

Towards Bilingual Lexicon Discovery From Visually Grounded Speech Audio. - Emmanuel Azuh; David Harwath; James R. Glass (INTERSPEECH, 2019)

Transfer Learning from Audio-Visual Grounding to Speech Recognition. - Wei-Ning Hsu; David Harwath; James R. Glass (INTERSPEECH, 2019)

Trilingual Semantic Embeddings of Visually Grounded Speech with Self-Attention Mechanisms. - Yasunori Ohishi; Akisato Kimura; Takahito Kawanishi; Kunio Kashino; David Harwath; James R. Glass (ICASSP, 2020)

Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech. - David Harwath; Wei-Ning Hsu; James R. Glass (ICLR, 2020)

Pair Expansion for Learning Multilingual Semantic Embeddings Using Disjoint Visually-Grounded Speech Audio Datasets. - Yasunori Ohishi; Akisato Kimura; Takahito Kawanishi; Kunio Kashino; David Harwath; James R. Glass (INTERSPEECH, 2020)

Text-Free Image-to-Speech Synthesis Using Learned Segmental Units. - Wei-Ning Hsu; David Harwath; Christopher Song; James R. Glass (CoRR, 2020)

Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input. - David Harwath; Adrià Recasens; Dídac Surís; Galen Chuang; Antonio Torralba 001; James R. Glass (Int. J. Comput. Vis., 2020)

Text-Free Image-to-Speech Synthesis Using Learned Segmental Units. - Wei-Ning Hsu; David Harwath; Tyler Miller; Christopher Song; James R. Glass (ACL/IJCNLP (1), 2021)

Spoken Moments: Learning Joint Audio-Visual Representations From Video Descriptions. - Mathew Monfort; SouYoung Jin; Alexander H. Liu; David Harwath; Rogério Feris; James R. Glass; Aude Oliva (CVPR, 2021)

Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos. - Brian Chen 001; Andrew Rouditchenko; Kevin Duarte; Hilde Kuehne; Samuel Thomas 001; Angie W. Boggust; Rameswar Panda; Brian Kingsbury; Rogério Feris; David Harwath; James R. Glass; Michael Picheny; et al. (ICCV, 2021)

Cascaded Multilingual Audio-Visual Learning from Videos. - Andrew Rouditchenko; Angie W. Boggust; David Harwath; Samuel Thomas 001; Hilde Kuehne; Brian Chen 001; Rameswar Panda; Rogério Feris; Brian Kingsbury; Michael Picheny; James R. Glass (Interspeech, 2021)

AVLnet: Learning Audio-Visual Language Representations from Instructional Videos. - Andrew Rouditchenko; Angie W. Boggust; David Harwath; Brian Chen 001; Dhiraj Joshi; Samuel Thomas 001; Kartik Audhkhasi; Hilde Kuehne; Rameswar Panda; Rogério Schmidt Feris; Brian Kingsbury; Michael Picheny; et al. (Interspeech, 2021)

Routing with Self-Attention for Multimodal Capsule Networks. - Kevin Duarte; Brian Chen 001; Nina Shvetsova; Andrew Rouditchenko; Samuel Thomas 001; Alexander H. Liu; David Harwath; James R. Glass; Hilde Kuehne; Mubarak Shah (CoRR, 2021)

Everything at Once - Multi-modal Fusion Transformer for Video Retrieval. - Nina Shvetsova; Brian Chen 001; Andrew Rouditchenko; Samuel Thomas 001; Brian Kingsbury; Rogério Feris; David Harwath; James R. Glass; Hilde Kuehne (CVPR, 2022)

Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality. - Anuj Diwan; Layne Berry; Eunsol Choi; David Harwath; Kyle Mahowald (EMNLP, 2022)

Fast-Slow Transformer for Visually Grounding Speech. - Puyuan Peng; David Harwath (ICASSP, 2022)

Adversarial Input Ablation for Audio-Visual Learning. - David Xu 006; David Harwath (ICASSP, 2022)

MAE-AST: Masked Autoencoding Audio Spectrogram Transformer. - Alan Baade; Puyuan Peng; David Harwath (INTERSPEECH, 2022)

Exploring Few-Shot Fine-Tuning Strategies for Models of Visually Grounded Speech. - Tyler Miller; David Harwath (INTERSPEECH, 2022)

Word Discovery in Visually Grounded, Self-Supervised Speech Models. - Puyuan Peng; David Harwath (INTERSPEECH, 2022)

Speak: A Toolkit Using Amazon Mechanical Turk to Collect and Validate Speech Audio Recordings. - Christopher Song; David Harwath; Tuka Alhanai; James R. Glass (LREC, 2022)

SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model. - Yi-Jen Shih; Hsuan-Fu Wang; Heng-Jui Chang; Layne Berry; Hung-yi Lee; David Harwath (SLT, 2022)

Phoneme Segmentation Using Self-Supervised Speech Models. - Luke Strgar; David Harwath (SLT, 2022)

Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling. - Puyuan Peng; David Harwath (CoRR, 2022)

Automated detection of foreground speech with wearable sensing in everyday home environments: A transfer learning approach. - Dawei Liang; Zifan Xu; Yinuo Chen; Rebecca Adaimi; David Harwath; Edison Thomaz (CoRR, 2022)

When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants. - Anuj Diwan; Eunsol Choi; David Harwath (ACL (2), 2023)

Audio-Visual Neural Syntax Acquisition. - Cheng-I Jeff Lai; Freda Shi; Puyuan Peng; Yoon Kim; Kevin Gimpel; Shiyu Chang; Yung-Sung Chuang; Saurabhchand Bhati; David D. Cox; David Harwath; Yang Zhang 001; Karen Livescu; et al. (ASRU, 2023)

M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval. - Layne Berry; Yi-Jen Shih; Hsuan-Fu Wang; Heng-Jui Chang; Hung-Yi Lee; David Harwath (ICASSP, 2023)

Learning Audio-Visual Dereverberation. - Changan Chen; Wei Sun; David Harwath; Kristen Grauman (ICASSP, 2023)

Continual Learning for On-Device Speech Recognition Using Disentangled Conformers. - Anuj Diwan; Ching-Feng Yeh; Wei-Ning Hsu; Paden Tomasello; Eunsol Choi; David Harwath; Abdelrahman Mohamed (ICASSP, 2023)

Unsupervised Fine-Tuning Data Selection for ASR Using Self-Supervised Speech Models. - Reem Gody; David Harwath (ICASSP, 2023)

A Dataset for Foreground Speech Analysis With Smartwatches In Everyday Home Environments. - Dawei Liang; Zifan Xu; Yinuo Chen; Rebecca Adaimi; David Harwath; Edison Thomaz (ICASSP Workshops, 2023)

C2KD: Cross-Lingual Cross-Modal Knowledge Distillation for Multilingual Text-Video Retrieval. - Andrew Rouditchenko; Yung-Sung Chuang; Nina Shvetsova; Samuel Thomas 001; Rogério Feris; Brian Kingsbury; Leonid Karlinsky; David Harwath; Hilde Kuehne; James R. Glass (ICASSP, 2023)

Contrastive Audio-Visual Masked Autoencoder. - Yuan Gong 001; Andrew Rouditchenko; Alexander H. Liu; David Harwath; Leonid Karlinsky; Hilde Kuehne; James R. Glass (ICLR, 2023)

Style-transfer based Speech and Audio-visual Scene understanding for Robot Action Sequence Acquisition from Videos. - Chiori Hori; Puyuan Peng; David Harwath; Xinyu Liu; Kei Ota; Siddarth Jain; Radu Corcodel; Devesh K. Jha; Diego Romeres; Jonathan Le Roux (INTERSPEECH, 2023)

Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model. - Puyuan Peng; Shang-Wen Li 001; Okko Räsänen; Abdelrahman Mohamed; David Harwath (INTERSPEECH, 2023)

Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalization. - Puyuan Peng; Brian Yan; Shinji Watanabe 001; David Harwath (INTERSPEECH, 2023)

Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages. - Andrew Rouditchenko; Sameer Khurana; Samuel Thomas 001; Rogério Feris; Leonid Karlinsky; Hilde Kuehne; David Harwath; Brian Kingsbury; James R. Glass (INTERSPEECH, 2023)

Learning to Map Efficiently by Active Echolocation. - Xixi Hu 001; Senthil Purushwalkam; David Harwath; Kristen Grauman (IROS, 2023)

Subject Generalization in Classifying Imagined and Spoken Speech with MEG. - Debadatta Dash; Paul Ferrari; Abbas Babajani-Feremi; David Harwath; Amir Borna; Jun Wang 037 (NER, 2023)

Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Mode. - Puyuan Peng; Shang-Wen Li 001; Okko Räsänen; Abdelrahman Mohamed; David Harwath (CoRR, 2023)

Unit-based Speech-to-Speech Translation Without Parallel Data. - Anuj Diwan; Anirudh Srinivasan; David Harwath; Eunsol Choi (CoRR, 2023)

AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models. - Yuan Tseng; Layne Berry; Yi-Ting Chen; I-Hsiang Chiu; Hsuan-Hao Lin; Max Liu; Puyuan Peng; Yi-Jen Shih; Hung-Yu Wang; Haibin Wu; Po-Yao Huang 001; Chun-Mao Lai; et al. (CoRR, 2023)

VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild. - Puyuan Peng; Po-Yao Huang 001; Shang-Wen Li 001; Abdelrahman Mohamed; David Harwath (ACL (1), 2024)

Multimodal Contextualized Semantic Parsing from Speech. - Jordan Voas; David Harwath; Raymond Mooney (ACL (1), 2024)

SoundingActions: Learning How Actions Sound from Narrated Egocentric Videos. - Changan Chen; Kumar Ashutosh; Rohit Girdhar; David Harwath; Kristen Grauman (CVPR, 2024)

Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos. - Changan Chen; Puyuan Peng; Ami Baid; Zihui Xue; Wei-Ning Hsu; David Harwath; Kristen Grauman (ECCV (70), 2024)

Textless Speech-to-Speech Translation With Limited Parallel Data. - Anuj Diwan; Anirudh Srinivasan; David Harwath; Eunsol Choi (EMNLP (Findings), 2024)

Integrating Self-Supervised Speech Model with Pseudo Word-Level Targets from Visually-Grounded Speech Model. - Hung-Chieh Fang; Nai-Xuan Ye; Yi-Jen Shih; Puyuan Peng; Hsuan-Fu Wang; Layne Berry; Hung-Yi Lee; David Harwath (ICASSP Workshops, 2024)

AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models. - Yuan Tseng; Layne Berry; Yiting Chen; I-Hsiang Chiu; Hsuan-Hao Lin; Max Liu; Puyuan Peng; Yi-Jen Shih; Hung-Yu Wang; Haibin Wu; Poyao Huang 001; Chun-Mao Lai; et al. (ICASSP, 2024)

SpeechCLIP+: Self-Supervised Multi-Task Representation Learning for Speech Via Clip and Speech-Image Data. - Hsuan-Fu Wang; Yi-Jen Shih; Heng-Jui Chang; Layne Berry; Puyuan Peng; Hung-Yi Lee; Hsin-Min Wang; David Harwath (ICASSP Workshops, 2024)

BAT: Learning to Reason about Spatial Sounds with Large Language Models. - Zhisheng Zheng; Puyuan Peng; Ziyang Ma 001; Xie Chen 001; Eunsol Choi; David Harwath (ICML, 2024)

Neural Codec Language Models for Disentangled and Textless Voice Conversion. - Alan Baade; Puyuan Peng; David Harwath (INTERSPEECH, 2024)

Direct Speech Synthesis from Non-Invasive, Neuromagnetic Signals. - Jinuk Kwon; David Harwath; Debadatta Dash; Paul Ferrari; Jun Wang 037 (INTERSPEECH, 2024)

Improving Audio Classification with Low-Sampled Microphone Input: An Empirical Study Using Model Self-Distillation. - Dawei Liang; Alice Zhang; David Harwath; Edison Thomaz (INTERSPEECH, 2024)

Interface Design for Self-Supervised Speech Models. - Yi-Jen Shih; David Harwath (INTERSPEECH, 2024)

Self-Supervised Speech Models For Word-Level Stuttered Speech Detection. - Yi-Jen Shih; Zoi Gkalitsiou; Alexandros G. Dimakis; David Harwath (SLT, 2024)

Measuring Sound Symbolism In Audio-Visual Models. - Wei-Cheng Tseng; Yi-Jen Shih; David Harwath; Raymond Mooney (SLT, 2024)

VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild. - Puyuan Peng; Po-Yao Huang 001; Daniel Li 006; Abdelrahman Mohamed; David Harwath (CoRR, 2024)

Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks. - Chien-yu Huang; Wei-Chih Chen; Shu-Wen Yang; Andy T. Liu; Chen-An Li; Yu-Xiang Lin; Wei-Cheng Tseng; Anuj Diwan; Yi-Jen Shih; Jiatong Shi; William Chen; Xuanjun Chen; et al. (CoRR, 2024)

How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario. - Shih-Heng Wang; Zih-Ching Chen; Jiatong Shi; Ming-To Chuang; Guan-Ting Lin; Kuan-Po Huang; David Harwath; Shang-Wen Li 001; Hung-yi Lee (CoRR, 2024)

SyllableLM: Learning Coarse Semantic Units for Speech Language Models. - Alan Baade; Puyuan Peng; David Harwath (ICLR, 2025)

Temporally Streaming Audio-Visual Synchronization for Real-World Videos. - Jordan Voas; Wei-Cheng Tseng; Layne Berry; Xixi Hu 001; Puyuan Peng; James Stuedemann; David Harwath (WACV, 2025)

Scaling Rich Style-Prompted Text-to-Speech Datasets. - Anuj Diwan; Zhisheng Zheng; David Harwath; Eunsol Choi (CoRR, 2025)

VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models. - Kim Sung-Bin; Jeongsoo Choi; Puyuan Peng; Joon Son Chung; Tae-Hyun Oh; David Harwath (CoRR, 2025)

Rhapsody: A Dataset for Highlight Detection in Podcasts. - Younghan Park; Anuj Diwan; David Harwath; Eunsol Choi (CoRR, 2025)

VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation. - Puyuan Peng; Shang-Wen Li 001; Abdelrahman Mohamed; David Harwath (CoRR, 2025)

Probing the Robustness Properties of Neural Speech Codecs. - Wei-Cheng Tseng; David Harwath (CoRR, 2025)