Diffusion Denoising Process for Perceptron Bias in Out-of-distribution Detection. - Luping Liu; Yi Ren 006; Xize Cheng; Zhou Zhao 001 (CoRR, 2022)

OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality Alignment. - Xize Cheng; Tao Jin 004; Linjun Li; Wang Lin; Xinyu Duan; Zhou Zhao 001 (ACL (1), 2023)

AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation. - Rongjie Huang 001; Huadai Liu; Xize Cheng; Yi Ren 006; Linjun Li; Zhenhui Ye; Jinzheng He; Lichao Zhang; Jinglin Liu; Xiang Yin 006; Zhou Zhao 001 (ACL (1), 2023)

Contrastive Token-Wise Meta-Learning for Unseen Performer Visual Temporal-Aligned Translation. - Linjun Li; Tao Jin 004; Xize Cheng; Ye Wang 018; Wang Lin; Rongjie Huang 001; Zhou Zhao 001 (ACL (Findings), 2023)

TAVT: Towards Transferable Audio-Visual Text Generation. - Wang Lin; Tao Jin 004; Wenwen Pan 003; Linjun Li; Xize Cheng; Ye Wang 018; Zhou Zhao 001 (ACL (1), 2023)

Semantic-conditioned Dual Adaptation for Cross-domain Query-based Visual Segmentation. - Ye Wang 018; Tao Jin 004; Wang Lin; Xize Cheng; Linjun Li; Zhou Zhao 001 (ACL (Findings), 2023)

Weakly-Supervised Spoken Video Grounding via Semantic Interaction Learning. - Ye Wang 018; Wang Lin; Shengyu Zhang 001; Tao Jin 004; Linjun Li; Xize Cheng; Zhou Zhao 001 (ACL (1), 2023)

3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding. - Zehan Wang 001; Haifeng Huang; Yang Zhao 022; Linjun Li; Xize Cheng; Yichen Zhu; Aoxiong Yin; Zhou Zhao 001 (EMNLP, 2023)

MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition. - Xize Cheng; Tao Jin 004; Rongjie Huang 001; Linjun Li; Wang Lin; Zehan Wang 001; Ye Wang 018; Huadai Liu; Aoxiong Yin; Zhou Zhao 001 (ICCV, 2023)

Exploring Group Video Captioning with Efficient Relational Approximation. - Wang Lin; Tao Jin 004; Ye Wang 018; Wenwen Pan 003; Linjun Li; Xize Cheng; Zhou Zhao 001 (ICCV, 2023)

Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding. - Zehan Wang 001; Haifeng Huang; Yang Zhao 022; Linjun Li; Xize Cheng; Yichen Zhu; Aoxiong Yin; Zhou Zhao 001 (ICCV, 2023)

Rethinking Missing Modality Learning from a Decoding Perspective. - Tao Jin 004; Xize Cheng; Linjun Li; Wang Lin; Ye Wang 018; Zhou Zhao 001 (ACM Multimedia, 2023)

Connecting Multi-modal Contrastive Representations. - Zehan Wang 001; Yang Zhao 022; Xize Cheng; Haifeng Huang; Jiageng Liu; Aoxiong Yin; Li Tang; Linjun Li; Yongqi Wang; Ziang Zhang; Zhou Zhao 001 (NeurIPS, 2023)

MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition. - Xize Cheng; Linjun Li; Tao Jin 004; Rongjie Huang 001; Wang Lin; Zehan Wang 001; Huangdai Liu; Ye Wang 018; Aoxiong Yin; Zhou Zhao 001 (CoRR, 2023)

Chat-3D v2: Bridging 3D Scene and Large Language Models with Object Identifiers. - Haifeng Huang; Zehan Wang 001; Rongjie Huang 001; Luping Liu; Xize Cheng; Yang Zhao 022; Tao Jin 004; Zhou Zhao 001 (CoRR, 2023)

TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation. - Xize Cheng; Rongjie Huang 001; Linjun Li; Tao Jin 004; Zehan Wang 001; Aoxiong Yin; Minglei Li; Xinyu Duan; Changpeng Yang; Zhou Zhao 001 (CoRR, 2023)

TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation. - Xize Cheng; Rongjie Huang 001; Linjun Li; Zehan Wang 001; Tao Jin 004; Aoxiong Yin; Feiyang Chen; Xinyu Duan; Baoxing Huai; Zhou Zhao 001 (ACL (Findings), 2024)

Text-to-Song: Towards Controllable Music Generation Incorporating Vocal and Accompaniment. - Zhiqing Hong; Rongjie Huang 001; Xize Cheng; Yongqi Wang; Ruiqi Li; Fuming You; Zhou Zhao 001; Zhimeng Zhang (ACL (1), 2024)

Rethinking the Multimodal Correlation of Multimodal Sequential Learning via Generalizable Attentional Results Alignment. - Tao Jin 004; Wang Lin; Ye Wang 018; Linjun Li; Xize Cheng; Zhou Zhao 001 (ACL (1), 2024)

Uni-Dubbing: Zero-Shot Speech Synthesis from Visual Articulation. - Songju Lei; Xize Cheng; Mengjiao Lyu; Jianqiao Hu; Jintao Tan; Runlin Liu; Lingyu Xiong; Tao Jin 004; Xiandong Li; Zhou Zhao 001 (ACL (1), 2024)

Wav2SQL: Direct Generalizable Speech-To-SQL Parsing. - Huadai Liu; Rongjie Huang 001; Jinzheng He; Gang Sun 004; Ran Shen; Xize Cheng; Zhou Zhao 001 (ACL (Findings), 2024)

AudioVSR: Enhancing Video Speech Recognition with Audio Data. - Xiaoda Yang; Xize Cheng; Jiaqi Duan; Hongshun Qiu; Minjie Hong; Minghui Fang 002; Shengpeng Ji; Jialong Zuo; Zhiqing Hong; Zhimeng Zhang; Tao Jin 004 (EMNLP, 2024)

Landmark-guided Diffusion Model for High-fidelity and Temporally Coherent Talking Head Generation. - Jintao Tan; Xize Cheng; Lingyu Xiong; Lei Zhu; Xiandong Li; Xianjia Wu; Kai Gong; Minglei Li; Yi Cai (ICME, 2024)

FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion. - Zehan Wang 001; Ziang Zhang; Xize Cheng; Rongjie Huang 001; Luping Liu; Zhenhui Ye; Haifeng Huang; Yang Zhao 022; Tao Jin 004; Peng Gao 007; Zhou Zhao 001 (ICML, 2024)

InstructSpeech: Following Speech Editing Instructions via Large Language Models. - Rongjie Huang 001; Ruofan Hu; Yongqi Wang; Zehan Wang 001; Xize Cheng; Ziyue Jiang 001; Zhenhui Ye; Dongchao Yang; Luping Liu; Peng Gao 007; Zhou Zhao 001 (ICML, 2024)

Boosting Speech Recognition Robustness to Modality-Distortion with Contrast-Augmented Prompts. - Dongjie Fu; Xize Cheng; Xiaoda Yang; Hanting Wang; Zhou Zhao 001; Tao Jin 004 (ACM Multimedia, 2024)

VoiceTuner: Self-Supervised Pre-training and Efficient Fine-tuning For Voice Generation. - Rongjie Huang 001; Yongqi Wang; Ruofan Hu; Xiaoshan Xu; Zhiqing Hong; Dongchao Yang; Xize Cheng; Zehan Wang 001; Ziyue Jiang 001; Zhenhui Ye; Luping Liu; Siqi Zheng; et al. (ACM Multimedia, 2024)

AudioLCM: Efficient and High-Quality Text-to-Audio Generation with Minimal Inference Steps. - Huadai Liu; Rongjie Huang 001; Yang Liu 278; Hengyuan Cao; Jialei Wang; Xize Cheng; Siqi Zheng; Zhou Zhao 001 (ACM Multimedia, 2024)

SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing. - Lingyu Xiong; Xize Cheng; Jintao Tan; Xianjia Wu; Xiandong Li; Lei Zhu; Fei Ma 006; Minglei Li 001; Huang Xu 003; Zhihui Hu (ACM Multimedia, 2024)

SyncTalklip: Highly Synchronized Lip-Readable Speaker Generation with Multi-Task Learning. - Xiaoda Yang; Xize Cheng; Dongjie Fu; Minghui Fang 002; Jialong Zuo; Shengpeng Ji; Zhou Zhao 001; Tao Jin 004 (ACM Multimedia, 2024)

Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers. - Haifeng Huang; Yilun Chen; Zehan Wang 001; Rongjie Huang 001; Runsen Xu; Tai Wang; Luping Liu; Xize Cheng; Yang Zhao 022; Jiangmiao Pang; Zhou Zhao 001 (NeurIPS, 2024)

MimicTalk: Mimicking a personalized and expressive 3D talking face in minutes. - Zhenhui Ye; Tianyun Zhong; Yi Ren 006; Ziyue Jiang 001; Jiawei Huang 008; Rongjie Huang 001; Jinglin Liu; Jinzheng He; Chen Zhang 020; Zehan Wang 001; Xize Cheng; Xiang Yin 006; et al. (NeurIPS, 2024)

Extending Multi-modal Contrastive Representations. - Ziang Zhang; Zehan Wang 001; Luping Liu; Rongjie Huang 001; Xize Cheng; Zhenhui Ye; Wang Lin; Huadai Liu; Haifeng Huang; Yang Zhao 022; Tao Jin 004; Siqi Zheng; et al. (NeurIPS, 2024)

Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompaniment. - Zhiqing Hong; Rongjie Huang 001; Xize Cheng; Yongqi Wang; Ruiqi Li; Fuming You; Zhou Zhao 001; Zhimeng Zhang (CoRR, 2024)

AudioLCM: Text-to-Audio Generation with Latent Consistency Models. - Huadai Liu; Rongjie Huang 001; Yang Liu 278; Hengyuan Cao; Jialei Wang; Xize Cheng; Siqi Zheng; Zhou Zhao 001 (CoRR, 2024)

ControlSpeech: Towards Simultaneous Zero-shot Speaker Cloning and Zero-shot Language Style Control With Decoupled Codec. - Shengpeng Ji; Jialong Zuo; Minghui Fang 002; Siqi Zheng; Qian Chen 003; Wen Wang 001; Ziyue Jiang 001; Hai Huang 013; Xize Cheng; Rongjie Huang 001; Zhou Zhao 001 (CoRR, 2024)

ACE: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling. - Minghui Fang 002; Shengpeng Ji; Jialong Zuo; Hai Huang 013; Yan Xia 006; Jieming Zhu; Xize Cheng; Xiaoda Yang; Wenrui Liu 003; Gang Wang; Zhenhua Dong; Zhou Zhao 001 (CoRR, 2024)

OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces. - Zehan Wang 001; Ziang Zhang; Hang Zhang; Luping Liu; Rongjie Huang 001; Xize Cheng; Hengshuang Zhao; Zhou Zhao 001 (CoRR, 2024)

WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling. - Shengpeng Ji; Ziyue Jiang 001; Xize Cheng; Yifu Chen; Minghui Fang 002; Jialong Zuo; Qian Yang 006; Ruiqi Li; Ziang Zhang; Xiaoda Yang; Rongjie Huang 001; Yidi Jiang; et al. (CoRR, 2024)

SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing. - Lingyu Xiong; Xize Cheng; Jintao Tan; Xianjia Wu; Xiandong Li; Lei Zhu; Fei Ma 006; Minglei Li 001; Huang Xu 003; Zhihu Hu (CoRR, 2024)

MuVi: Video-to-Music Generation with Semantic Alignment and Rhythmic Synchronization. - Ruiqi Li; Siqi Zheng; Xize Cheng; Ziang Zhang; Shengpeng Ji; Zhou Zhao 001 (CoRR, 2024)

OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup. - Xize Cheng; Siqi Zheng; Zehan Wang 001; Minghui Fang 002; Ziang Zhang; Rongjie Huang 001; Ziyang Ma; Shengpeng Ji; Jialong Zuo; Tao Jin 004; Zhou Zhao 001 (CoRR, 2024)

WavChat: A Survey of Spoken Dialogue Models. - Shengpeng Ji; Yifu Chen; Minghui Fang 002; Jialong Zuo; Jingyu Lu; Hanting Wang; Ziyue Jiang 001; Long Zhou; Shujie Liu 001; Xize Cheng; Xiaoda Yang; Zehan Wang 001; et al. (CoRR, 2024)

A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter. - Zirun Guo; Xize Cheng; Yangyang Wu; Tao Jin 004 (AAAI, 2025)

T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback. - Zehan Wang 001; Ke Lei; Chen Zhu; Jiawei Huang 008; Sashuai Zhou; Luping Liu; Xize Cheng; Shengpeng Ji; Zhenhui Ye; Tao Jin 004; Zhou Zhao 001 (ACL (1), 2025)

CART: A Generative Cross-Modal Retrieval Framework With Coarse-To-Fine Semantic Modeling. - Minghui Fang 002; Shengpeng Ji; Jialong Zuo; Hai Huang 013; Yan Xia 006; Jieming Zhu; Xize Cheng; Xiaoda Yang; Wenrui Liu 003; Gang Wang; Zhenhua Dong; Zhou Zhao 001 (ACL (1), 2025)

ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control. - Shengpeng Ji; Qian Chen 003; Wen Wang 001; Jialong Zuo; Minghui Fang 002; Ziyue Jiang 004; Hai Huang 013; Zehan Wang 001; Xize Cheng; Siqi Zheng; Zhou Zhao 001 (ACL (1), 2025)

Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching. - Jialong Zuo; Shengpeng Ji; Minghui Fang 002; Mingze Li; Ziyue Jiang 001; Xize Cheng; Xiaoda Yang; Feiyang Chen; Xinyu Duan; Zhou Zhao 001 (ACL (1), 2025)

VoxpopuliTTS: a large-scale multilingual TTS corpus for zero-shot speech generation. - Wenrui Liu 003; Jionghao Bai; Xize Cheng; Jialong Zuo; Ziyue Jiang 001; Shengpeng Ji; Minghui Fang 002; Xiaoda Yang; Qian Yang 006; Zhou Zhao 001 (COLING, 2025)

SpatialCLIP: Learning 3D-aware Image Representations from Spatially Discriminative Language. - Zehan Wang 001; Sashuai Zhou; Shaoxuan He; Haifeng Huang; Lihe Yang; Ziang Zhang; Xize Cheng; Shengpeng Ji; Tao Jin 004; Hengshuang Zhao; Zhou Zhao 001 (CVPR, 2025)

Curriculum Learning aided Audio-Visual Speech Recognition with Arbitrary Speaker Number. - Yuxiao Lin; Tao Jin 004; Xize Cheng; Zhou Zhao 001; Fei Wu 001 (ICASSP, 2025)

Enhancing Expressive Voice Conversion with Discrete Pitch-Conditioned Flow Matching Model. - Jialong Zuo; Shengpeng Ji; Minghui Fang 002; Ziyue Jiang 001; Xize Cheng; Qian Yang 006; Wenrui Liu 003; Guangyan Zhang; Zehai Tu; Yiwen Guo; Zhou Zhao 001 (ICASSP, 2025)

OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces. - Zehan Wang 001; Ziang Zhang; Minjie Hong; Hang Zhang; Luping Liu; Rongjie Huang 001; Xize Cheng; Shengpeng Ji; Tao Jin 004; Hengshuang Zhao; Zhou Zhao 001 (ICLR, 2025)

VoxDialogue: Can Spoken Dialogue Systems Understand Information Beyond Words? - Xize Cheng; Ruofan Hu; Xiaoda Yang; Jingyu Lu; Dongjie Fu; Zehan Wang 001; Shengpeng Ji; Rongjie Huang 001; Boyang Zhang; Tao Jin 004; Zhou Zhao 001 (ICLR, 2025)

OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup. - Xize Cheng; Siqi Zheng; Zehan Wang 001; Minghui Fang 002; Ziang Zhang; Rongjie Huang 001; Shengpeng Ji; Jialong Zuo; Tao Jin 004; Zhou Zhao 001 (ICLR, 2025)

WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling. - Shengpeng Ji; Ziyue Jiang 001; Wen Wang 001; Yifu Chen; Minghui Fang 002; Jialong Zuo; Qian Yang 006; Xize Cheng; Zehan Wang 001; Ruiqi Li; Ziang Zhang; Xiaoda Yang; et al. (ICLR, 2025)

OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios. - Xize Cheng; Dongjie Fu; Xiaoda Yang; Minghui Fang 002; Ruofan Hu; Jingyu Lu; Jionghao Bai; Zehan Wang 001; Shengpeng Ji; Rongjie Huang 001; Linjun Li; Yu Chen; et al. (CoRR, 2025)

Unleashing the Power of Natural Audio Featuring Multiple Sound Sources. - Xize Cheng; Slytherin Wang; Zehan Wang 001; Rongjie Huang 001; Tao Jin 004; Zhou Zhao 001 (CoRR, 2025)

WavReward: Spoken Dialogue Models With Generalist Reward Evaluators. - Shengpeng Ji; Tianle Liang; Yangzhuo Li; Jialong Zuo; Minghui Fang 002; Jinzheng He; Yifu Chen; Zhengqing Liu; Ziyue Jiang 001; Xize Cheng; Siqi Zheng; Jin Xu; et al. (CoRR, 2025)