Low-Rank HOCA: Efficient High-Order Cross-Modal Attention for Video Captioning. - Tao Jin 004; Siyu Huang; Yingming Li; Zhongfei Zhang (EMNLP/IJCNLP (1), 2019)

Recurrent convolutional video captioning with global and local attention. - Tao Jin 004; Yingming Li; Zhongfei Zhang (Neurocomputing, 2019)

Dual Low-Rank Multimodal Fusion. - Tao Jin 004; Siyu Huang; Yingming Li; Zhongfei Zhang (EMNLP (Findings), 2020)

SBAT: Video Captioning with Sparse Boundary-Aware Transformer. - Tao Jin 004; Siyu Huang; Ming Chen; Yingming Li; Zhongfei Zhang (IJCAI, 2020)

Contrastive Disentangled Meta-Learning for Signer-Independent Sign Language Translation. - Tao Jin 004; Zhou Zhao 001 (ACM Multimedia, 2021)

Generalizable Multi-linear Attention Network. - Tao Jin 004; Zhou Zhao 001 (NeurIPS, 2021)

Prior Knowledge and Memory Enriched Transformer for Sign Language Translation. - Tao Jin 004; Zhou Zhao 001; Meng Zhang 019; Xingshan Zeng (ACL (Findings), 2022)

MC-SLT: Towards Low-Resource Signer-Adaptive Sign Language Translation. - Tao Jin 004; Zhou Zhao 001; Meng Zhang 019; Xingshan Zeng (ACM Multimedia, 2022)

Interaction augmented transformer with decoupled decoding for video captioning. - Tao Jin 004; Zhou Zhao 001; Peng Wang; Jun Yu 002; Fei Wu 001 (Neurocomputing, 2022)

OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality Alignment. - Xize Cheng; Tao Jin 004; Linjun Li; Wang Lin; Xinyu Duan; Zhou Zhao 001 (ACL (1), 2023)

Contrastive Token-Wise Meta-Learning for Unseen Performer Visual Temporal-Aligned Translation. - Linjun Li; Tao Jin 004; Xize Cheng; Ye Wang 018; Wang Lin; Rongjie Huang 001; Zhou Zhao 001 (ACL (Findings), 2023)

TAVT: Towards Transferable Audio-Visual Text Generation. - Wang Lin; Tao Jin 004; Wenwen Pan 003; Linjun Li; Xize Cheng; Ye Wang 018; Zhou Zhao 001 (ACL (1), 2023)

Semantic-conditioned Dual Adaptation for Cross-domain Query-based Visual Segmentation. - Ye Wang 018; Tao Jin 004; Wang Lin; Xize Cheng; Linjun Li; Zhou Zhao 001 (ACL (Findings), 2023)

Weakly-Supervised Spoken Video Grounding via Semantic Interaction Learning. - Ye Wang 018; Wang Lin; Shengyu Zhang 001; Tao Jin 004; Linjun Li; Xize Cheng; Zhou Zhao 001 (ACL (1), 2023)

DATE: Domain Adaptive Product Seeker for E-Commerce. - Haoyuan Li 002; Hao Jiang 014; Tao Jin 004; Mengyan Li; Yan Chen; Zhijie Lin 001; Yang Zhao 022; Zhou Zhao 001 (CVPR, 2023)

Gloss Attention for Gloss-free Sign Language Translation. - Aoxiong Yin; Tianyun Zhong; Li Tang; Weike Jin; Tao Jin 004; Zhou Zhao 001 (CVPR, 2023)

MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition. - Xize Cheng; Tao Jin 004; Rongjie Huang 001; Linjun Li; Wang Lin; Zehan Wang 001; Ye Wang 018; Huadai Liu; Aoxiong Yin; Zhou Zhao 001 (ICCV, 2023)

Exploring Group Video Captioning with Efficient Relational Approximation. - Wang Lin; Tao Jin 004; Ye Wang 018; Wenwen Pan 003; Linjun Li; Xize Cheng; Zhou Zhao 001 (ICCV, 2023)

Rethinking Missing Modality Learning from a Decoding Perspective. - Tao Jin 004; Xize Cheng; Linjun Li; Wang Lin; Ye Wang 018; Zhou Zhao 001 (ACM Multimedia, 2023)

MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition. - Xize Cheng; Linjun Li; Tao Jin 004; Rongjie Huang 001; Wang Lin; Zehan Wang 001; Huangdai Liu; Ye Wang 018; Aoxiong Yin; Zhou Zhao 001 (CoRR, 2023)

Extending Multi-modal Contrastive Representations. - Zehan Wang 001; Ziang Zhang; Luping Liu; Yang Zhao 022; Haifeng Huang; Tao Jin 004; Zhou Zhao 001 (CoRR, 2023)

Chat-3D v2: Bridging 3D Scene and Large Language Models with Object Identifiers. - Haifeng Huang; Zehan Wang 001; Rongjie Huang 001; Luping Liu; Xize Cheng; Yang Zhao 022; Tao Jin 004; Zhou Zhao 001 (CoRR, 2023)

TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation. - Xize Cheng; Rongjie Huang 001; Linjun Li; Tao Jin 004; Zehan Wang 001; Aoxiong Yin; Minglei Li; Xinyu Duan; Changpeng Yang; Zhou Zhao 001 (CoRR, 2023)

Electromagnetic Imaging Boosted Visual Object Recognition Under Difficult Visual Conditions. - Min Tan 005; Tao Jin 004; Danhui Ye; Kuiwen Xu; Xiaoling Gu; Jun Yu 002 (IEEE Trans. Geosci. Remote. Sens., 2023)

TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation. - Xize Cheng; Rongjie Huang 001; Linjun Li; Zehan Wang 001; Tao Jin 004; Aoxiong Yin; Feiyang Chen; Xinyu Duan; Baoxing Huai; Zhou Zhao 001 (ACL (Findings), 2024)

Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognition. - Zirun Guo; Tao Jin 004; Zhou Zhao 001 (ACL (1), 2024)

Rethinking the Multimodal Correlation of Multimodal Sequential Learning via Generalizable Attentional Results Alignment. - Tao Jin 004; Wang Lin; Ye Wang 018; Linjun Li; Xize Cheng; Zhou Zhao 001 (ACL (1), 2024)

Uni-Dubbing: Zero-Shot Speech Synthesis from Visual Articulation. - Songju Lei; Xize Cheng; Mengjiao Lyu; Jianqiao Hu; Jintao Tan; Runlin Liu; Lingyu Xiong; Tao Jin 004; Xiandong Li; Zhou Zhao 001 (ACL (1), 2024)

MPOD123: One Image to 3D Content Generation Using Mask-Enhanced Progressive Outline-to-Detail Optimization. - Jimin Xu; Tianbao Wang; Tao Jin 004; Shengyu Zhang 001; Dongjie Fu; Zhe Wang; Jiangjing Lyu; Chengfei Lv; Chaoyue Niu; Zhou Yu 001; Zhou Zhao 001; Fei Wu 001 (CVPR, 2024)

AudioVSR: Enhancing Video Speech Recognition with Audio Data. - Xiaoda Yang; Xize Cheng; Jiaqi Duan; Hongshun Qiu; Minjie Hong; Minghui Fang 002; Shengpeng Ji; Jialong Zuo; Zhiqing Hong; Zhimeng Zhang; Tao Jin 004 (EMNLP, 2024)

FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion. - Zehan Wang 001; Ziang Zhang; Xize Cheng; Rongjie Huang 001; Luping Liu; Zhenhui Ye; Haifeng Huang; Yang Zhao 022; Tao Jin 004; Peng Gao 007; Zhou Zhao 001 (ICML, 2024)

Non-confusing Generation of Customized Concepts in Diffusion Models. - Wang Lin; Jingyuan Chen; Jiaxin Shi; Yichen Zhu; Chen Liang; Junzhong Miao; Tao Jin 004; Zhou Zhao 001; Fei Wu 001; Shuicheng Yan; Hanwang Zhang (ICML, 2024)

EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration. - Ye Wang 018; Jiahao Xun; Minjie Hong; Jieming Zhu; Tao Jin 004; Wang Lin; Haoyuan Li 002; Linjun Li; Yan Xia 006; Zhou Zhao 001; Zhenhua Dong (KDD, 2024)

Calibrating Prompt from History for Continual Vision-Language Retrieval and Grounding. - Tao Jin 004; Weicai Yan; Ye Wang 018; Sihang Cai; Qifan Shuai; Zhou Zhao 001 (ACM Multimedia, 2024)

Boosting Speech Recognition Robustness to Modality-Distortion with Contrast-Augmented Prompts. - Dongjie Fu; Xize Cheng; Xiaoda Yang; Hanting Wang; Zhou Zhao 001; Tao Jin 004 (ACM Multimedia, 2024)

Low-rank Prompt Interaction for Continual Vision-Language Retrieval. - Weicai Yan; Ye Wang 018; Wang Lin; Zirun Guo; Zhou Zhao 001; Tao Jin 004 (ACM Multimedia, 2024)

SyncTalklip: Highly Synchronized Lip-Readable Speaker Generation with Multi-Task Learning. - Xiaoda Yang; Xize Cheng; Dongjie Fu; Minghui Fang 002; Jialong Zuo; Shengpeng Ji; Zhou Zhao 001; Tao Jin 004 (ACM Multimedia, 2024)

Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt. - Yongqi Wang; Ruofan Hu; Rongjie Huang 001; Zhiqing Hong; Ruiqi Li; Wenrui Liu 003; Fuming You; Tao Jin 004; Zhou Zhao 001 (NAACL-HLT, 2024)

Classifier-guided Gradient Modulation for Enhanced Multimodal Learning. - Zirun Guo; Tao Jin 004; Jingyuan Chen; Zhou Zhao 001 (NeurIPS, 2024)

Action Imitation in Common Action Space for Customized Action Image Synthesis. - Wang Lin; Jingyuan Chen; Jiaxin Shi; Zirun Guo; Yichen Zhu; Zehan Wang 001; Tao Jin 004; Zhou Zhao 001; Fei Wu 001; Shuicheng Yan; Hanwang Zhang (NeurIPS, 2024)

E3: Exploring Embodied Emotion Through A Large-Scale Egocentric Video Dataset. - Wang Lin; Yueying Feng; WenKang Han; Tao Jin 004; Zhou Zhao 001; Fei Wu 001; Chang Yao; Jingyuan Chen (NeurIPS, 2024)

Extending Multi-modal Contrastive Representations. - Ziang Zhang; Zehan Wang 001; Luping Liu; Rongjie Huang 001; Xize Cheng; Zhenhui Ye; Wang Lin; Huadai Liu; Haifeng Huang; Yang Zhao 022; Tao Jin 004; Siqi Zheng; et al. (NeurIPS, 2024)

EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration. - Ye Wang 018; Jiahao Xun; Mingjie Hong; Jieming Zhu; Tao Jin 004; Wang Lin; Haoyuan Li 002; Linjun Li; Yan Xia 006; Zhou Zhao 001; Zhenhua Dong (CoRR, 2024)

OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup. - Xize Cheng; Siqi Zheng; Zehan Wang 001; Minghui Fang 002; Ziang Zhang; Rongjie Huang 001; Ziyang Ma; Shengpeng Ji; Jialong Zuo; Tao Jin 004; Zhou Zhao 001 (CoRR, 2024)

GTADT: Gated tone-sensitive acne grading via augmented domain transfer. - Min Tan 005; Ruirui Wang; Ankur Purwar; Tao Jin 004; Jun Yu 002; Alex C. Kot (Multim. Tools Appl., 2024)

Multi-Granularity Relational Attention Network for Audio-Visual Question Answering. - Linjun Li; Tao Jin 004; Wang Lin; Hao Jiang 014; Wenwen Pan 003; Jian Wang 119; Shuwen Xiao; Yan Xia 006; Weihao Jiang; Zhou Zhao 001 (IEEE Trans. Circuits Syst. Video Technol., 2024)

Bridging the Gap for Test-Time Multimodal Sentiment Analysis. - Zirun Guo; Tao Jin 004; Wenlong Xu; Wang Lin; Yangyang Wu (AAAI, 2025)

A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter. - Zirun Guo; Xize Cheng; Yangyang Wu; Tao Jin 004 (AAAI, 2025)

Speech Watermarking with Discrete Intermediate Representations. - Shengpeng Ji; Ziyue Jiang 001; Jialong Zuo; Minghui Fang 002; Yifu Chen; Tao Jin 004; Zhou Zhao 001 (AAAI, 2025)

T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback. - Zehan Wang 001; Ke Lei; Chen Zhu; Jiawei Huang 008; Sashuai Zhou; Luping Liu; Xize Cheng; Shengpeng Ji; Zhenhui Ye; Tao Jin 004; Zhou Zhao 001 (ACL (1), 2025)

TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis. - Yu Zhang 126; Wenxiang Guo; Changhao Pan; Dongyu Yao; Zhiyuan Zhu; Ziyue Jiang 001; Yuhan Wang; Tao Jin 004; Zhou Zhao 001 (ACL (Findings), 2025)

SpatialCLIP: Learning 3D-aware Image Representations from Spatially Discriminative Language. - Zehan Wang 001; Sashuai Zhou; Shaoxuan He; Haifeng Huang; Lihe Yang; Ziang Zhang; Xize Cheng; Shengpeng Ji; Tao Jin 004; Hengshuang Zhao; Zhou Zhao 001 (CVPR, 2025)

ConceptGuard: Continual Personalized Text-to-Image Generation with Forgetting and Confusion Mitigation. - Zirun Guo; Tao Jin 004 (CVPR, 2025)

Non-Natural Image Understanding with Advancing Frequency-based Vision Encoders. - Wang Lin; Qingsong Wang; Yueying Feng; Shulei Wang; Tao Jin 004; Zhou Zhao 001; Fei Wu 001; Chang Yao; Jingyuan Chen (CVPR, 2025)

Towards Transformer-Based Aligned Generation with Self-Coherence Guidance. - Shulei Wang; Wang Lin; Hai Huang 013; Hanting Wang; Sihang Cai; WenKang Han; Tao Jin 004; Jingyuan Chen; Jiacheng Sun; Jieming Zhu; Zhou Zhao 001 (CVPR, 2025)

Curriculum Learning aided Audio-Visual Speech Recognition with Arbitrary Speaker Number. - Yuxiao Lin; Tao Jin 004; Xize Cheng; Zhou Zhao 001; Fei Wu 001 (ICASSP, 2025)

OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces. - Zehan Wang 001; Ziang Zhang; Minjie Hong; Hang Zhang; Luping Liu; Rongjie Huang 001; Xize Cheng; Shengpeng Ji; Tao Jin 004; Hengshuang Zhao; Zhou Zhao 001 (ICLR, 2025)

VoxDialogue: Can Spoken Dialogue Systems Understand Information Beyond Words? - Xize Cheng; Ruofan Hu; Xiaoda Yang; Jingyu Lu; Dongjie Fu; Zehan Wang 001; Shengpeng Ji; Rongjie Huang 001; Boyang Zhang; Tao Jin 004; Zhou Zhao 001 (ICLR, 2025)

OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup. - Xize Cheng; Siqi Zheng; Zehan Wang 001; Minghui Fang 002; Ziang Zhang; Rongjie Huang 001; Shengpeng Ji; Jialong Zuo; Tao Jin 004; Zhou Zhao 001 (ICLR, 2025)

Smoothing the Shift: Towards Stable Test-Time Adaptation under Complex Multimodal Noises. - Zirun Guo; Tao Jin 004 (ICLR, 2025)

Diff-Prompt: Diffusion-Driven Prompt Generator with Mask Supervision. - Weicai Yan; Wang Lin; Zirun Guo; Ye Wang 018; Fangming Feng; Xiaoda Yang; Zehan Wang 001; Tao Jin 004 (ICLR, 2025)

Efficient Prompting for Continual Adaptation to Missing Modalities. - Zirun Guo; Shulei Wang; Wang Lin; Weicai Yan; Yangyang Wu; Tao Jin 004 (NAACL (Long Papers), 2025)

Data-Efficiently Learn Large Language Model for Universal 3D Scene Perception. - Zehan Wang 001; Haifeng Huang; Yang Zhao 022; Ziang Zhang; Tao Jin 004; Zhou Zhao 001 (NAACL (Findings), 2025)

Omni-Chart-600K: A Comprehensive Dataset of Chart Types for Chart Understanding. - Shulei Wang; Shuai Yang; Wang Lin; Zirun Guo; Sihang Cai; Hai Huang 013; Ye Wang 018; Jingyuan Chen; Tao Jin 004 (NAACL (Findings), 2025)

Recognize-and-tell: Generating video captions with textual cue in scene. - Tao Jin 004; Wang Lin; Hao Jiang 014; Jian Wang 119; Xiao Jin; Zhimeng Zhang; Jingyuan Chen; Zhou Zhao 001; Zhongfei Zhang (Expert Syst. Appl., 2025)

OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios. - Xize Cheng; Dongjie Fu; Xiaoda Yang; Minghui Fang 002; Ruofan Hu; Jingyu Lu; Jionghao Bai; Zehan Wang 001; Shengpeng Ji; Rongjie Huang 001; Linjun Li; Yu Chen; et al. (CoRR, 2025)

Unleashing the Power of Natural Audio Featuring Multiple Sound Sources. - Xize Cheng; Slytherin Wang; Zehan Wang 001; Rongjie Huang 001; Tao Jin 004; Zhou Zhao 001 (CoRR, 2025)

ISDrama: Immersive Spatial Drama Generation through Multimodal Prompting. - Yu Zhang 126; Wenxiang Guo; Changhao Pan; Zhiyuan Zhu; Tao Jin 004; Zhou Zhao 001 (CoRR, 2025)

Observe-R1: Unlocking Reasoning Abilities of MLLMs with Dynamic Progressive Reinforcement Learning. - Zirun Guo; Minjie Hong; Tao Jin 004 (CoRR, 2025)

IRBridge: Solving Image Restoration Bridge with Pre-trained Generative Diffusion Models. - Hanting Wang; Tao Jin 004; Wang Lin; Shulei Wang; Hai Huang 013; Shengpeng Ji; Zhou Zhao 001 (CoRR, 2025)

Vela: Scalable Embeddings with Voice Large Language Models for Multimodal Retrieval. - Ruofan Hu; Yan Xia 006; Minjie Hong; Jieming Zhu; Bo Chen 023; Xiaoda Yang; Minghui Fang 002; Tao Jin 004 (CoRR, 2025)

APO: Enhancing Reasoning Ability of MLLMs via Asymmetric Policy Optimization. - Minjie Hong; Zirun Guo; Yan Xia 006; Zehan Wang 001; Ziang Zhang; Tao Jin 004; Zhou Zhao 001 (CoRR, 2025)

Open-set Cross Modal Generalization via Multimodal Unified Representation. - Hai Huang 013; Yan Xia 006; Shulei Wang; Hanting Wang; Minghui Fang 002; Shengpeng Ji; Sashuai Zhou; Tao Jin 004; Zhou Zhao 001 (CoRR, 2025)

TAP: Parameter-efficient Task-Aware Prompting for Adverse Weather Removal. - Hanting Wang; Shengpeng Ji; Shulei Wang; Hai Huang 013; Xiao Jin; Qifei Zhang; Tao Jin 004 (CoRR, 2025)