Rethinking the Value of Transformer Components. - Wenxuan Wang 001; Zhaopeng Tu (COLING, 2020)

Language Models are Good Translators. - Shuo Wang 013; Zhaopeng Tu; Zhixing Tan; Wenxuan Wang 001; Maosong Sun 001; Yang Liu 005 (CoRR, 2021)

APIBench: A Benchmark Dataset for Evaluating API Recommendation Approaches in Python and Java. - Yun Peng 003; Shuqing Li 001; Wenwei Gu; Yichen Li 003; Wenxuan Wang 001; Cuiyun Gao 001; Michael R. Lyu (Zenodo, 2021)

Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation. - Wenxuan Wang 001; Wenxiang Jiao; Yongchang Hao; Xing Wang 007; Shuming Shi 001; Zhaopeng Tu; Michael R. Lyu (ACL (1), 2022)

Improving Adversarial Transferability via Neuron Attribution-based Attacks. - Jianping Zhang 002; Weibin Wu 002; Jen-tse Huang 001; Yizhan Huang; Wenxuan Wang 001; Yuxin Su 001; Michael R. Lyu (CVPR, 2022)

AEON: a method for automatic evaluation of NLP test cases. - Jen-tse Huang 001; Jianping Zhang 002; Wenxuan Wang 001; Pinjia He; Yuxin Su 001; Michael R. Lyu (ISSTA, 2022)

Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languages. - Wenxiang Jiao; Zhaopeng Tu; Jiarui Li; Wenxuan Wang 001; Jen-tse Huang 001; Shuming Shi 001 (WMT, 2022)

Improving the Transferability of Adversarial Samples by Path-Augmented Method. - Jianping Zhang 002; Jen-tse Huang 001; Wenxuan Wang 001; Yichen Li 003; Weibin Wu 002; Xiaosen Wang; Yuxin Su 001; Michael R. Lyu (CVPR, 2023)

ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback. - Wenxiang Jiao; Jen-tse Huang 001; Wenxuan Wang 001; Zhiwei He 002; Tian Liang; Xing Wang 007; Shuming Shi 001; Zhaopeng Tu (EMNLP (Findings), 2023)

MTTM: Metamorphic Testing for Textual Content Moderation Software. - Wenxuan Wang 001; Jen-tse Huang 001; Weibin Wu 002; Jianping Zhang 002; Yizhan Huang; Shuqing Li 001; Pinjia He; Michael R. Lyu (ICSE, 2023)

Validating Multimedia Content Moderation Software via Semantic Fusion. - Wenxuan Wang 001; Jingyuan Huang; Chang Chen; Jiazhen Gu; Jianping Zhang 002; Weibin Wu 002; Pinjia He; Michael R. Lyu (ISSTA, 2023)

What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs? - Shuzheng Gao; Xin-Cheng Wen; Cuiyun Gao 001; Wenxuan Wang 001; Hongyu Zhang 002; Michael R. Lyu (ASE, 2023)

Generative Type Inference for Python. - Yun Peng 003; Chaozheng Wang; Wenxuan Wang 001; Cuiyun Gao 001; Michael R. Lyu (ASE, 2023)

An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software. - Wenxuan Wang 001; Jingyuan Huang; Jen-tse Huang 001; Chang Chen; Jiazhen Gu; Pinjia He; Michael R. Lyu (ASE, 2023)

BiasAsker: Measuring the Bias in Conversational AI System. - Yuxuan Wan; Wenxuan Wang 001; Pinjia He; Jiazhen Gu; Haonan Bai; Michael R. Lyu (ESEC/SIGSOFT FSE, 2023)

Is ChatGPT A Good Translator? A Preliminary Study. - Wenxiang Jiao; Wenxuan Wang 001; Jen-tse Huang 001; Xing Wang 007; Zhaopeng Tu (CoRR, 2023)

ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark. - Haoran Wu; Wenxuan Wang 001; Yuxuan Wan; Wenxiang Jiao; Michael R. Lyu (CoRR, 2023)

ParroT: Translating During Chat Using Large Language Models. - Wenxiang Jiao; Jen-tse Huang 001; Wenxuan Wang 001; Xing Wang 007; Shuming Shi 001; Zhaopeng Tu (CoRR, 2023)

Constructing Effective In-Context Demonstration for Code Intelligence Tasks: An Empirical Study. - Shuzheng Gao; Xin-Cheng Wen; Cuiyun Gao 001; Wenxuan Wang 001; Michael R. Lyu (CoRR, 2023)

ChatGPT an ENFJ, Bard an ISTJ: Empirical Study on Personalities of Large Language Models. - Jen-tse Huang 001; Wenxuan Wang 001; Man Ho Lam; Eric John Li; Wenxiang Jiao; Michael R. Lyu (CoRR, 2023)

Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench. - Jen-tse Huang 001; Man Ho Lam; Eric John Li; Shujie Ren; Wenxuan Wang 001; Wenxiang Jiao; Zhaopeng Tu; Michael R. Lyu (CoRR, 2023)

All Languages Matter: On the Multilingual Safety of Large Language Models. - Wenxuan Wang 001; Zhaopeng Tu; Chang Chen; Youliang Yuan; Jen-tse Huang 001; Wenxiang Jiao; Michael R. Lyu (CoRR, 2023)

Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench. - Jen-tse Huang 001; Wenxuan Wang 001; Eric John Li; Man Ho Lam; Shujie Ren; Youliang Yuan; Wenxiang Jiao; Zhaopeng Tu; Michael R. Lyu (CoRR, 2023)

Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models. - Tian Liang; Zhiwei He 002; Jen-tse Huang 001; Wenxuan Wang 001; Wenxiang Jiao; Rui Wang 015; Yujiu Yang; Zhaopeng Tu; Shuming Shi 001; Xing Wang 007 (CoRR, 2023)

Revisiting, Benchmarking and Exploring API Recommendation: How Far Are We? - Yun Peng 003; Shuqing Li 001; Wenwei Gu; Yichen Li 003; Wenxuan Wang 001; Cuiyun Gao 001; Michael R. Lyu (IEEE Trans. Software Eng., 2023)

APIBench: A Benchmark Dataset for Evaluating API Recommendation Approaches in Python and Java. - Yun Peng 003; Shuqing Li 001; Wenwei Gu; Yichen Li 003; Wenxuan Wang 001; Cuiyun Gao 001; Michael R. Lyu (Zenodo, 2023)

Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models. - Wenxuan Wang 001; Wenxiang Jiao; Jingyuan Huang; Ruyi Dai; Jen-tse Huang 001; Zhaopeng Tu; Michael R. Lyu (ACL (1), 2024)

All Languages Matter: On the Multilingual Safety of LLMs. - Wenxuan Wang 001; Zhaopeng Tu; Chang Chen; Youliang Yuan; Jen-tse Huang 001; Wenxiang Jiao; Michael R. Lyu (ACL (Findings), 2024)

Does ChatGPT Know That It Does Not Know? Evaluating the Black-Box Calibration of ChatGPT. - Youliang Yuan; Wenxuan Wang 001; Qingshuo Guo; Yiming Xiong; Chihao Shen; Pinjia He (LREC/COLING, 2024)

On the Reliability of Psychological Scales on Large Language Models. - Jen-tse Huang 001; Wenxiang Jiao; Man Ho Lam; Eric John Li; Wenxuan Wang 001; Michael R. Lyu (EMNLP, 2024)

LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models. - Yuxuan Wan; Wenxuan Wang 001; Yiliu Yang; Youliang Yuan; Jen-tse Huang 001; Pinjia He; Wenxiang Jiao; Michael R. Lyu (EMNLP, 2024)

On the Humanity of Conversational AI: Evaluating the Psychological Portrayal of LLMs. - Jen-tse Huang 001; Wenxuan Wang 001; Eric John Li; Man Ho Lam; Shujie Ren; Youliang Yuan; Wenxiang Jiao; Zhaopeng Tu; Michael R. Lyu (ICLR, 2024)

GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher. - Youliang Yuan; Wenxiang Jiao; Wenxuan Wang 001; Jen-tse Huang 001; Pinjia He; Shuming Shi 001; Zhaopeng Tu (ICLR, 2024)

A Systematic Evaluation of Large Code Models in API Suggestion: When, Which, and How. - Chaozheng Wang; Shuzheng Gao; Cuiyun Gao 001; Wenxuan Wang 001; Chun Yong Chong; Shan Gao; Michael R. Lyu (ASE, 2024)

New Job, New Gender? Measuring the Social Bias in Image Generation Models. - Wenxuan Wang 001; Haonan Bai; Jen-tse Huang 001; Yuxuan Wan; Youliang Yuan; Haoyi Qiu; Nanyun Peng 001; Michael R. Lyu (ACM Multimedia, 2024)

Apathetic or Empathetic? Evaluating LLMs' Emotional Alignments with Humans. - Jen-tse Huang 001; Man Ho Lam; Eric John Li; Shujie Ren; Wenxuan Wang 001; Wenxiang Jiao; Zhaopeng Tu; Michael R. Lyu (NeurIPS, 2024)

A & B == B & A: Triggering Logical Reasoning Failures in Large Language Models. - Yuxuan Wan; Wenxuan Wang 001; Yiliu Yang; Youliang Yuan; Jen-tse Huang 001; Pinjia He; Wenxiang Jiao; Michael R. Lyu (CoRR, 2024)

The Earth is Flat? Unveiling Factual Errors in Large Language Models. - Wenxuan Wang 001; Juluan Shi; Zhaopeng Tu; Youliang Yuan; Jen-tse Huang 001; Wenxiang Jiao; Michael R. Lyu (CoRR, 2024)

Asclepius: A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models. - Wenxuan Wang 001; Yihang Su; Jingyuan Huan; Jie Liu 044; Wenting Chen; Yudi Zhang 005; Cheng-Yi Li; Kao-Jung Chang; Xiaohan Xin; Linlin Shen; Michael R. Lyu (CoRR, 2024)

How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments. - Jen-tse Huang 001; Eric John Li; Man Ho Lam; Tian Liang; Wenxuan Wang 001; Youliang Yuan; Wenxiang Jiao; Xing Wang 007; Zhaopeng Tu; Michael R. Lyu (CoRR, 2024)

How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHO. - Man Tik Ng; Hui Tung Tse; Jen-tse Huang 001; Jingjing Li 007; Wenxuan Wang 001; Michael R. Lyu (CoRR, 2024)

Exploring Multi-Lingual Bias of Large Code Models in Code Generation. - Chaozheng Wang; Zongjie Li; Cuiyun Gao 001; Wenxuan Wang 001; Ting Peng; Hailiang Huang; Yuetang Deng; Shuai Wang 011; Michael R. Lyu (CoRR, 2024)

Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approach. - Yuxuan Wan; Chaozheng Wang; Yi Dong; Wenxuan Wang 001; Shuqing Li 001; Yintong Huo; Michael R. Lyu (CoRR, 2024)

On the Resilience of Multi-Agent Systems with Malicious Agents. - Jen-tse Huang 001; Jiaxu Zhou; Tailin Jin; Xuhui Zhou; Zixi Chen; Wenxuan Wang 001; Youliang Yuan; Maarten Sap; Michael R. Lyu (CoRR, 2024)

Learning to Ask: When LLMs Meet Unclear Instruction. - Wenxuan Wang 001; Juluan Shi; Chaozheng Wang; Cheryl Lee; Youliang Yuan; Jen-tse Huang 001; Michael R. Lyu (CoRR, 2024)

Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step. - Wenxuan Wang 001; Kuiyi Gao; Zihan Jia; Youliang Yuan; Jen-tse Huang 001; Qiuzhi Liu; Shuai Wang 011; Wenxiang Jiao; Zhaopeng Tu (CoRR, 2024)

Medchain: Bridging the Gap Between LLM Agents and Clinical Practice through Interactive Sequential Benchmarking. - Jie Liu 044; Wenxuan Wang 001; Zizhan Ma; Guolin Huang; Yihang Su; Kao-Jung Chang; Wenting Chen; Haoliang Li; Linlin Shen; Michael R. Lyu (CoRR, 2024)

MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs. - Yuxuan Wan; Yi Dong; Jingyu Xiao; Yintong Huo; Wenxuan Wang 001; Michael R. Lyu (CoRR, 2024)

Understanding and Mitigating the Uncertainty in Zero-Shot Translation. - Wenxuan Wang 001; Wenxiang Jiao; Shuo Wang 013; Zhaopeng Tu; Michael R. Lyu (IEEE ACM Trans. Audio Speech Lang. Process., 2024)

Chain-of-Jailbreak Attack for Image Generation Models via Step by Step Editing. - Wenxuan Wang 001; Kuiyi Gao; Youliang Yuan; Jen-tse Huang 001; Qiuzhi Liu; Shuai Wang 001; Wenxiang Jiao; Zhaopeng Tu (ACL (Findings), 2025)

Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs. - Wenxuan Wang 001; Xiaoyuan Liu; Kuiyi Gao; Jen-tse Huang 001; Youliang Yuan; Pinjia He; Shuai Wang 011; Zhaopeng Tu (ACL (1), 2025)

EAGLE: Expert-Guided Self-Enhancement for Preference Alignment in Pathology Large Vision-Language Model. - Meidan Ding; Jipeng Zhang; Wenxuan Wang 001; Haiqin Zhong; Xiaoqin Wang; Xinheng Lyu; Wenting Chen; Linlin Shen (ACL (1), 2025)

Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMs. - Xiaoyuan Liu; Wenxuan Wang 001; Youliang Yuan; Jen-tse Huang 001; Qiuzhi Liu; Pinjia He; Zhaopeng Tu (ACL (1), 2025)

Asclepius: A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models. - Jie Liu 044; Wenxuan Wang 001; Yihang Su; Jingyuan Huang; Yudi Zhang 005; Cheng-Yi Li; Wenting Chen; Xiaohan Xing; Kao-Jung Chang; Linlin Shen; Michael R. Lyu (ACL (1), 2025)

A Survey of LLM-based Agents in Medicine: How far are we from Baymax? - Wenxuan Wang 001; Zizhan Ma; Zheng Wang; Chenghan Wu; Jiaming Ji; Wenting Chen; Xiang Li 001; Yixuan Yuan (ACL (Findings), 2025)

Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training. - Youliang Yuan; Wenxiang Jiao; Wenxuan Wang 001; Jen-tse Huang 001; Jiahao Xu; Tian Liang; Pinjia He; Zhaopeng Tu (ACL (1), 2025)

Competing Large Language Models in Multi-Agent Gaming Environments. - Jen-tse Huang 001; Eric John Li; Man Ho Lam; Tian Liang; Wenxuan Wang 001; Youliang Yuan; Wenxiang Jiao; Xing Wang 007; Zhaopeng Tu; Michael R. Lyu (ICLR, 2025)

On the shortcut learning in multilingual neural machine translation. - Wenxuan Wang 001; Wenxiang Jiao; Jen-tse Huang 001; Zhaopeng Tu; Michael R. Lyu (Neurocomputing, 2025)

Divide-and-Conquer: Generating UI Code from Screenshots. - Yuxuan Wan; Chaozheng Wang; Yi Dong; Wenxuan Wang 001; Shuqing Li 001; Yintong Huo; Michael R. Lyu (Proc. ACM Softw. Eng., 2025)

How Should I Build A Benchmark? Revisiting Code-Related Benchmarks For LLMs. - Jialun Cao; Yuk-Kit Chan; Zixuan Ling; Wenxuan Wang 001; Shuqing Li 001; Mingwei Liu 002; Ruixi Qiao; Yuting Han; Chaozheng Wang; Boxi Yu; Pinjia He; Shuai Wang 011; et al. (CoRR, 2025)

Fact-or-Fair: A Checklist for Behavioral Testing of AI Models on Fairness-Related Queries. - Jen-tse Huang 001; Yuhang Yan 002; Linqi Liu; Yixin Wan; Wenxuan Wang 001; Kai-Wei Chang; Michael R. Lyu (CoRR, 2025)

VLMs as GeoGuessr Masters: Exceptional Performance, Hidden Biases, and Privacy Risks. - Jingyuan Huang; Jen-tse Huang 001; Ziyi Liu; Xiaoyuan Liu; Wenxuan Wang 001; Jieyu Zhao 001 (CoRR, 2025)

A Survey of LLM-based Agents in Medicine: How far are we from Baymax? - Wenxuan Wang 001; Zizhan Ma; Zheng Wang; Chenghan Wu; Wenting Chen; Xiang Li 001; Yixuan Yuan (CoRR, 2025)

VisFactor: Benchmarking Fundamental Visual Cognition in Multimodal Large Language Models. - Jen-tse Huang 001; Dasen Dai; Jen-Yuan Huang; Youliang Yuan; Xiaoyuan Liu; Wenxuan Wang 001; Wenxiang Jiao; Pinjia He; Zhaopeng Tu (CoRR, 2025)

VisBias: Measuring Explicit and Implicit Social Biases in Vision Language Models. - Jen-tse Huang 001; Jiantong Qin; Jianping Zhang 002; Youliang Yuan; Wenxuan Wang 001; Jieyu Zhao 001 (CoRR, 2025)

STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Models. - Xunguang Wang; Wenxuan Wang 001; Zhenlan Ji; Zongjie Li; Pingchuan Ma 004; Daoyuan Wu; Shuai Wang 011 (CoRR, 2025)

DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning. - Zhiwei He 002; Tian Liang; Jiahao Xu; Qiuzhi Liu; Xingyu Chen; Yue Wang; Linfeng Song; Dian Yu 001; Zhenwen Liang; Wenxuan Wang 001; Zhuosheng Zhang 001; Rui Wang 015; et al. (CoRR, 2025)

A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron? - Ada Chen; Yongjiang Wu; Junyuan Zhang; Jingyu Xiao; Shu Yang 010; Jen-tse Huang 001; Kun Wang 056; Wenxuan Wang 001; Shuai Wang 011 (CoRR, 2025)

IP Leakage Attacks Targeting LLM-Based Multi-Agent Systems. - Liwen Wang; Wenxuan Wang 001; Shuai Wang 011; Zongjie Li; Zhenlan Ji; Zongyi Lyu; Daoyuan Wu; Shing-Chi Cheung (CoRR, 2025)

Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards. - Xiaoyuan Liu; Tian Liang; Zhiwei He 002; Jiahao Xu; Wenxuan Wang 001; Pinjia He; Zhaopeng Tu; Haitao Mi; Dong Yu 001 (CoRR, 2025)

Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training. - Mengru Wang; Xingyu Chen; Yue Wang; Zhiwei He 002; Jiahao Xu; Tian Liang; Qiuzhi Liu; Yunzhi Yao; Wenxuan Wang 001; Ruotian Ma; Haitao Mi; Ningyu Zhang 001; et al. (CoRR, 2025)

Towards Evaluating Proactive Risk Awareness of Multimodal Language Models. - Youliang Yuan; Wenxiang Jiao; Yuejin Xie; Chihao Shen; Menghan Tian; Wenxuan Wang 001; Jen-tse Huang 001; Pinjia He (CoRR, 2025)

SoK: Evaluating Jailbreak Guardrails for Large Language Models. - Xunguang Wang; Zhenlan Ji; Wenxuan Wang 001; Zongjie Li; Daoyuan Wu; Shuai Wang 011 (CoRR, 2025)

3D Software Synthesis Guided by Constraint-Expressive Intermediate Representation. - Shuqing Li 001; Anson Y. Lam; Yun Peng 003; Wenxuan Wang 001; Michael R. Lyu (CoRR, 2025)

Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications. - Wenxuan Wang 001; Zizhan Ma; Meidan Ding; Shiyi Zheng; Shengyuan Liu; Jie Liu 044; Jiaming Ji; Wenting Chen; Xiang Li 001; Linlin Shen; Yixuan Yuan (CoRR, 2025)

Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models. - Zizhan Ma; Wenxuan Wang 001; Guo Yu; Yiu-Fai Cheung; Meidan Ding; Jie Liu 044; Wenting Chen; Linlin Shen (CoRR, 2025)