An Empirical Study on the Scalability of Selective Mutation Testing. - Jie Zhang 050; Muyao Zhu; Dan Hao 001; Lu Zhang 023 (ISSRE, 2014)

Search-based inference of polynomial metamorphic relations. - Jie Zhang 050; Junjie Chen 003; Dan Hao 001; Yingfei Xiong 001; Bing Xie; Lu Zhang 023; Hong Mei 001 (ASE, 2014)

Scalability Studies on Selective Mutation Testing. - Jie Zhang 050 (ICSE (2), 2015)

A survey on bug-report analysis. - Jie Zhang 050; Xiaoyin Wang; Dan Hao 001; Bing Xie; Lu Zhang 023; Hong Mei 001 (Sci. China Inf. Sci., 2015)

Predictive mutation testing. - Jie Zhang 050; Ziyi Wang; Lingming Zhang 001; Dan Hao 001; Lei Zang; Shiyang Cheng 002; Lu Zhang 023 (ISSTA, 2016)

Isomorphic regression testing: executing uncovered branches without test augmentation. - Jie Zhang 050; Yiling Lou; Lingming Zhang 001; Dan Hao 001; Lu Zhang 023; Hong Mei 001 (SIGSOFT FSE, 2016)

To Detect Abnormal Program Behaviours via Mutation Deduction. - Jie Zhang 050; Dan Hao 001; Lingming Zhang 001; Lu Zhang 023 (ICST Workshops, 2018)

Automated refactoring of nested-IF formulae in spreadsheets. - Jie Zhang 050; Shi Han; Dan Hao 001; Lu Zhang 023; Dongmei Zhang 001 (ESEC/SIGSOFT FSE, 2018)

How Does Bug-Handling Effort Differ Among Different Programming Languages? - Jie Zhang 050; Feng Li 037; Dan Hao 001; Meng Wang 002; Lu Zhang 023 (CoRR, 2018)

Do Pseudo Test Suites Lead to Inflated Correlation in Measuring Test Effectiveness? - Jie M. Zhang; Lingming Zhang 001; Dan Hao 001; Meng Wang 002; Lu Zhang 023 (ICST, 2019)

An Empirical Comparison of Mutant Selection Assessment Metrics. - Jie M. Zhang; Lingming Zhang 001; Dan Hao 001; Lu Zhang 023; Mark Harman (ICST Workshops, 2019)

Chapter Six - Mutation Testing Advances: An Analysis and Survey. - Mike Papadakis; Marinos Kintis; Jie Zhang 050; Yue Jia 001; Yves Le Traon; Mark Harman (Adv. Comput., 2019)

Perturbed Model Validation: A New Framework to Validate Model Relevance. - Jie M. Zhang; Earl T. Barr; Benjamin Guedj; Mark Harman; John Shawe-Taylor (CoRR, 2019)

Predictive Mutation Testing. - Jie Zhang 050; Lingming Zhang 001; Mark Harman; Dan Hao 001; Yue Jia 001; Lu Zhang 023 (IEEE Trans. Software Eng., 2019)

Automatic testing and improvement of machine translation. - Zeyu Sun 004; Jie M. Zhang; Mark Harman; Mike Papadakis; Lu Zhang 023 (ICSE, 2020)

Automatic Improvement of Machine Translation Using Mutamorphic Relation: Invited Talk Paper. - Jie M. Zhang (ICSE (Workshops), 2020)

FrUITeR: a framework for evaluating UI test reuse. - Yixue Zhao; Justin Chen; Adriana Sejfia; Marcelo Schmitt Laser; Jie Zhang 050; Federica Sarro; Mark Harman; Nenad Medvidovic (ESEC/SIGSOFT FSE, 2020)

"Ignorance and Prejudice" in Software Fairness. - Jie M. Zhang; Mark Harman (ICSE, 2021)

Fairea: a model behaviour mutation approach to benchmarking bias mitigation methods. - Max Hort; Jie M. Zhang; Federica Sarro; Mark Harman (ESEC/SIGSOFT FSE, 2021)

A Study of Bug Resolution Characteristics in Popular Programming Languages. - Jie M. Zhang; Feng Li 037; Dan Hao 001; Meng Wang 002; Hao Tang; Lu Zhang 023; Mark Harman (IEEE Trans. Software Eng., 2021)

Leveraging Automated Unit Tests for Unsupervised Code Translation. - Baptiste Rozière; Jie Zhang 050; François Charton; Mark Harman; Gabriel Synnaeve; Guillaume Lample (ICLR, 2022)

Improving Machine Translation Systems via Isotopic Replacement. - Zeyu Sun 004; Jie M. Zhang; Yingfei Xiong 001; Mark Harman; Mike Papadakis; Lu Zhang 023 (ICSE, 2022)

Natural Test Generation for Precise Testing of Question Answering Software. - Qingchao Shen; Junjie Chen 003; Jie M. Zhang; Haoyu Wang 017; Shuang Liu 007; Menghan Tian (ASE, 2022)

GraphCode2Vec: Generic Code Embedding via Lexical and Program Dependence Analyses. - Wei Ma; Mengjie Zhao; Ezekiel O. Soremekun; Qiang Hu; Jie M. Zhang; Mike Papadakis; Maxime Cordy; Xiaofei Xie; Yves Le Traon (MSR, 2022)

MAAT: a novel ensemble approach to addressing fairness and performance bugs for machine learning software. - Zhenpeng Chen; Jie M. Zhang; Federica Sarro; Mark Harman (ESEC/SIGSOFT FSE, 2022)

A Comprehensive Empirical Study of Bias Mitigation Methods for Software Fairness. - Zhenpeng Chen; Jie M. Zhang; Federica Sarro; Mark Harman (CoRR, 2022)

Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey. - Max Hort; Zhenpeng Chen; Jie M. Zhang; Federica Sarro; Mark Harman (CoRR, 2022)

Is Self-Attention Powerful to Learn Code Syntax and Semantics? - Wei Ma; Mengjie Zhao; Xiaofei Xie; Qiang Hu; Shangqing Liu; Jie Zhang 050; Wenhan Wang; Yang Liu 003 (CoRR, 2022)

Mutation analysis and its industrial applications. - Rahul Gopinath; Jie M. Zhang; Marinos Kintis; Mike Papadakis (Softw. Test. Verification Reliab., 2022)

Machine Learning Testing: Survey, Landscapes and Horizons. - Jie M. Zhang; Mark Harman; Lei Ma 003; Yang Liu 003 (IEEE Trans. Software Eng., 2022)

On Testing Ethical Autonomous Decision-Making. - Michael E. Akintunde; Martim Brandão; Gunel Jahangirova; Hector Menendez; Mohammad Reza Mousavi 001; Jie Zhang 050 (Applicable Formal Methods for Safe Industrial Products, 2023)

A Black-Box Attack on Code Models via Representation Nearest Neighbor Search. - Jie Zhang 050; Wei Ma; Qiang Hu; Shangqing Liu; Xiaofei Xie; Yves Le Traon; Yang Liu 003 (EMNLP (Findings), 2023)

Large Language Models for Software Engineering: Survey and Open Problems. - Angela Fan; Beliz Gokkaya; Mark Harman; Mitya Lyubarskiy; Shubho Sengupta; Shin Yoo; Jie M. Zhang (ICSE-FoSE, 2023)

Vulnerability Detection with Graph Simplification and Enhanced Graph Representation Learning. - Xin-Cheng Wen; Yupan Chen; Cuiyun Gao 001; Hongyu Zhang 002; Jie M. Zhang; Qing Liao 001 (ICSE, 2023)

Who Judges the Judge: An Empirical Study on Online Judge Tests. - Kaibo Liu; Yudong Han; Jie M. Zhang; Zhenpeng Chen; Federica Sarro; Mark Harman; Gang Huang 001; Yun Ma 002 (ISSTA, 2023)

RNNS: Representation Nearest Neighbor Search Black-Box Attack on Code Models. - Jie Zhang 050; Wei Ma; Qiang Hu; Xiaofei Xie; Yves Le Traon; Yang Liu 003 (CoRR, 2023)

An Empirical Study on Fairness Improvement with Multiple Protected Attributes. - Zhenpeng Chen; Jie M. Zhang; Federica Sarro; Mark Harman (CoRR, 2023)

LLM is Like a Box of Chocolates: the Non-determinism of ChatGPT in Code Generation. - Shuyin Ouyang; Jie M. Zhang; Mark Harman; Meng Wang 002 (CoRR, 2023)

Dark-Skin Individuals Are at More Risk on the Street: Unmasking Fairness Issues of Autonomous Driving Systems. - Xinyue Li; Zhenpeng Chen; Jie M. Zhang; Federica Sarro; Ying Zhang 012; Xuanzhe Liu (CoRR, 2023)

COCO: Testing Code Generation Systems via Concretized Instructions. - Ming Yan 010; Junjie Chen 003; Jie M. Zhang; Xuejie Cao; Chen Yang; Mark Harman (CoRR, 2023)

Large Language Models in Fault Localisation. - Yonghao Wu; Zheng Li 002; Jie M. Zhang; Mike Papadakis; Mark Harman; Yong Liu 030 (CoRR, 2023)

Bias Assessment and Mitigation in LLM-based Code Generation. - Dong Huang 005; Qingwen Bu; Jie Zhang 050; Xiaofei Xie; Junjie Chen 003; Heming Cui (CoRR, 2023)

ConDefects: A New Dataset to Address the Data Leakage Concern for LLM-based Fault Localization and Program Repair. - Yonghao Wu; Zheng Li 002; Jie M. Zhang; Yong Liu 030 (CoRR, 2023)

AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation. - Dong Huang 005; Qingwen Bu; Jie M. Zhang; Michael Luck; Heming Cui (CoRR, 2023)

Model validation using mutated training labels: An exploratory study. - Jie M. Zhang; Mark Harman; Benjamin Guedj; Earl T. Barr; John Shawe-Taylor (Neurocomputing, 2023)

A Comprehensive Empirical Study of Bias Mitigation Methods for Machine Learning Classifiers. - Zhenpeng Chen; Jie M. Zhang; Federica Sarro; Mark Harman (ACM Trans. Softw. Eng. Methodol., 2023)

Using Protected Attributes to Consider Fairness in Multi-Agent Systems. - Gabriele La Malfa; Jie M. Zhang; Michael Luck; Elizabeth Black (AEQUITAS@ECAI, 2024)

Real-Fake: Effective Training Data Synthesis Through Distribution Matching. - Jianhao Yuan; Jie Zhang 050; Shuyang Sun; Philip Torr 001; Bo Zhao (ICLR, 2024)

Fairness Improvement with Multiple Protected Attributes: How Far Are We? - Zhenpeng Chen; Jie M. Zhang; Federica Sarro; Mark Harman (ICSE, 2024)

Evaluating Fault Localization and Program Repair Capabilities of Existing Closed-Source General-Purpose LLMs. - Shengbei Jiang; Jiabao Zhang; Wei Chen; Bo Wang 050; Jianyi Zhou; Jie Zhang 050 (LLM4CODE@ICSE, 2024)

TrickyBugs: A Dataset of Corner-case Bugs in Plausible Programs. - Kaibo Liu; Yudong Han; Yiyang Liu; Jie M. Zhang; Zhenpeng Chen; Federica Sarro; Gang Huang 001; Yun Ma 002 (MSR, 2024)

EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization. - Dong Huang 005; Jianbo Dai; Han Weng; Puzhen Wu; Yuhao Qing; Heming Cui; Zhijiang Guo; Jie Zhang 050 (NeurIPS, 2024)

EffiBench: Benchmarking the Efficiency of Automatically Generated Code. - Dong Huang 005; Yuhao Qing; Weiyi Shang; Heming Cui; Jie Zhang 050 (NeurIPS, 2024)

ConDefects: A Complementary Dataset to Address the Data Leakage Concern for LLM-Based Fault Localization and Program Repair. - Yonghao Wu; Zheng Li 002; Jie M. Zhang; Yong Liu 030 (SIGSOFT FSE Companion, 2024)

EffiBench: Benchmarking the Efficiency of Automatically Generated Code. - Dong Huang 005; Jie M. Zhang; Yuhao Qing; Heming Cui (CoRR, 2024)

LLM-Powered Test Case Generation for Detecting Tricky Bugs. - Kaibo Liu; Yiyang Liu; Zhenpeng Chen; Jie M. Zhang; Yudong Han; Yun Ma 002; Ge Li 001; Gang Huang 001 (CoRR, 2024)

SOAP: Enhancing Efficiency of Generated Code via Self-Optimization. - Dong Huang 005; Jianbo Dai; Han Weng; Puzhen Wu; Yuhao Qing; Jie M. Zhang; Heming Cui; Zhijiang Guo (CoRR, 2024)

An Exploratory Study on Using Large Language Models for Mutation Testing. - Bo Wang 050; Mingda Chen; Youfang Lin; Mike Papadakis; Jie M. Zhang (CoRR, 2024)

Rethinking the Influence of Source Code on Test Case Generation. - Dong Huang 005; Jie M. Zhang; Mingzhe Du; Mark Harman; Heming Cui (CoRR, 2024)

Effi-Code: Unleashing Code Efficiency in Language Models. - Dong Huang 005; Guangtao Zeng; Jianbo Dai; Meng Luo; Han Weng; Yuhao Qing; Heming Cui; Zhijiang Guo; Jie M. Zhang (CoRR, 2024)

Benchmarking Bias in Large Language Models during Role-Playing. - Xinyue Li; Zhenpeng Chen; Jie M. Zhang; Yiling Lou; Tianlin Li; Weisong Sun; Yang Liu 003; Xuanzhe Liu (CoRR, 2024)

Research artifacts in software engineering publications: Status and trends. - Mugeng Liu; Xiaolong Huang; Wei He; Yibing Xie; Jie M. Zhang; Xiang Jing; Zhenpeng Chen; Yun Ma 002 (J. Syst. Softw., 2024)

MirrorFair: Fixing Fairness Bugs in Machine Learning Software via Counterfactual Predictions. - Ying Xiao; Jie M. Zhang; Yepang Liu 001; Mohammad Reza Mousavi 001; Sicen Liu; Dingyuan Xue (Proc. ACM Softw. Eng., 2024)

Mutation analysis for evaluating code translation. - Giovani Guizzo; Jie M. Zhang; Federica Sarro; Christoph Treude; Mark Harman (Empir. Softw. Eng., 2024)

Search-based Automatic Repair for Fairness and Accuracy in Decision-making Software. - Max Hort; Jie M. Zhang; Federica Sarro; Mark Harman (Empir. Softw. Eng., 2024)

Stealthy Backdoor Attack for Code Models. - Zhou Yang 003; Bowen Xu; Jie M. Zhang; Hong Jin Kang; Jieke Shi; Junda He; David Lo 001 (IEEE Trans. Software Eng., 2024)

Fairness Testing: A Comprehensive Survey and Analysis of Trends. - Zhenpeng Chen; Jie M. Zhang; Max Hort; Mark Harman; Federica Sarro (ACM Trans. Softw. Eng. Methodol., 2024)

Fairness Testing of Machine Translation Systems. - Zeyu Sun 004; Zhenpeng Chen; Jie Zhang 050; Dan Hao 001 (ACM Trans. Softw. Eng. Methodol., 2024)

Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities. - Wei Ma; Shangqing Liu; Mengjie Zhao; Xiaofei Xie; Wenhan Wang; Qiang Hu; Jie Zhang 050; Yang Liu 003 (ACM Trans. Softw. Eng. Methodol., 2024)

SEA4DQ 2024 Workshop Summary. - Tim Menzies; Bowen Xu; Hong Jin Kang; Jie M. Zhang; Jiri Gesi; Sagar Sen; Beatriz Cassoli; Nicolas Jourdan 001; Jieke Shi; Phu Hong Nguyen; Valentina Golendukhina (ACM SIGSOFT Softw. Eng. Notes, 2024)

Personality-Guided Code Generation Using Large Language Models. - Yaoqi Guo; Zhenpeng Chen; Jie M. Zhang; Yang Liu 003; Yun Ma 002 (ACL (1), 2025)

LLM-Powered Test Case Generation for Detecting Bugs in Plausible Programs. - Kaibo Liu; Zhenpeng Chen; Yiyang Liu; Jie M. Zhang; Mark Harman; Yudong Han; Yun Ma 002; Yihong Dong; Ge Li 001; Gang Huang 001 (ACL (1), 2025)

Diversity Drives Fairness: Ensemble of Higher Order Mutants for Intersectional Fairness of Machine Learning Software. - Zhenpeng Chen; Xinyue Li; Jie M. Zhang; Federica Sarro; Yang Liu 003 (ICSE, 2025)

Knowledge-Enhanced Program Repair for Data Science Code. - Shuyin Ouyang; Jie M. Zhang; Zeyu Sun 004; Albert Meroño-Peñuela (ICSE, 2025)

Robustness evaluation of code generation systems via concretizing instructions. - Ming Yan 010; Junjie Chen 003; Jie M. Zhang; Xuejie Cao; Chen Yang; Mark Harman (Inf. Softw. Technol., 2025)

Software Fairness Dilemma: Is Bias Mitigation a Zero-Sum Game? - Zhenpeng Chen; Xinyue Li; Jie M. Zhang; Weisong Sun; Ying Xiao; Tianlin Li; Yiling Lou; Yang Liu 003 (Proc. ACM Softw. Eng., 2025)

Hallucination Detection in Large Language Models with Metamorphic Relations. - Borui Yang; Md Afif Al Mamun; Jie M. Zhang; Gias Uddin 001 (Proc. ACM Softw. Eng., 2025)

A Systematic Exploration of Mutation-Based Fault Localization Formulae. - Bo Wang 050; Jinkang Wei; Mingda Chen; Chong Chen; Youfang Lin; Jie M. Zhang (Softw. Test. Verification Reliab., 2025)

Fairness Aware Reinforcement Learning via Proximal Policy Optimization. - Gabriele La Malfa; Jie M. Zhang; Michael Luck; Elizabeth Black (CoRR, 2025)

An Empirical Study of the Non-Determinism of ChatGPT in Code Generation. - Shuyin Ouyang; Jie M. Zhang; Mark Harman; Meng Wang 002 (ACM Trans. Softw. Eng. Methodol., 2025)

Promptware Engineering: Software Engineering for LLM Prompt Development. - Zhenpeng Chen; Chong Wang 013; Weisong Sun; Guang Yang; Xuanzhe Liu; Jie M. Zhang; Yang Liu 003 (CoRR, 2025)

LLMs are Bug Replicators: An Empirical Study on LLMs' Capability in Completing Bug-prone Code. - Liwei Guo; Sixiang Ye; Zeyu Sun 004; Xiang Chen 005; Yuxia Zhang; Bo Wang 050; Jie M. Zhang; Zheng Li 002; Yong Liu 030 (CoRR, 2025)

LLMs Love Python: A Study of LLMs' Bias for Programming Languages and Libraries. - Lukas Twist; Jie M. Zhang; Mark Harman; Don Syme; Joost Noppen; Detlef D. Nauck (CoRR, 2025)

Bias behind the Wheel: Fairness Testing of Autonomous Driving Systems. - Xinyue Li; Zhenpeng Chen; Jie M. Zhang; Federica Sarro; Ying Zhang 012; Xuanzhe Liu (ACM Trans. Softw. Eng. Methodol., 2025)

EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code. - Yuhao Qing; Boyu Zhu; Mingzhe Du; Zhijiang Guo; Terry Yue Zhuo; Qianru Zhang; Jie M. Zhang; Heming Cui; Siu-Ming Yiu; Dong Huang 005; See-Kiong Ng; Luu Anh Tuan (CoRR, 2025)

DS-Bench: A Realistic Benchmark for Data Science Code Generation. - Shuyin Ouyang; Dong Huang 005; Jingwen Guo; Zeyu Sun 004; Qihao Zhu; Jie M. Zhang (CoRR, 2025)

AMQA: An Adversarial Dataset for Benchmarking Bias of LLMs in Medicine and Healthcare. - Ying Xiao; Jie Huang; Ruijuan He; Jing Xiao; Mohammad Reza Mousavi 001; Yepang Liu 001; Kezhi Li; Zhenpeng Chen; Jie M. Zhang (CoRR, 2025)

Large Language Models Miss the Multi-Agent Mark. - Emanuele La Malfa; Gabriele La Malfa; Samuele Marro; Jie M. Zhang; Elizabeth Black; Micheal Luck; Philip Torr 001; Michael J. Wooldridge (CoRR, 2025)

YATE: The Role of Test Repair in LLM-Based Unit Test Generation. - Michael Konstantinou; Renzo Degiovanni; Jie M. Zhang; Mark Harman; Mike Papadakis (CoRR, 2025)

Benchmarking LLMs for Unit Test Generation from Real-World Functions. - Dong Huang 005; Jie M. Zhang; Mark Harman; Qianru Zhang; Mingzhe Du; See-Kiong Ng (CoRR, 2025)

TRACY: Benchmarking Execution Efficiency of LLM-Based Code Translation. - Zhihao Gong; Zeyu Sun 004; Dong Huang 005; Qingyuan Liang; Jie M. Zhang; Dan Hao 001 (CoRR, 2025)