On Testing for Biases in Peer Review. - Ivan Stelmakh; Nihar B. Shah; Aarti Singh (NeurIPS, 2019)

A Large Scale Randomized Controlled Trial on Herding in Peer-Review Discussions. - Ivan Stelmakh; Charvi Rastogi; Nihar B. Shah; Aarti Singh; Hal Daumé III (CoRR, 2020)

Debiasing Evaluations That Are Biased by Evaluations. - Jingyan Wang 001; Ivan Stelmakh; Yuting Wei 001; Nihar B. Shah (AAAI, 2021)

Towards Fair, Equitable, and Efficient Peer Review. - Ivan Stelmakh (AAAI, 2021)

Catch Me if I Can: Detecting Strategic Behaviour in Peer Assessment. - Ivan Stelmakh; Nihar B. Shah; Aarti Singh (AAAI, 2021)

A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences. - Ivan Stelmakh; Nihar B. Shah; Aarti Singh; Hal Daumé III (AAAI, 2021)

VLDB 2021 Crowd Science Challenge on Aggregating Crowdsourced Audio Transcriptions. - Dmitry Ustalov; Nikita Pavlichenko; Ivan Stelmakh; Dmitriy Kuznetsov (CSW@VLDB, 2021)

CrowdSpeech and Vox DIY: Benchmark Dataset for Crowdsourced Audio Transcription. - Nikita Pavlichenko; Ivan Stelmakh; Dmitry Ustalov (NeurIPS Datasets and Benchmarks, 2021)

Vox Populi, Vox DIY: Benchmark Dataset for Crowdsourced Audio Transcription. - Nikita Pavlichenko; Ivan Stelmakh; Dmitry Ustalov (CoRR, 2021)

PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review. - Ivan Stelmakh; Nihar B. Shah; Aarti Singh (J. Mach. Learn. Res., 2021)

Prior and Prejudice: The Novice Reviewers' Bias against Resubmissions in Conference Peer Review. - Ivan Stelmakh; Nihar B. Shah; Aarti Singh; Hal Daumé III (Proc. ACM Hum. Comput. Interact., 2021)

ASQA: Factoid Questions Meet Long-Form Answers. - Ivan Stelmakh; Yi Luan; Bhuwan Dhingra; Ming-Wei Chang (EMNLP, 2022)

No Rose for MLE: Inadmissibility of MLE for Evaluation Aggregation Under Levels of Expertise. - Charvi Rastogi; Ivan Stelmakh; Nihar B. Shah; Sivaraman Balakrishnan (ISIT, 2022)

Cite-seeing and Reviewing: A Study on Citation Bias in Peer Review. - Ivan Stelmakh; Charvi Rastogi; Ryan Liu 001; Shuchi Chawla 001; Federico Echenique; Nihar B. Shah (CoRR, 2022)

To ArXiv or not to ArXiv: A Study Quantifying Pros and Cons of Posting Preprints Online. - Charvi Rastogi; Ivan Stelmakh; Xinwei Shen; Marina Meila; Federico Echenique; Shuchi Chawla 001; Nihar B. Shah (CoRR, 2022)

How do Authors' Perceptions of their Papers Compare with Co-authors' Perceptions and Peer-review Decisions? - Charvi Rastogi; Ivan Stelmakh; Alina Beygelzimer; Yann N. Dauphin; Percy Liang; Jennifer Wortman Vaughan; Zhenyu Xue; Hal Daumé III; Emma Pierson; Nihar B. Shah (CoRR, 2022)

Making Scientific Peer Review Scientific - Ivan Stelmakh (2022)

A Gold Standard Dataset for the Reviewer Assignment Problem. - Ivan Stelmakh; John Wieting; Graham Neubig; Nihar B. Shah (CoRR, 2023)

Peer Reviews of Peer Reviews: A Randomized Controlled Trial and Other Experiments. - Alexander Goldberg; Ivan Stelmakh; Kyunghyun Cho; Alice H. Oh; Alekh Agarwal; Danielle Belgrave; Nihar B. Shah (CoRR, 2023)

A Randomized Controlled Trial on Anonymizing Reviewers to Each Other in Peer Review Discussions. - Charvi Rastogi; Xiangchen Song; Zhijing Jin 001; Ivan Stelmakh; Hal Daumé III; Kun Zhang 001; Nihar B. Shah (CoRR, 2024)

Causal Effect of Group Diversity on Redundancy and Coverage in Peer-Reviewing. - Navita Goyal; Ivan Stelmakh; Nihar B. Shah; Hal Daumé III (CoRR, 2024)

A Gold Standard Dataset for the Reviewer Assignment Problem. - Ivan Stelmakh; John Frederick Wieting; Yang Xi; Graham Neubig; Nihar B. Shah (Trans. Mach. Learn. Res., 2025)