Code context-based reviewer recommendation

Abstract

Code review is a critical process in software development, contributing to the overall quality of the product by identifying errors early. A key aspect of this process is the selection of appropriate reviewers to scrutinize changes made to source code. However, in large-scale open-source projects, selecting the most suitable reviewers for a specific change can be a challenging task. To address this, we introduce the Code Context Based Reviewer Recommendation (CCB-RR), a model that leverages information from changesets to recommend the most suitable reviewers. The model takes into consideration the paths of modified files and the context derived from the changesets, including their titles and descriptions. Additionally, CCB-RR employs KeyBERT to extract the most relevant keywords and compare the semantic similarity across changesets. The model integrates the paths of modified files, keyword information, and the context of code changes to form a comprehensive picture of the changeset. We conducted extensive experiments on four open-source projects, demonstrating the effectiveness of CCB-RR. The model achieved a Top-1 accuracy of 60%, 55%, 51%, and 45% on the Android, OpenStack, QT, and LibreOffice projects respectively. For Mean Reciprocal Rank (MRR), CCB achieved 71%, 62%, 52%, and 68% on the same projects respectively, thereby highlighting its potential for practical application in code reviewer recommendation.

Graphical abstract

Keywords

code reviewer recommendation / code context / pull request

Dawei Yuan is a PhD student in the School of Computer Science and Engineering, Macau University of Science and Technology (MUST), China. Before joining MUST, he worked as a software engineer at United Imaging Healthcare Corporation (UIH), China. He also got an MSc in Software Engineering from the University of Science and Technology of China and an BSc degree in Network Engineering from Nanjing University of Posts and Telecommunications, China. His research interests mainly focus on using artificial intelligence techniques to solve problems in software engineering, such as bug localization and code analysis

Xiao Peng received the BSc in information and computing science from the University of Jinan, China in 2020, and MSc in applied mathematics and data science from Macau University of Science and Technology, China in 2022. She is currently working toward the PhD degree in science at Macau University of Science and Technology, China. Her research interests include software engineering and deep learning

Zijie Chen received the MSc degrees in Applied Mathematics and Data Science from Macau University of Science and Technology, China, and the BSc degree in Software Engineering from Beijing Institute of Technology, China. He is currently pursuing the PhD degree in the School of Computer Science and Engineering, Macau University of Science and Technology, China. His research interests include deep learning, human activity recognition, and Internet of Things

Tao Zhang received the BS degree in automation, the MEng degree in software engineering from Northeastern University, China, and the PhD degree in computer science from the University of Seoul, Republic of Korea. After that, he spent one year with The Hong Kong Polytechnic University, China as a postdoctoral research fellow. Currently, he is an associate professor with the School of Computer Science and Engineering, Macau University of Science and Technology (MUST), China. Before joining MUST, he was the faculty member of Harbin Engineering University and Nanjing University of Posts and Telecommunications, China. He published more than 90 high-quality papers at renowned software engineering and security journals and concerences. He served as the General Chair of SANER 2023 and the PC members of several top-tier SE concerences such as FSE and ASE. He is a senior member of ACM and IEEE

Ruijia Lei received the BSc degree in Software Engineering from Macau University of Science and Technology, China in 2021, the MSc degree in Computer Science from University of Amsterdam, the Netherlands in 2023, respectively. His research interests include software technology, machine learning, large-scale data processing, distributed systems, and principles and applications of LLMs

References

[1] Xia X, Lo D, Wang X, Yang X. Who should review this change?: Putting text and file location analyses together for more accurate recommendations. In: Proceedings of the 31st IEEE International Conference on Software Maintenance and Evolution. 2015, 261–270

[2] Shull F, Seaman C . Inspecting the history of inspections: an example of evidence-based technology diffusion. IEEE Software, 2008, 25( 1): 88–90

[3] Chueshev A, Lawall J, Bendraou R, Ziadi T. Expanding the number of reviewers in open-source projects by recommending appropriate developers. In: Proceedings of the 36th IEEE International Conference on Software Maintenance and Evolution. 2020, 499–510

[4] Thongtanunam P, Tantithamthavorn C, Kula R G, Yoshida N, Iida H, Matsumoto K I. Who should review my code? A file location-based code-reviewer recommendation approach for modern code review. In: Proceedings of the 22nd International Conference on Software Analysis, Evolution, and Reengineering. 2015, 141–150

[5] Goto I, Tanaka H. Detecting untranslated content for neural machine translation. In: Proceedings of the 1st Workshop on Neural Machine Translation. 2017, 47–55

[6] Stahlberg F . Neural machine translation: a review. Journal of Artificial Intelligence Research, 2020, 69: 343–418

[7] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000–6010

[8] Yu Y, Wang H, Yin G, Wang T. Reviewer recommendation for pull-requests in GitHub: what can we learn from code review and bug assignment? Information and Software Technology, 2016, 74: 204−218

[9] Zanjani M B, Kagdi H, Bird C . Automatically recommending peer reviewers in modern code review. Transactions on Software Engineering, 2016, 42( 6): 530–543

[10] Bacchelli A, Bird C. Expectations, outcomes, and challenges of modern code review. In: Proceedings of the 35th International Conference on Software Engineering. 2013, 712–721

[11] Rigby P C, Storey M A. Understanding broadcast based peer review on open source software projects. In: Proceedings of the 33rd International Conference on Software Engineering. 2011, 541–550

[12] Yu Y, Wang H, Filkov V, Devanbu P, Vasilescu B. Wait for it: determinants of pull request evaluation latency on GitHub. In: Proceedings of the 12th Working Conference on Mining Software Repositories. 2015, 367–371

[13] Tymchuk Y, Mocci A, Lanza M. Code review: veni, ViDI, vici. In: Proceedings of the 22nd International Conference on Software Analysis, Evolution, and Reengineering. 2015, 151–160

[14] Qiu X, Sun T, Xu Y, Shao Y, Dai N, Huang X . Pre-trained models for natural language processing: a survey. Science China Technological Sciences, 2020, 63( 10): 1872–1897

[15] Lavrenko V, Croft W B . Relevance-based language models. ACM SIGIR Forum, 2017, 51( 2): 260–267

[16] Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 3111–3119

[17] Pennington J, Socher R, Manning C. GloVe: global vectors for word representation. In: Proceedings of the 19th Conference on Empirical Methods in Natural Language Processing. 2014, 1532–1543

[18] Cherney L R, Kaye R C, Lee J B, van Vuuren S . Impact of personal relevance on acquisition and generalization of script training for aphasia: a preliminary analysis. American Journal of Speech-Language Pathology, 2015, 24( 4): S913–S922

[19] Lourie N, Le Bras R, Bhagavatula C, Choi Y. UNICORN on RAINBOW: a universal commonsense reasoning model on a new multitask benchmark. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 13480–13488

[20] Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov R, Le Q V. XLNet: generalized autoregressive pretraining for language understanding. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 517

[21] Liu Z, Lin W, Shi Y, Zhao J. A robustly optimized BERT pre-training approach with post-training. In: Proceedings of the 20th China National Conference on Chinese Computational Linguistics. 2021, 471–484

[22] Chi P H, Chung P H, Wu T H, Hsieh C C, Chen Y H, Li S W, Lee H Y. Audio albert: a lite bert for self-supervised learning of audio representation. In: Proceedings of 2021 IEEE Spoken Language Technology Workshop. 2021, 344−350

[23] Brown T B, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler D M, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D. Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 159

[24] Hannebauer C, Patalas M, Stünkel S, Gruhn V. Automatically recommending code reviewers based on their expertise: an empirical comparison. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. 2016, 99−110

[25] Doğan E, Tüzün E, Tecimer K A, Güvenir H A. Investigating the validity of ground truth in code reviewer recommendation studies. In: Proceedings of the 13th International Symposium on Empirical Software Engineering and Measurement. 2019, 1−6

[26] Ye X, Zheng Y, Aljedaani W, Mkaouer M W . Recommending pull request reviewers based on code changes. Soft Computing, 2021, 25( 7): 5619–5632

[27] Fejzer M, Przymus P, Stencel K . Profile based recommendation of code reviewers. Journal of Intelligent Information Systems, 2018, 50( 3): 597–619

[28] Ye X . Learning to rank reviewers for pull requests. IEEE Access, 2019, 7: 85382–85391

[29] Firoozeh N, Nazarenko A, Alizon F, Daille B . Keyword extraction: issues and methods. Natural Language Engineering, 2020, 26( 3): 259–291

[30] Piskorski J, Stefanovitch N, Jacquet G, Podavini A. Exploring linguistically-lightweight keyword extraction techniques for indexing news articles in a multilingual set-up. In: Proceedings of the 16th EACL Hackashop on News Media Content Analysis and Automated Report Generation. 2021, 35−44

[31] Weninger F, Geiger J, Wöllmer M, Schuller B, Rigoll G . Feature enhancement by deep LSTM networks for ASR in reverberant multisource environments. Computer Speech & Language, 2014, 28( 4): 888–902

[32] Jiang N, Lutellier T, Tan L. CURE: code-aware neural machine translation for automatic program repair. In: Proceedings of the 43rd International Conference on Software Engineering. 2021, 1161−1173

[33] Ouni A, Kula R G, Inoue K. Search-based peer reviewers recommendation in modern code review. In: Proceedings of the 32nd International Conference on Software Maintenance and Evolution. 2016, 367−377

[34] Chouchen M, Ouni A, Mkaouer M W, Kula R G, Inoue K . WhoReview: a multi-objective search-based approach for code reviewers recommendation in modern code review. Applied Soft Computing, 2021, 100: 106908

[35] Campos R, Mangaravite V, Pasquali A, Jorge A, Nunes C, Jatowt A . YAKE! Keyword extraction from single documents using multiple local features. Information Sciences, 2020, 509: 257–289

[36] Nasrabadi N M, King R A . Image coding using vector quantization: a review. IEEE Transactions on Communications, 1988, 36( 8): 957–971

[37] de Lima Junior M L, Soares D M, Plastino A, Murta L . Automatic assignment of integrators to pull requests: the importance of selecting appropriate attributes. Journal of Systems and Software, 2018, 144: 181–196

[38] Mirsaeedi E, Rigby P C. Mitigating turnover with code review recommendation: balancing expertise, workload, and knowledge distribution. In: Proceedings of the 42nd International Conference on Software Engineering. 2020, 1183−1195

[39] Al-Zubaidi W H A, Thongtanunam P, Dam H K, Tantithamthavorn C, Ghose A. Workload-aware reviewer recommendation using a multi-objective search-based approach. In: Proceedings of the 16th ACM International Conference on Predictive Models and Data Analytics in Software Engineering. 2020, 21−30

[40] Balachandran V. Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation. In: Proceedings of the 35th International Conference on Software Engineering. 2013, 931−940

[41] Asthana S, Kumar R, Bhagwan R, Bird C, Bansal C, Maddila C, Mehta S, Ashok B. WhoDo: automating reviewer suggestions at scale. In: Proceedings of the 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2019, 937−945

[42] Salton G, Buckley C . Term-weighting approaches in automatic text retrieval. Information Processing & Management, 1988, 24( 5): 513–523

[43] Mihalcea R, Tarau P. TextRank: bringing order into text. In: Proceedings of the 2nd Conference on Empirical Methods in Natural Language Processing. 2004, 404−411

[44] Erkan G, Radev D R . LexRank: graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research, 2004, 22: 457–479

[45] Nguyen T D, Luong M T. WINGNUS: keyphrase extraction utilizing document logical structure. In: Proceedings of the 5th International Workshop on Semantic Evaluation. 2010, 166−169

[46] Rebai S, Amich A, Molaei S, Kessentini M, Kazman R . Multi-objective code reviewer recommendations: balancing expertise, availability and collaborations. Automated Software Engineering, 2020, 27( 3-4): 301–328

[47] Çetin H A, Doğan E, Tüzün E . A review of code reviewer recommendation studies: challenges and future directions. Science of Computer Programming, 2021, 208: 102652

[48] Kong D, Chen Q, Bao L, Sun C, Xia X, Li S. Recommending code reviewers for proprietary software projects: a large scale study. In: Proceedings of 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). 2022, 630−640

[49] Li R, Liang P, Avgeriou P. Code reviewer recommendation for architecture violations: an exploratory study. In: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. 2023, 42−51

Acknowledgement

This work was supported in part by the Science and Technology Development Fund (FDCT), Macau SAR, China (Nos. 0047/2020/A1 and 0014/2022/A).

Competing interests

The authors declare that they have no competing interests or financial conflicts to disclose.

RIGHTS & PERMISSIONS

2025 Higher Education Press