Code context-based reviewer recommendation

Abstract

Code review is a critical process in software development, contributing to the overall quality of the product by identifying errors early. A key aspect of this process is the selection of appropriate reviewers to scrutinize changes made to source code. However, in large-scale open-source projects, selecting the most suitable reviewers for a specific change can be a challenging task. To address this, we introduce the Code Context Based Reviewer Recommendation (CCB-RR), a model that leverages information from changesets to recommend the most suitable reviewers. The model takes into consideration the paths of modified files and the context derived from the changesets, including their titles and descriptions. Additionally, CCB-RR employs KeyBERT to extract the most relevant keywords and compare the semantic similarity across changesets. The model integrates the paths of modified files, keyword information, and the context of code changes to form a comprehensive picture of the changeset. We conducted extensive experiments on four open-source projects, demonstrating the effectiveness of CCB-RR. The model achieved a Top-1 accuracy of 60%, 55%, 51%, and 45% on the Android, OpenStack, QT, and LibreOffice projects respectively. For Mean Reciprocal Rank (MRR), CCB achieved 71%, 62%, 52%, and 68% on the same projects respectively, thereby highlighting its potential for practical application in code reviewer recommendation.

Graphical abstract

Keywords

code reviewer recommendation / code context / pull request

References

[1] Xia X, Lo D, Wang X, Yang X. Who should review this change?: Putting text and file location analyses together for more accurate recommendations. In: Proceedings of the 31st IEEE International Conference on Software Maintenance and Evolution. 2015, 261–270

[2] Shull F, Seaman C . Inspecting the history of inspections: an example of evidence-based technology diffusion. IEEE Software, 2008, 25( 1): 88–90

[3] Chueshev A, Lawall J, Bendraou R, Ziadi T. Expanding the number of reviewers in open-source projects by recommending appropriate developers. In: Proceedings of the 36th IEEE International Conference on Software Maintenance and Evolution. 2020, 499–510

[4] Thongtanunam P, Tantithamthavorn C, Kula R G, Yoshida N, Iida H, Matsumoto K I. Who should review my code? A file location-based code-reviewer recommendation approach for modern code review. In: Proceedings of the 22nd International Conference on Software Analysis, Evolution, and Reengineering. 2015, 141–150

[5] Goto I, Tanaka H. Detecting untranslated content for neural machine translation. In: Proceedings of the 1st Workshop on Neural Machine Translation. 2017, 47–55

[6] Stahlberg F . Neural machine translation: a review. Journal of Artificial Intelligence Research, 2020, 69: 343–418

[7] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000–6010

[8] Yu Y, Wang H, Yin G, Wang T. Reviewer recommendation for pull-requests in GitHub: what can we learn from code review and bug assignment? Information and Software Technology, 2016, 74: 204−218

[9] Zanjani M B, Kagdi H, Bird C . Automatically recommending peer reviewers in modern code review. Transactions on Software Engineering, 2016, 42( 6): 530–543

[10] Bacchelli A, Bird C. Expectations, outcomes, and challenges of modern code review. In: Proceedings of the 35th International Conference on Software Engineering. 2013, 712–721

[11] Rigby P C, Storey M A. Understanding broadcast based peer review on open source software projects. In: Proceedings of the 33rd International Conference on Software Engineering. 2011, 541–550

[12] Yu Y, Wang H, Filkov V, Devanbu P, Vasilescu B. Wait for it: determinants of pull request evaluation latency on GitHub. In: Proceedings of the 12th Working Conference on Mining Software Repositories. 2015, 367–371

[13] Tymchuk Y, Mocci A, Lanza M. Code review: veni, ViDI, vici. In: Proceedings of the 22nd International Conference on Software Analysis, Evolution, and Reengineering. 2015, 151–160

[14] Qiu X, Sun T, Xu Y, Shao Y, Dai N, Huang X . Pre-trained models for natural language processing: a survey. Science China Technological Sciences, 2020, 63( 10): 1872–1897

[15] Lavrenko V, Croft W B . Relevance-based language models. ACM SIGIR Forum, 2017, 51( 2): 260–267

[16] Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013, 3111–3119

[17] Pennington J, Socher R, Manning C. GloVe: global vectors for word representation. In: Proceedings of the 19th Conference on Empirical Methods in Natural Language Processing. 2014, 1532–1543

[18] Cherney L R, Kaye R C, Lee J B, van Vuuren S . Impact of personal relevance on acquisition and generalization of script training for aphasia: a preliminary analysis. American Journal of Speech-Language Pathology, 2015, 24( 4): S913–S922

[19] Lourie N, Le Bras R, Bhagavatula C, Choi Y. UNICORN on RAINBOW: a universal commonsense reasoning model on a new multitask benchmark. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 13480–13488

[20] Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov R, Le Q V. XLNet: generalized autoregressive pretraining for language understanding. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 517

[21] Liu Z, Lin W, Shi Y, Zhao J. A robustly optimized BERT pre-training approach with post-training. In: Proceedings of the 20th China National Conference on Chinese Computational Linguistics. 2021, 471–484

[22] Chi P H, Chung P H, Wu T H, Hsieh C C, Chen Y H, Li S W, Lee H Y. Audio albert: a lite bert for self-supervised learning of audio representation. In: Proceedings of 2021 IEEE Spoken Language Technology Workshop. 2021, 344−350

[23] Brown T B, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler D M, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D. Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 159

[24] Hannebauer C, Patalas M, Stünkel S, Gruhn V. Automatically recommending code reviewers based on their expertise: an empirical comparison. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. 2016, 99−110

[25] Doğan E, Tüzün E, Tecimer K A, Güvenir H A. Investigating the validity of ground truth in code reviewer recommendation studies. In: Proceedings of the 13th International Symposium on Empirical Software Engineering and Measurement. 2019, 1−6

[26] Ye X, Zheng Y, Aljedaani W, Mkaouer M W . Recommending pull request reviewers based on code changes. Soft Computing, 2021, 25( 7): 5619–5632

[27] Fejzer M, Przymus P, Stencel K . Profile based recommendation of code reviewers. Journal of Intelligent Information Systems, 2018, 50( 3): 597–619

[28] Ye X . Learning to rank reviewers for pull requests. IEEE Access, 2019, 7: 85382–85391

[29] Firoozeh N, Nazarenko A, Alizon F, Daille B . Keyword extraction: issues and methods. Natural Language Engineering, 2020, 26( 3): 259–291

[30] Piskorski J, Stefanovitch N, Jacquet G, Podavini A. Exploring linguistically-lightweight keyword extraction techniques for indexing news articles in a multilingual set-up. In: Proceedings of the 16th EACL Hackashop on News Media Content Analysis and Automated Report Generation. 2021, 35−44

[31] Weninger F, Geiger J, Wöllmer M, Schuller B, Rigoll G . Feature enhancement by deep LSTM networks for ASR in reverberant multisource environments. Computer Speech & Language, 2014, 28( 4): 888–902

[32] Jiang N, Lutellier T, Tan L. CURE: code-aware neural machine translation for automatic program repair. In: Proceedings of the 43rd International Conference on Software Engineering. 2021, 1161−1173

[33] Ouni A, Kula R G, Inoue K. Search-based peer reviewers recommendation in modern code review. In: Proceedings of the 32nd International Conference on Software Maintenance and Evolution. 2016, 367−377

[34] Chouchen M, Ouni A, Mkaouer M W, Kula R G, Inoue K . WhoReview: a multi-objective search-based approach for code reviewers recommendation in modern code review. Applied Soft Computing, 2021, 100: 106908

[35] Campos R, Mangaravite V, Pasquali A, Jorge A, Nunes C, Jatowt A . YAKE! Keyword extraction from single documents using multiple local features. Information Sciences, 2020, 509: 257–289

[36] Nasrabadi N M, King R A . Image coding using vector quantization: a review. IEEE Transactions on Communications, 1988, 36( 8): 957–971

[37] de Lima Junior M L, Soares D M, Plastino A, Murta L . Automatic assignment of integrators to pull requests: the importance of selecting appropriate attributes. Journal of Systems and Software, 2018, 144: 181–196

[38] Mirsaeedi E, Rigby P C. Mitigating turnover with code review recommendation: balancing expertise, workload, and knowledge distribution. In: Proceedings of the 42nd International Conference on Software Engineering. 2020, 1183−1195

[39] Al-Zubaidi W H A, Thongtanunam P, Dam H K, Tantithamthavorn C, Ghose A. Workload-aware reviewer recommendation using a multi-objective search-based approach. In: Proceedings of the 16th ACM International Conference on Predictive Models and Data Analytics in Software Engineering. 2020, 21−30

[40] Balachandran V. Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation. In: Proceedings of the 35th International Conference on Software Engineering. 2013, 931−940

[41] Asthana S, Kumar R, Bhagwan R, Bird C, Bansal C, Maddila C, Mehta S, Ashok B. WhoDo: automating reviewer suggestions at scale. In: Proceedings of the 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2019, 937−945

[42] Salton G, Buckley C . Term-weighting approaches in automatic text retrieval. Information Processing & Management, 1988, 24( 5): 513–523

[43] Mihalcea R, Tarau P. TextRank: bringing order into text. In: Proceedings of the 2nd Conference on Empirical Methods in Natural Language Processing. 2004, 404−411

[44] Erkan G, Radev D R . LexRank: graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research, 2004, 22: 457–479

[45] Nguyen T D, Luong M T. WINGNUS: keyphrase extraction utilizing document logical structure. In: Proceedings of the 5th International Workshop on Semantic Evaluation. 2010, 166−169

[46] Rebai S, Amich A, Molaei S, Kessentini M, Kazman R . Multi-objective code reviewer recommendations: balancing expertise, availability and collaborations. Automated Software Engineering, 2020, 27( 3-4): 301–328

[47] Çetin H A, Doğan E, Tüzün E . A review of code reviewer recommendation studies: challenges and future directions. Science of Computer Programming, 2021, 208: 102652

[48] Kong D, Chen Q, Bao L, Sun C, Xia X, Li S. Recommending code reviewers for proprietary software projects: a large scale study. In: Proceedings of 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). 2022, 630−640

[49] Li R, Liang P, Avgeriou P. Code reviewer recommendation for architecture violations: an exploratory study. In: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. 2023, 42−51

RIGHTS & PERMISSIONS

Higher Education Press