An improved quantum software challenges classification approach using transfer learning and explainable AI

Abstract

Quantum Software Engineering (QSE) has emerged as a research direction practiced by tech joints. Quantum developers face challenges in optimizing quantum computing and QSE concepts. Developers use Stack Overflow to discuss quantum challenges using specialized tags to label posts. These tags often refer to technical quantum aspects. Categorizing quantum practitioners’ questions by concept can help identify common QSE challenges. We conducted studies to classify quantum developers’ questions into various challenges. We extracted 2,829 developers questions from Q&A platforms using quantum-related tags. The posts were analyzed to identify frequent quantum-related challenges and develop a novel grounded theory. The challenges identified include Tooling, Theoretical, Learning, Conceptual, Errors, and API Usage. Through content analysis and grounded theory, the developers’ discussions were annotated with commonly reported quantum challenges to develop a ground truth dataset. ChatGPT was used to validate human annotations and resolve disagreements. Various fine-tuned transformer algorithms, including BERT, DistilBERT, and RoBERTa, were used to classify developer discussions into commonly reported quantum challenges. We achieved an average accuracy of 95% with BERT DistilBERT algorithms, compared to fine-tuned Deep and Machine Learning (D&ML) classifiers, including Feedforward Neural Networks (FNN), Convolutional Neural Networks (CNN), and Long Short-Term Memory networks (LSTM), which achieved accuracies of 89%, 86%, and 84%, respectively. The proposed Transformer-based approach outperforms the previous D&ML-based approach with a 6% increase in accuracy by processing actual developer discussions, i.e., without data augmentation. Furthermore, we applied SHAP (SHapley Additive exPlanations) to provide model interpretability, revealing how specific linguistic features drive predictions and enhancing transparency in the classification process. These improved research findings can help quantum vendors and developers’ discussion forums to better organize developers’ discussions for improved access and readability.

Graphical abstract

Keywords

quantum software engineering / repository mining / developer forums / stack overflow / transfer learning / natural language processing

References

[1] Nielsen M A, Chuang I L. Quantum Computation and Quantum Information. Cambridge: Cambridge University Press, 2010

[2] Preskill J . Reliable quantum computers. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 1998, 454( 1969): 385–410

[3] Arute F, Arya K, Babbush R, Bacon D, Bardin J C, Barends R, . et al. Quantum supremacy using a programmable superconducting processor. Nature, 2019, 574( 7779): 505–510

[4] Gambetta J. IBM’S roadmap for scaling quantum technology. Yorktown Heights: IBM Quantum, 2020 (https://www.ibm.com/quantum/blog/ibm-quantum-roadmap)

[5] Rigetti Computing. Create New Opportunities with Tomorrow’s Most Advanced Computing Technology. Berkeley: Rigetti Computing, 2020 (https://www.rigetti.com/why-quantum)

[6] Piattini M, Peterssen G, Pérez-Castillo R, Hevia J L, Serrano M A, et al. The Talavera manifesto for quantum software engineering and programming. In: Proceedings of the 1st International Workshop on the Quantum Software Engineering & Programming. 2020, 1−5

[7] Murillo J M, Garcia-Alonso J, Moguel E, Barzen J, Leymann F, Ali S, Yue T, Arcaini P, Pérez-Castillo R, García-Rodríguez de Guzmán I, Piattini M, Ruiz-Cortés A, Brogi A, Zhao J, Miranskyy A, Wimmer M . Quantum software engineering: roadmap and challenges ahead. ACM Transactions on Software Engineering and Methodology, 2025, 34( 5): 154

[8] Mandal A K, Nadim M, Roy C K, Roy B, Schneider K A . Quantum software engineering and potential of quantum computing in software engineering research: a review. Automated Software Engineering, 2025, 32( 1): 27

[9] Zhao J. Quantum software engineering: landscapes and horizons. 2021, arXiv preprint arXiv: 2007.07047

[10] Svore K, Geller A, Troyer M, Azariah J, Granade C, Heim B, Kliuchnikov V, Mykhailova M, Paz A, Roetteler M. Q#: enabling scalable quantum computing and development with a high-level DSL. In: Proceedings of Real World Domain Specific Languages Workshop 2018. 2018, 7

[11] JavadiAbhari A, Patil S, Kudrow D, Heckey J, Lvov A, Chong F T, Martonosi M . ScaffCC: scalable compilation and analysis of quantum programs. Parallel Computing, 2015, 45: 2–17

[12] IBM Quantum. Qiskit: an open-source SDK for working with quantum computers. IBM, 2024 (https://www.ibm.com/quantum/qiskit)

[13] El Aoun M R, Li H, Khomh F, Openja M. Understanding quantum software engineering challenges an empirical study on stack exchange forums and GitHub issues. In: Proceedings of 2021 IEEE International Conference on Software Maintenance and Evolution. 2021, 343−354

[14] Yue T, Ali S, Arcaini P. Towards quantum software requirements engineering. In: Proceedings of 2023 IEEE International Conference on Quantum Computing and Engineering. 2023, 161−164

[15] Ahmad A, Khan A A, Waseem M, Fahmideh M, Mikkonen T. Towards process centered architecting for quantum software systems. In: Proceedings of 2022 IEEE International Conference on Quantum Software. 2022, 26−31

[16] Khan A A, Ye B, Akbar M A, Khan J A, Mougouei D, Ma X. Mining Q&A platforms for empirical evidence on quantum software programming. 2025, arXiv preprint arXiv: 2503.05240

[17] Akbar M A, Khan A A, Rafi S . A systematic decision-making framework for tackling quantum software engineering challenges. Automated Software Engineering, 2023, 30( 2): 22

[18] Ali S, Yue T. Quantum software testing: a brief introduction. In: Proceedings of 2023 IEEE/ACM 45th International Conference on Software Engineering: Companion Proceedings. 2023, 332−333

[19] Ahmad A, Feng C, Ge S, Yousif A . A survey on mining stack overflow: question and answering (Q&A) community. Data Technologies and Applications, 2018, 52( 2): 190–247

[20] De Dieu M J, Liang P, Shahin M . How do OSS developers reuse architectural solutions from Q&A sites: an empirical study. IEEE Transactions on Software Engineering, 2025, 51( 7): 2015–2043

[21] Khan A A, Khan J A, Akbar M A, Zhou P, Fahmideh M . Insights into software development approaches: mining Q &A repositories. Empirical Software Engineering, 2024, 29( 1): 8

[22] Li T, Zhang X, Wang Y, Zhou Q, Wang Y, Dong F . Machine learning for requirements engineering (ML4RE): a systematic literature review complemented by practitioners’ voices from stack overflow. Information and Software Technology, 2024, 172: 107477

[23] Beyer S, Macho C, Di Penta M, Pinzger M . What kind of questions do developers ask on stack overflow? A comparison of automated approaches to classify posts into question categories. Empirical Software Engineering, 2020, 25( 3): 2258–2301

[24] Husain M, Khan M S, Khan J A, Khan N D, Khan A, Akbar M A. Exploring developers discussion forums for quantum software engineering: a fine-grained classification approach using large language model (ChatGPT). In: Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. 2025, 1742−1755

[25] Gill S S, Kumar A, Singh H, Singh M, Kaur K, Usman M, Buyya R . Quantum computing: a taxonomy, systematic review and future directions. Software: Practice and Experience, 2022, 52( 1): 66–114

[26] Vietz D, Barzen J, Leymann F, Wild K. On decision support for quantum application developers: categorization, comparison, and analysis of existing technologies. In: Proceedings of the 21st International Conference on Computational Science. 2021, 127−141

[27] Haghparast M, Mikkonen T, Nurminen J K, Stirbu V. Quantum software engineering challenges from developers’ perspective: mapping research challenges to the proposed workflow model. In: Proceedings of 2023 IEEE International Conference on Quantum Computing and Engineering. 2023, 173−176

[28] Treude C, Barzilay O, Storey M A. How do programmers ask and answer questions on the web? (NIER track). In: Proceedings of the 33rd International Conference on Software Engineering. 2011, 804−807

[29] Iftikhar H U, Rehman A U, Kalugina O A, Umer Q, Khan H A . Deep learning-based correct answer prediction for developer forums. IEEE Access, 2021, 9: 128166–128177

[30] Zhu W, Zhang H, Hassan A E, Godfrey M W . An empirical study of question discussions on stack overflow. Empirical Software Engineering, 2022, 27( 6): 148

[31] Khan J A, Yasin A, Fatima R, Vasan D, Khan A A, Khan A W . Valuating requirements arguments in the online user’s forum for requirements decision-making: the crowdRE-VArg framework. Software: Practice and Experience, 2022, 52( 12): 2537–2573

[32] Ali Khan J, Liu L, Wen L . Requirements knowledge acquisition from online user forums. IET Software, 2020, 14( 3): 242–253

[33] Beyer S, Pinzger M. A manual categorization of android app development issues on stack overflow. In: Proceedings of 2014 IEEE International Conference on Software Maintenance and Evolution. 2014, 531−535

[34] Zhang H, Shafiq M O . Survey of transformers and towards ensemble learning using transformers for natural language processing. Journal of Big Data, 2024, 11( 1): 25

[35] de Araújo A F, Marcacini R M. RE-BERT: automatic extraction of software requirements from app reviews using BERT language model. In: Proceedings of the 36th Annual ACM Symposium on Applied Computing. 2021, 1321−1327

[36] Sainani A, Anish P R, Joshi V, Ghaisas S. Extracting and classifying requirements from software engineering contracts. In: Proceedings of the 28th IEEE International Requirements Engineering Conference. 2020, 147−157

[37] Deshpande G, Sheikhi B, Chakka S, Zotegouon D L, Masahati M N, Ruhe G. Is BERT the new silver bullet?-An empirical investigation of requirements dependency classification. In: Proceedings of the 29th IEEE International Requirements Engineering Conference Workshops. 2021, 136−145

[38] Hey T, Keim J, Koziolek A, Tichy W F. NorBERT: transfer learning for requirements classification. In: Proceedings of the 28th IEEE International Requirements Engineering Conference. 2020, 169−179

[39] Henao P R, Fischbach J, Spies D, Frattini J, Vogelsang A. Transfer learning for mining feature requests and bug reports from tweets and app store reviews. In: Proceedings of the 29th IEEE International Requirements Engineering Conference Workshops. 2021, 80−86

[40] Kaur K, Kaur P. BERT-RCNN: an automatic classification of app reviews using transfer learning based RCNN deep model. 2023 (https://www.researchsquare.com/article/rs-2503700/v1)

[41] Bilal M, Almazroi A A . Effectiveness of fine-tuned BERT model in classification of helpful and unhelpful online customer reviews. Electronic Commerce Research, 2023, 23( 4): 2737–2757

[42] Devlin J, Chang M W, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171–4186

[43] Strauss A L, Corbin J M. Basics of Qualitative Research. 2nd ed. Thousand Oaks: Sage Publications, 1998

[44] Neuendorf K A. The Content Analysis Guidebook. 2nd ed. Los Angeles: SAGE, 2017

[45] Khan N D, Khan J A, Li J, Ullah T, Zhao Q . Mining software insights: uncovering the frequently occurring issues in low-rating software applications. PeerJ Computer Science, 2024, 10: e2115

[46] Khan N D, Khan J A, Li J, Ullah T, Zhao Q . Leveraging large language model ChatGPT for enhanced understanding of end-user emotions in social media feedbacks. Expert Systems with Applications, 2025, 261: 125524

[47] Rosen C, Shihab E . What are mobile developers asking about? A large scale study using stack overflow. Empirical Software Engineering, 2016, 21( 3): 1192–1223

[48] Allamanis M, Sutton C. Why, when, and what: analyzing stack overflow questions by topic, type, and code. In: Proceedings of the 10th Working Conference on Mining Software Repositories. 2013, 53−56

[49] Uddin G, Khomh F . Automatic mining of opinions expressed about APIs in stack overflow. IEEE Transactions on Software Engineering, 2021, 47( 3): 522–559

[50] Baquero J F, Camargo J E, Restrepo-Calle F, Aponte J H, González F A. Predicting the programming language: extracting knowledge from stack overflow posts. In: Proceedings of the 12th Colombian Conference on Computing. 2017, 199−210

[51] Tan Y, Xu S, Wang Z, Zhang T, Xu Z, Luo X . Bug severity prediction using question-and-answer pairs from stack overflow. Journal of Systems and Software, 2020, 165: 110567

[52] González-Carvajal S, Garrido-Merchán E C. Comparing BERT against traditional machine learning text classification. 2021, arXiv preprint arXiv: 2005.13012

[53] Sanh V, Debut L, Chaumond J, Wolf T. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. 2020, arXiv preprint arXiv: 1910.01108

[54] Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. RoBERTa: a robustly optimized BERT preprint approach. 2019, arXiv preprint arXiv: 1907.11692

[55] Yousuf M M, Sofi S A . Bug classification in quantum software: a rule-based framework and its evaluation. Automated Software Engineering, 2026, 33( 1): 39

[56] Aktar M S, Liang P, Waseem M, Tahir A, Ahmad A, Zhang B, Li Z . Architecture decisions in quantum software systems: an empirical study on stack exchange and GitHub. Information and Software Technology, 2025, 177: 107587

[57] Upadhyay K, Chhetri V, Siddique A B, Farooq U. Analyzing the evolution and maintenance of quantum software repositories. In: Proceedings of 2025 IEEE International Conference on Quantum Software. 2025, 173−184

Rights & permissions

Higher Education Press