Q&A system for international construction contracts driven by large language model and knowledge graph

Abstract

As international construction projects continue to expand, construction enterprises are accumulating vast amounts of contract-related text data, making the effective management and extraction of knowledge from these dense texts essential to mitigate knowledge loss and ensure efficient contract management. The advent of large language models (LLMs) presents a promising avenue for enhancing contract knowledge management through intelligent systems. However, challenges such as hallucination, inflexibility, and lack of interpretability often diminish practitioners’ confidence in applying these models to real-world scenarios. This study seeks to develop a knowledge-based question-and-answer (Q&A) system for international construction contracts by integrating both the knowledge graph (KG) and the LLM. Built upon a domain-specific KG derived from the 2022 edition of the Fédération Internationale des Ingénieurs-Conseils (FIDIC) Yellow Book and the NEC4 Conditions of Contract, the system leverages LLM to conduct synergistic reasoning with the KG, enabling it to answer complex queries using both tacit knowledge and external sources. Experimental results demonstrate that the proposed approach markedly enhances the model’s performance in Q&A tasks of contract knowledge, achieving an average success rate exceeding 87% in terms of both accuracy and interpretability. This model provides a specialized Q&A system for international construction enterprises, facilitating flexible knowledge acquisition and task-oriented analysis in contract management, while also introducing a novel framework for integrating AI technologies into the management of international construction contracts.

Graphical abstract

Keywords

international construction contract / knowledge graph / large language model / knowledge management

References

[1] Adlakha V, BehnamGhader P, Lu X H, Meade N, Reddy S, (2024). Evaluating correctness and faithfulness of instruction-following models for question answering. Transactions of the Association for Computational Linguistics, 12: 681–699

[2] Aghaei SMasoudi SChhetri T RFensel A (2022). Question answering over knowledge graphs: A graph-driven approach. In: IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), 296–302

[3] Alomari E A, (2024). Unlocking the potential: A comprehensive systematic review of ChatGPT in natural language processing tasks. Computer Modeling in Engineering and Sciences, 141( 1): 43–85

[4] Banerjee S, Potts C M, Jhala A H, Jaselskis E J, (2023). Developing a construction domain–specific artificial intelligence language model for NCDOT’s CLEAR program to promote organizational innovation and institutional knowledge. Journal of Computing in Civil Engineering, 37( 3): 04023007

[5] Baek JAji A FSaffari A (2023). Knowledge-augmented language model prompting for zero-shot knowledge graph question answering. arXiv:2306.04136

[6] Bang YCahyawijaya SLee NDai WSu DWilie BLovenia HJi ZYu TChung WDo Q VXu YFung P (2023). A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. arXiv:2302.04023

[7] Candaş A B, Tokdemir O B, (2022). Automating Coordination Efforts for Reviewing Construction Contracts with Multilabel Text Classification. Journal of Construction Engineering and Management, 148( 6): 04022027

[8] Cerovsek T, (2011). A review and outlook for a ‘Building Information Model’(BIM): A multi-standpoint framework for technological development. Advanced Engineering Informatics, 25( 2): 224–244

[9] Chang Y, Wang X, Wang J, Wu Y, Yang L, Zhu K, Chen H, Yi X, Wang C, Wang Y, Ye W, Zhang Y, Chang Y, Yu P S, Yang Q, Xie X, (2024). A Survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15( 3): 1–45

[10] Chen G HChen SLiu ZJiang FWang B (2024). Humans or LLMs as the judge? A study on judgement Biases. arXiv:2402.10669

[11] Chen SHou YCui YChe WLiu TYu X (2020). Recall and learn: Fine-tuning deep pretrained language models with less forgetting. arXiv:2004.12651

[12] Chen Y, Liang B, Hu H, (2025). Research on ontology-based construction risk knowledge base development in deep foundation pit excavation. Journal of Asian Architecture and Building Engineering, 24( 3): 1640–1658

[13] Chen Y, Wang W, Zhang S, You J, (2018). Understanding the multiple functions of construction contracts: The anatomy of FIDIC model contracts. Construction Management and Economics, 36( 8): 472–485

[14] Choi S J, Gulati M, Posner E A, (2013). The dynamics of contract evolution. New York University Law Review, 88( 1): 1–50

[15] Cortes E G, Woloszyn V, Barone D, Möller S, Vieira R, (2022). A systematic review of question answering systems for non-factoid questions. Journal of Intelligent Information Systems, 58( 3): 453–480

[16] Chowdhary K R (2020). Natural language processing. Fundamentals of Artificial Intelligence, 603–649

[17] Danilevsky MQian KAharonov RKatsis YKawas BSen P (2020). A survey of the state of explainable AI for natural language processing. arXiv:2010.00711

[18] Deng H, Xu Y, Deng Y, Lin J, (2022). Transforming knowledge management in the construction industry through information and communications technology: A 15-year review. Automation in Construction, 142: 104530

[19] Dettmers T, Pagnoni A, Holtzman A, Zettlemoyer L, (2024). Qlora: Efficient finetuning of quantized LLMs. Advances in Neural Information Processing Systems, 36: 10088–10115

[20] Devlin JChang M-WLee KToutanova K (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805

[21] Ding XLi ZLiu TLiao K (2019). ELG: An event logic graph. arXiv:1907.08015

[22] Edge DTrinh HCheng NBradley JChao AMody ATruitt SLarson J (2024). From local to global: A graph RAG approach to query-focused summarization. arXiv:2404.16130

[23] Ehrlinger L, Wöß W, (2016). Towards a definition of knowledge graphs. SEMANTiCS (Posters, Demos, SuCCESS), 48( 1‒4): 2

[24] El-Diraby T A, Lima C, Feis B, (2005). Domain taxonomy for construction concepts: Toward a formal ontology for construction knowledge. Journal of Computing in Civil Engineering, 19( 4): 394–406

[25] Elkhayat Y, Marzouk M, (2022). Selecting feasible standard form of construction contracts using text analysis. Advanced Engineering Informatics, 52: 101569

[26] Fan WDing YNing LWang SLi HYin DChua T SLi Q (2024). A survey on RAG meeting LLMs: Towards retrieval-augmented large language models. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 6491‒6501

[27] Fu T, Liu S, Li P, (2024). Intelligent smelting process management system: Efficient and intelligent management strategy by incorporating large language model. Frontiers of Engineering Management, 11( 3): 396–412

[28] Gao Y, Gan Y, Chen Y, Chen Y, (2025). Application of large language models to intelligently analyze long construction contract texts. Construction Management and Economics, 43( 3): 226–242

[29] Gao YXiong YWang MWang H (2024). Modular RAG: transforming RAG systems into LEGO-like reconfigurable frameworks. arXiv:2407.21059

[30] Gilardi F, Alizadeh M, Kubli M, (2023). ChatGPT outperforms crowd workers for text-annotation tasks. In: Proceedings of the National Academy of Sciences of the United States of America, 120( 30): e2305016120

[31] Gruber T R, (1995). Toward principles for the design of ontologies used for knowledge sharing. International Journal of Human-Computer Studies, 43( 5–6): 907–928

[32] Guan S, Cheng X, Bai L, Zhang F, Li Z, Zeng Y, Jin X, Guo J, (2022). What is event knowledge graph: A Survey. IEEE Transactions on Knowledge and Data Engineering, 35( 7): 1–21

[33] Hassan F U, Le T, Lv X, (2021). Addressing legal and contractual matters in construction using natural language processing: A critical review. Journal of Construction Engineering and Management, 147( 9): 03121004

[34] Huang HQu YBu XZhou HLiu JYang MXu BZhao T (2024). An empirical study of LLM-as-a-Judge for LLM evaluation: Fine-tuned judge model is not a general substitute for GPT-4. arXiv:2403.02839

[35] Ji S, Pan S, Cambria E, Marttinen P, Yu P S, (2022). A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 33( 2): 494–514

[36] Kalogeraki M, Antoniou F, (2024). Claim management and dispute resolution in the construction industry: Current research trends using novel technologies. Buildings, 14( 4): 967

[37] Kivrak S, Arslan G, Dikmen I, Birgonul M T, (2008). Capturing knowledge in construction projects: Knowledge platform for contractors. Journal of Management Engineering, 24( 2): 87–95

[38] Knez T, Žitnik S, (2023). Event-centric tmporal knowledge graph construction: A survey. Mathematics, 11( 23): 4852

[39] Kojima T, Gu S S, Reid M, Matsuo Y, Iwasawa Y, (2022). Large language models are zero-shot reasoners. Advances in Neural Information Processing Systems, 35: 22199–22213

[40] Kwiatkowski T, Palomaki J, Redfield O, Collins M, Parikh A, Alberti C, Epstein D, Polosukhin I, Devlin J, Lee K, Toutanova K, Jones L, Kelcey M, Chang M-W, Dai A M, Uszkoreit J, Le Q, Petrov S, (2019). Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7: 453–466

[41] Lacosta AThomas C (2020). Enterprise social networking knowledge management tools and knowledge dynamics. In: 2020 IEEE International Conference on Technology Management, Operations and Decisions (ICTMOD), 1–8

[42] Ling CZhao XLu JDeng CZheng CWang JChowdhury TLi YCui HZhang XZhao TPanalkar ACheng WWang HLiu YChen ZChen HWhite CGu QZhao L (2023). Domain specialization as the key to make large language models disruptive: A comprehensive survey. arXiv:2305.18703

[43] Liu H, Lu M, Al-Hussein M, (2016). Ontology-based semantic approach for construction-oriented quantity take-off from BIM models in the light-frame building industry. Advanced Engineering Informatics, 30( 2): 190–207

[44] Liu WZhou PZhao ZWang ZJu QDeng HWang P (2020). K-bert: Enabling language representation with knowledge graph. In: Proceedings of the AAAI Conference on Artificial Intelligence, 34(03), 2901–2908

[45] Lu Y, Li Y, Skibniewski M, Wu Z, Wang R, Le Y, (2015). Information and communication technology applications in architecture, engineering, and construction prganizations: A 15-year review. Journal of Management Engineering, 31( 1): A4014010

[46] Lünig J N, Seiß S, Schwerdtner P, Melzner J, (2025). Reducing construction quality costs through ontology-based inspection planning. Advanced Engineering Informatics, 68: 103650

[47] Luo HSpecia L (2024). From Understanding to Utilization: A survey on explainability for large language models. arXiv:2401.12874

[48] Martínez-Rojas M, Marín N, Vila M A, (2016). The role of information technologies to address data handling in construction project management. Journal of Computing in Civil Engineering, 30( 4): 04015064

[49] McCoy R TPavlick ELinzen T (2019). Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference. arXiv:1902.01007

[50] Mikolov TChen KCorrado GDean J (2013a). Efficient estimation of word representations in vector space. arXiv:1301.3781

[51] Mikolov T, Sutskever I, Chen K, Corrado G S, Dean J, (2013b). Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 26: 3111–3119

[52] Mishra A, Jain S K, (2016). A survey on question answering systems with classification. Journal of King Saud University - Computer and Information Sciences, 28( 3): 345–361

[53] Newell AShaw J CSimon H A (1959). Report on a general problem solving program. In: IFIP Congress, 256, 64

[54] Niu J, Issa R R A, (2015). Developing taxonomy for the domain ontology of construction contractual semantics: A case study on the AIA A201 document. Advanced Engineering Informatics, 29( 3): 472–482

[55] OpenAI JAchiam SAdler SAgarwal LAhmad IAkkaya F LAleman DAlmeida JAltenschmidt SAltman SAnadkat RAvila IBabuschkin SBalaji VBalcom PBaltescu HBao MBavarian JBelgum … , et al. (2024). GPT-4 technical report. arXiv:2303.08774

[56] Pan S, Luo L, Wang Y, Chen C, Wang J, Wu X, (2024). Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering, 36( 7): 3580–3599

[57] Pan S J, Yang Q, (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22( 10): 1345–1359

[58] Pennington JSocher RManning C D (2014). Glove: Global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1532–1543

[59] Petroni FRocktäschel TRiedel SLewis PBakhtin AWu YMiller A (2019). Language models as knowledge bases? arXiv:1909.01066

[60] Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I, (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1( 8): 9

[61] Ren X, Deng X, Liang L, (2018). Knowledge transfer between projects within project-based organizations: The project nature perspective. Journal of Knowledge Management, 22( 5): 1082–1103

[62] Roberts ARaffel CShazeer N (2020). How much knowledge can you pack ito the prameters of a language model? arXiv:2002.08910

[63] Shaw C, De Andrade Pereira F, De Riet M, Hoare C, Farghaly K, O’Donnell J, (2025). Knowledge graph for policy- and practice-aligned life cycle analysis and reporting. Automation in Construction, 176: 106282

[64] Shen TMao YHe PLong GTrischler AChen W (2020). Exploiting structured knowledge in text via graph-guided representation learning. arXiv:2004.14224

[65] Soman K, Rose P W, Morris J H, Akbas R E, Smith B, Peetoom B, Villouta-Reyes C, Cerono G, Shi Y, Rizk-Jackson A, Israni S, Nelson C A, Huang S, Baranzini S E, (2024). Biomedical knowledge graph-optimized prompt generation for large language models. Bioinformatics, 40( 9): btae560

[66] Speiser K, Seiß S, Boukamp F, Melzner J, Teizer J, (2025). From fragmented data to unified construction safety knowledge: A process-based ontology framework for safer work. Automation in Construction, 176: 106293

[67] Sun JXu CTang LWang SLin CGong YNi L MShum H YGuo J (2024). Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph. arXiv:2307.07697

[68] Sun YWang SFeng SDing SPang CShang JLiu JChen XZhao YLu YLiu WWu ZGong WLiang JShang ZSun PLiu WOuyang XYu D… Wang H (2021). ERNIE 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation. arXiv:2107.02137

[69] Tian D, Li M, Ren Q, Zhang X, Han S, Shen Y, (2023). Intelligent question answering method for construction safety hazard knowledge based on deep semantic mining. Automation in Construction, 145: 104670

[70] Uschold M, Grüninger M, (1996). Ontologies: Principles, methods and applications. Knowledge Engineering Review, 11( 2): 93–136

[71] Wang CLiu XYue YTang XZhang TJiayang CYao YGao WHu XQi ZWang YYang LWang JXie XZhang ZZhang Y (2023). Survey on factuality in large language models: Knowledge, retrieval and domain-specificity. arXiv:2310.07521

[72] Wang Q, Mao Z, Wang B, Guo L, (2017). Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 29( 12): 2724–2743

[73] Wen YWang ZSun J (2023). MindMap: Knowledge graph prompting sparks graph of thoughts in large language models. arXiv: 2308.09729

[74] Wen Y, Zhu X, Zhang L, (2022). CQACD: A concept question-answering system for intelligent tutoring using a domain ontology with rich semantics. IEEE Access: Practical Innovations, Open Solutions, 10: 67247–67261

[75] Xu J, Bu J, Li J, (2024). A knowledge transfer framework based on deep-reinforcement learning for multistage construction projects. IEEE Transactions on Engineering Management, 71: 11361–11374

[76] Xu J, He M, Jiang Y, (2022). A novel framework of knowledge transfer system for construction projects based on knowledge graph and transfer learning. Expert Systems with Applications, 199: 116964

[77] Yang L, Chen H, Li Z, Ding X, Wu X, (2024). Give us the facts: Enhancing large language models with knowledge graphs for fact-aware language modeling. IEEE Transactions on Knowledge and Data Engineering, 36( 7): 3091–3110

[78] Yang Y, Xiang P, Wang D, (2025). Knowledge graph and mitigation measures recommendation for safety hazards in large-scale hydropower projects using diverse heterogeneous inspection data. Automation in Construction, 178: 106419

[79] Ye HLiu TZhang AHua WJia W (2023). Cognitive Mirage: A review of hallucinations in large language models. arXiv:2309.06794

[80] Ye JWang YHuang YChen DZhang QMoniz NGao TGeyer WHuang CChen P YChawla N VZhang X (2024). Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge. arXiv:2410.02736

[81] Yu Z, Gong Y, (2024). ChatGPT, AI-generated content, and engineering management. Frontiers of Engineering Management, 11( 1): 159–166

[82] Zhang D, Ma G, Qu T, Wang X, Zhou W, Wang X, (2025). A knowledge graph-enhanced large language model for question answering of hydraulic structure safety management. Advanced Engineering Informatics, 66: 103468

[83] Zhang S, Boukamp F, Teizer J, (2015). Ontology-based semantic modeling of construction safety knowledge: Towards automated safety planning for job hazard analysis (JHA). Automation in Construction, 52: 29–41

[84] Zhang YLi YCui LCai DLiu LFu THuang XZhao EZhang YChen YWang LLuu A TBi WShi FShi S (2023). Siren’s Song in the AI ocean: A survey on Hallucination in large language models. arXiv:2309.01219

[85] Zhao H, Chen H, Yang F, Liu N, Deng H, Cai H, Wang S, Yin D, Du M, (2024). Explainability for Large Language Models: A Survey. ACM Transactions on Intelligent Systems and Technology, 15( 2): 1–38

[86] Zhao W XZhou KLi JTang TWang XHou YMin YZhang BZhang JDong ZDu YYang CChen YChen ZJiang JRen RLi YTang XLiu ZLiu PNie JWen J (2023). A survey of large language models. arXiv:2303.18223

[87] Zheng CWong SSu XTang Y (2023a). A knowledge representation approach for construction contract knowledge modeling. arXiv: 2309.12132

[88] Zheng C, Wong S, Su X, Tang Y, Nawaz A, Kassem M, (2025). Automating construction contract review using knowledge graph-enhanced large language models. Automation in Construction, 175: 106179

[89] Zheng L, Chiang W L, Sheng Y, Zhuang S, Wu Z, Zhuang Y, Lin Z, Li Z, Li D, Xing E P, Zhang H, Gonzalez J E, Stoica I, (2023b). Judging LLM-as-a-Judge with MT-bench and Chatbot Arena. Advances in Neural Information Processing Systems, 36: 46595–46623

[90] Zhong B, He W, Huang Z, Love P E, Tang J, Luo H, (2020). A building regulation question answering system: A deep learning methodology. Advanced Engineering Informatics, 46: 101195

[91] Zhou S, Liu K, Li D, Fu C, Ning Y, Ji W, Liu X, Xiao B, Wei R, (2025). Augmenting general-purpose large-language models with domain-specific multimodal knowledge graph for question-answering in construction project management. Advanced Engineering Informatics, 65: 103142

[92] Zhu FLei WWang CZheng JPoria SChua T S (2021). Retrieving and reading: A comprehensive survey on open-domain question answering. arXiv:2101.00774

RIGHTS & PERMISSIONS

Higher Education Press