Optimising LoRA for fine-tuning lightweight LLM expert agents in cross-regional collaboration in the construction industry: Evidence from the Greater Bay Area

Abstract

Under the “one country, two systems, three legal jurisdictions” framework, cross-regional collaboration in the construction industry of the Guangdong-Hong Kong-Macao Greater Bay Area (GBA) is more complex than other urban agglomerations, and this makes traditional methods relying on manual analysis of influencing factors inefficient and subjective. While existing large language models (LLMs) can meet the needs of intelligent applications, they lack specific domain knowledge. Considering the intelligent advantages of LLMs, this research proposes a lightweight expert agent through providing an optimised low-rank adaptation (LoRA) model SVDSR-LoRA, integrating domain knowledge using the fine-tuning method. Experiments on the 1.5b lightweight base model of qwen2.5 and deepseek-r1 show that the proposed SVDSR-LoRA training method can increase the mid-term convergence speed by 36%-50% compared with the standard LoRA method. A lightweight multi-expert agent influencing factor analysis system is constructed to simulate a collaborative analysis environment with experiments showing the hit rate in high-frequency influencing factors reached 100%. The comprehensive evaluation scored at 4.03/5.0 and demonstrated the capability in revealing the external environmental factors influencing cross-regional collaboration in the construction industry of the GBA systematically, which was significantly better than that of a single model (2.52-3.22/5.0). The proposed SVDSR-LoRA training model and the MAS system establishing pattern can provide a reference for rapid and effective lightweight agent system construction methods for intelligent analysis tasks of cross-regional collaboration in the construction industry of the GBA, as well as other similar LLM-based cross-regional collaboration research

Keywords

Guangdong-Hong Kong-Macao Greater Bay Area (GBA) / cross-regional collaboration / construction industry / large language models (LLM) / lightweight expert agent

RIGHTS & PERMISSIONS

Higher Education Press 2026

Just Accepted

This article has successfully passed peer review and final editorial review, and will soon enter typesetting, proofreading and other publishing processes. The currently displayed version is the accepted final manuscript. The officially published version will be updated with format, DOI and citation information upon launch. We recommend that you pay attention to subsequent journal notifications and preferentially cite the officially published version. Thank you for your support and cooperation.