How to optimize large language model performance for human-centric post-occupancy evaluation of underground public space: From model selection to model tuning

Abstract

Underground public space (UPS) is vital for three-dimensional urban development, and its post-occupancy evaluation (POE) forms a basis for human-centric planning, design and management. Although large language models (LLMs) provide powerful tools for POEs based on social media data (SMD), LLM application sare often constrained by lacking domain-specific knowledge. This study developed a framework to optimize LLM performance for SMD-based POEs. Model selection, parameter setting, prompt tuning and fine-tuning were examined for two specialized annotation tasks using 34 UPS cases and 14 LLMs. The methods proved satisfactory effects, achieving peak macro accuracy of approximately 0.95 and F1-scores over 0.83 for the two tasks. Reasoning models outperformed general-purpose models by up to 25.74% (F1-scores) in complex tasks, though with lower consistency. Prompt tuning was highly effective but task-dependent with a peak increment of 21.99% in macro F1-scores. Parameter settings and system role modifying had minimal effects. Excessively long prompts may reduce performance. Fine-tuning significantly improved performance and reduced prompt dependency. POEs identified spatial form, wayfinding and operational management as high-priority factors for future design and renovation of UPS. The findings offered a practical and cost-effective optimization pathway for applying LLMs to other similar textual analysis in underground space studies.

Graphical abstract

Keywords

UPS / POE / LLM / prompt tuning / SMD

1 Introduction

As cities grow denser, the strategic development of three-dimensional public space systems has become integral to sustainable urban growth [1,2]. Underground public space (UPS) represents a vital component of this vertical urban expansion, supplementing surficial urban fabric by accommodating essential functions such as transportation, commerce and social gathering, thereby optimizing land use efficiency [3–5]. After years of use, early-built UPS has experienced space degradation, confronted with problems such as poor internal environments, insufficient public facilities, outdated functions, and inefficient operation or management. The evolving public cognition of urban underground space (UUS) also emphasizes the importance of human-centered design of UPS [6,7], further highlighting the necessity of post-occupancy evaluations (POEs) before the construction and renovation.

POE is a systematic process to assess how well a built environment performs in meeting user needs [8]. For UPS, POEs serve as effective approaches to identify user satisfaction and unveil critical factors affecting it, providing essential feedback for human-centric planning, design and management improvements [9]. Structured questionnaires, interviews and physiological sensors are classic methods for underground space POEs [10–12]. However, these approaches are often constrained by high costs, limited spatiotemporal coverage and labor-intensive implementation [13,14]. As reactive researches, they may also introduce subjective biases [15]. In the era of big data, scholars are turning to novel digital traces to understand interactions between people and space. Social media data emerges as a powerful and non-intrusive source for capturing spontaneous public perceptions and emotional responses. Although it offers a dynamic and extensive dataset for human-centric POEs, complementing traditional approaches, limited studies explored its potential in UUS studies.

The recent rise of large language models (LLMs) offers a transformative tool for decoding vast amount of social media texts, promising a deeper understanding of public opinions [14,16–18]. Compared to traditional natural language processing techniques (e.g., Bidirectional Encoder Representations from Transformers (BERT), machine learning, hand coding), LLMs generally require less or no training data, and they are more flexible and easier to adapt to a wide range of tasks [19]. However, a significant gap exists between the general capabilities of LLMs and the specialized needs of UUS fields. Owing to a lack of domain-specific knowledge, LLMs often fall short of expert-level performance in specialized tasks [15]. Moreover, the high cost of developing or fully fine-tuning LLMs makes them inaccessible to many research teams [20]. Therefore, a critical and practical question arises: How can researchers effectively and affordably optimize LLMs for high-quality and domain-specific POEs of UPS based on social media data (SMD)? Currently, there is limited evidence on the best strategies for this optimization, from choosing the right model to tuning it for the tasks.

To address this gap, this study developed a practical framework for optimizing LLM performance in human-centric POEs for UPS. 34 cases from four Chinese megacities with their social media reviews were selected for the case study. LLMs with different types, size and deployment methods, and three model tuning approaches, including parameter settings, prompt tuning and fine-tuning, were compared to reveal their optimization effects on model performance. Two core POE tasks were conducted using LLMs: 1) identifying detailed user perceptions for UPS; and 2) classifying the sentiments behind them. The following five questions were answered: 1) How do model types and size affect performance; 2) Do lower temperature and Top-p improve results; 3) What prompt templates balance performance and costs; 4) How much does model fine-tuning help; and 5) What key user insights do POEs reveal. The remainder of this paper is structured as follows. Section 2 reviews related studies on user perception-based POEs and LLM usage in scientific studies. Section 3 introduces the methodology. Section 4 presents the results along with the discussion in Section 5. Section 6 provides concluding remarks.

2 Literature review

2.1 Underground public space perceptions and human-centric post-occupancy evaluations

User perceptions constitute a dynamic cognitive process through which individuals interpret and evaluate building environments by integrating personal experiences with spatial information [6,21]. Traditional methods to collect user perceptions in UPS, such as questionnaires, semi-structured interviews, wearable and physiological sensors, are resource-intensive and geographically limited in scope [11,22–26]. Zhou et al. [27] proposed a machine learning method to assess comfort satisfaction using photos, which only provided visual perceptions. In contrast, SMD offers a new data source with large sample size, easy access, time efficiency and broad spatiotemporal coverage [28], it implies interactions between the public and space, capturing user behaviours, opinions and emotions in a non-reactive manner [29–31]. Prior studies have demonstrated its high efficiency for urban space POEs to form a basis for more effective space design and spatial planning [32,33].

Human-centric POEs aim to extract user needs and preferences across multi-dimensional factors such as space environment, design, functionality, atmosphere, facilities and management [9]. Based on mining user perceptions and their sentiments, POEs focus on quantifying user satisfaction and identifying key influential factors [9]. Prior studies have developed several frameworks for POEs. Fan and Cui [34] established a factor system based on Maslow’s needs theory. Wang et al. [10] categorized POE dimensions into space connectivity and positioning, physical environment, safety, convenience, facility, landscape, application of smart technology and environmental diversity. Zhao et al. [35] further constructed an assessment system integrating spatial forms, physiological environment, functional service and aesthetic experience. To quantify these factors, methods such as semantic differential analysis [36], Kano models [6], principal component analysis [9] and analytic hierarchy process [37] have been widely. Additionally, importance-performance analysis (IPA) was also used to identify critical factors by cross-referencing perceived importance with current factor performance [10,12].

Despite its potential, the use of SMD specifically for investigating user perceptions for UPS POEs remains underexplored. A significant gap exists in systematically extracting and quantifying user experiences embedded in unstructured textual data. Moreover, there is a lack of robust and efficient framework that can process large volumes of social media texts to perform the dual POE tasks of fine-grained perception factor identification and sentiment analysis.

2.2 Large language model usage and its performance optimization

LLMs are pre-trained models based on the Transformer architecture [38]. They learn contextual information from massive textual data and can generate human-like texts [14,21]. Despite challenges such as artificial intelligence hallucination and output inconsistency [39–41], LLMs’ strong capabilities have prompted researchers to explore their usage as alternatives to traditional natural language processing methods across various fields. In economic management and accounting fields, Niu et al. [42] and de Kok [19] employed LLMs to identify specific context in 10-K filings or earnings conference calls, acquiring accuracy and F1-scores ranging from 0.86 to 0.96 and 0.73 to 0.87, respectively. For urban studies, LLMs were adopted to monitor public responses for urban policies or events [29,30]. Luo et al. [21] adopted several LLMs to identify cultural ecosystem services from social media texts, reporting accuracy and F1-scores reaching up to 0.87 and 0.80. In underground space engineering, Gan et al. [43] and Yu et al. [44] introduced large multimodal models in tunnel defect detection and maintenance. Atangana Njock et al. [45] compared LLMs and DistilBERT to reveal their capabilities in risk assessments for tunnel structural failure.

However, LLM performance is constrained by its pre-training data and underlying algorithms [15], necessitating optimization for specific domain tasks. Törnberg [46] emphasized prompt tuning and parameter settings as two common optimizing methods despite without empirical analysis. Studies investigating specific prompt components such as system roles [47], number of examples or shots [48], demo positions [49] and background contexts [15,21] have reported inconsistent outcomes across tasks and models. For Application Programming Interface (API)-based models, complex prompts also lead to more tokens with higher costs. However, analysis of cost-effectiveness in prompt tuning is still limited. Due to the cost-prohibitive nature of LLMs, fine-tuning methods (e.g., Low-rank adaptation (LoRA), P-tuning, Adapter) and Retrieval-augmented Generation (RAG) have emerged as cost-effective alternatives to specialize LLMs for specific tasks with limited compute resources [50]. By training LLMs on domain-specific data, these methods can improve understanding and content alignment with specific requirements [50,51].

In summary, although LLMs show promise for automated textual analysis for underground space POEs, their application to mining user perceptions and sentiments in UPS is still in its early stages. Critical gaps persist in empirically verifying the impact of key optimization strategies, including parameter settings, prompt design and fine-tuning for domain-specific fields. Few studies have systematically compared the suitability of reasoning models versus general-purpose LLMs for such professional tasks. Meanwhile, the cost-effectiveness of optimizing LLM performance also needs further investigation.

3 Methodology

3.1 Study areas and research framework

Beijing, Shanghai, Guangzhou, and Shenzhen are four major cities known for their underground space development. With the construction of metro systems and urban regeneration, numerous UPS cases have emerged in downtown areas. This study selected 34 cases, including 18 underground streets, 14 underground malls and 2 UPS, as shown in Table 1. Functionally, underground streets are defined as pure public space, typically integrated with metro stations and supporting pedestrian walking, transportation and commercial activities. Underground malls, located beneath urban squares and green space, serve as semi-public space that remains accessible for over 12 h per day without access control, providing a mix of commercial and public services. Underground service space, also categorized as pure public space, is mainly for community activities and cultural displays.

Figure 1 illustrates the analytical framework. Social media texts were crawled to scrutinize the embedded user perceptions for UPS. After data processing, optimization methods were examined based on sample tests by conducting the following two tasks successively: 1) user perception mining; and 2) perception sentiment analysis. Finally, user perception patterns, UPS satisfaction and IPA results were analyzed for POEs.

3.2 Step 1: Data processing

SMD was collected from Dazhongdianping (Dianping), one of China’s leading third-party review platforms. It offers multimodal review data, including texts, photos and videos, with clear categories and a long-time span, making it a rich and reliable source for studying space perceptions and sentiment evaluations. Dai and Chen [52] demonstrated the strong correlation between Dianping reviews and on-site research findings. This study employed the website crawler to extract all available review texts and user information from 2006 to 2024 across 34 UPS cases. Retrieved data included usernames, gender, reviews, commenting time and ratings (0.5–5.0 stars). After deduplication and removal of blank information, the final dataset comprised 57429 complete reviews, with detailed descriptive statistics shown in Table C1 of Appendix C in Electronic Supplementary Material.

3.3 Step 2: Optimizing large language model performance

3.3.1 Annotation tasks

Beijing, Shanghai, Guangzhou and Shenzhen represent distinct Chinese dialect regions, where linguistic variations may influence written expression in user reviews. A test sample of 600 Dianping reviews was constructed by randomly selecting 150 review comments from the dataset of each city, with an average text length of 110 characters.

Two annotation tasks of varying complexity were designed. Task 1 was user perception mining, requiring LLMs to identify user perception factors and their corresponding text segments within each review. It was a complex domain-specific task with customized labels and professional terms in UUS studies. Based on literature reviews, 28 perception factors are defined across six dimensions as listed in Table 2, including environment, design, function, atmosphere, facility and management. If no pre-defined factor was detected, the model was instructed to output ‘Others’. Table C2 of Appendix C in Electronic Supplementary Material proposes detailed factor definitions and references. Task 2 focused on annotating the sentimental polarity (positive, neutral, or negative) of each identified perception factor, which was a more universal and simpler task. Sentence-level sentiment analysis was applied to factor-annotated review segments to enhance granularity [48].

3.3.2 Optimizing methods

In this study, strategies of model selections, parameter settings, prompt tuning and model fine-tuning were adopted to optimize LLM performance. For Task 1, we examined the effects of model types, parameter settings and five prompt templates. For Task 2, we assessed prompt tuning and model fine-tuning across different LLM types and model size.

1) Model selection

Totally 14 widely used models from five leading LLM developers (four Chinese and one American) were selected in this study. Model descriptions can be checked in Table C3 of Appendix C in Electronic Supplementary Material. Based on reasoning capabilities, LLMs can be categorized into general-purpose models and reasoning models. General-purpose models are designed for conversation and multimodal input processing, while reasoning models are optimized for logic reasoning tasks [53]. Performance of five reasoning LLMs (Deepseek-r1, Ernie-x1-turbo-32k, Qwq-plus, o4-mini and Gemini-2.5-flash) and three general-purpose models (Deepseek-v3, Ernie-4.5-turbo-128k and Qwen-plus) for the two tasks was compared to reveal which LLM type could acquire better performance. Furthermore, it was supposed that the model size also determined capabilities at the expense of cost and speed [19]. To ensure comparability, the six Qwen2.5 models (0.5b, 1b, 1.5b, 7b, 14b, 32b) were tested for Task 2 to unveil effects of model size on annotation performance. Additionally, LLM deployment methods included API calls and local deployment. API-based models were accessible through a paid API, and the costs were computed based on a combination of input and output tokens, with substantial variation in per-token rates among different models. Locally-deployed models relied on local Graphics Processing Unit (GPU) resources, which can be fully controlled and fine-tuned by local users with high data security. Except for the six Qwen2.5 models, the other models were all used via APIs in this study. Total costs of API calls and computer resource investment for locally-deployed models were also compared to provide insights for deployment strategies.

2) Parameter setting

Temperature and Top-p regulate output diversity in text generation. Temperature adjusts output probability distribution, while Top-p selects words whose cumulative probabilities reach a threshold p [54]. Lower values typically yield more deterministic outputs, which were often preferable for annotation tasks [55]. To further evaluate effects of parameter settings for the two tasks, two parameter series were applied under consistent prompts and default settings for Task 1: 1) Top-p fixed at 0.8 with temperature values of 0, 0.1, 0.3, 0.5, 0.7, and 0.9; and 2) temperature fixed at 0.7 with Top-p values of 0.2, 0.4, 0.6, 0.8, and 1.0.

3) Prompt tuning

Standard annotation prompts include contexts, questions, definitions, few-shots and constraints [21,55,56]. For domain-specific tasks, it is essential to balance input complexity with cost efficiency. Five prompt templates were designed in Chinese for both tasks to examine how different prompt formats affected performance and cost-effectiveness, and determined the sensitivity of prompt tuning to varying difficulty tasks. Prompt templates and their Chinese character counts are provided in Table 3. P1 served as the baseline with a default and zero-shot context. P2 introduced an improved system role to examine its influence. P3 to P5 incorporated simple or detailed examples to assess their effects. All prompt components were consistently structured, with demos in prominent positions [49]. Detailed prompts are provided in Appendices A and B in Electronic Supplementary Material.

4) Model fine-tuning

Fine-tuning adapts pre-trained models to specialized annotation tasks [57]. As LLM size increases, available computational resources and training data make full fine-tuning less feasible [20]. LoRA offers a low-resource but efficient approach by freezing pre-trained weights and introducing trainable low-rank matrices (A and B), enabling task-specific specialization while preserving general capabilities [20,51]. To assess how much fine-tuning can boost annotation performance and whether larger models yield better fine-tuning effects, six Qwen2.5 series LLMs of activation-aware weight quantization versions were fine-tuned using LoRA for Task 2. Two NVIDIA GeForce RTX 4090 were deployed for the examination.

3.3.3 Performance validation

Three domain experts in UUS fields manually annotated the data to establish a gold standard for validation. Annotation performance for each perception factor or sentimental polarity was evaluated using accuracy and F1-scores (Eqs. (1)–(4)) [58], both of which ranged from 0 to 1. The values closer to 1 indicated better performance. For comprehensive evaluations of the multi-label tasks, macro accuracy (arithmetic mean across all labels) and macro F1-scores (Eq. (5)) were applied [59].

(1)\text{Accuracy}=\frac{TP+TN}{TP+FP+TN+FN},

(2)\text{F1}=2\times \frac{Precision\times Recall}{Precision+Recall},

(3)Precision=\frac{TP}{TP+FP},

(4)Recall=\frac{TP}{TP+FN},

where TP and TN denote the number of true positives and true negatives, respectively; FP and FN denote the number of false positives and false negatives, respectively.

(5)\text{Macro F1}=2\times \frac{{Precision}_{\text{macro}}\times {Recall}_{\text{macro}}}{{Precision}_{\text{macro}}+{Recall}_{\text{macro}}},

where Precision_macro and Recall_macro denote the arithmetic mean of Precision and Recall of all the annotated labels.

Furthermore, a comprehensive indicator was built to measure the cost-effectiveness of prompt tuning as the following Eq. (6). A higher rate indicated a more efficient tuned prompt.

(6)R_{\text{efficiency}}=\frac{r_{\text{performance}}}{r_{\text{tokens}}},

where R_efficiency denotes the efficiency rate for prompt tuning; r_performance and r_tokens denote the growth rates of the performance indicator and tokens for the tested prompt template compared to the baseline prompt (P1), respectively.

3.4 Step 3: Case study for post-occupancy evaluation

Using the optimized LLM methods obtained from Step 2, user perceptions and sentimental polarity were extracted from all collected review texts. POEs included UPS comprehensive satisfaction and IPA. Perception sentiments were quantified by assigning values of 1, 0.5, and 0 to positive, neutral and negative labels, respectively. The final sentiments for each factor were calculated as the average of all corresponding sentimental scores within a complete review. Perception weights were calculated by normalizing relative frequencies of factors for each UPS. Final satisfaction values were computed as the product of factor weights and their corresponding sentimental scores. Furthermore, a classic IPA model was used to identify key perception factors for improvement. This strategic management tool measured attribute importance and performance [60], which has been widely adopted in urban studies [10,12,61]. The sentimental scores and relative frequencies of perception factors were set as the x- and y-coordinate to divide the matrix into four quadrants, including keeping up the good work, concentrating here, low priority, and possible overkill [60].

4 Results

4.1 Optimizing effects

4.1.1 Parameter setting

As some of the selected models did not support parameter settings, only Deepseek-v3, Ernie-4.5-turbo-128k, Qwq-plus and Qwen-plus were used to test parameter adjustment impacts on Task 1. The P3 prompt template was used throughout the analysis. Figure 2 shows the annotation performance under different combinations of temperature and Top-p. Overall, the influence of the parameters on LLM performance was limited. Coefficients of variation (CV) ranged from 0.09% to 0.40% in macro accuracy and from 0.85% to1.61% in macro F1-scores among different parameter combinations. Contrary to the conclusions by Törnberg [55], lower values of temperature or Top-p did not consistently yield higher accuracy or F1-scores. Deepseek-v3 exhibited fluctuating performance gains with increasing parameters, peaking in accuracy at temperature 0.7 and Top-p 0.8, and in F1-score at temperature 0.7 and Top-p 1.0. In contrast, Qwq-plus displayed a nearly opposite trend. For the other two models, temperature and Top-p showed divergent effects on performance. These results indicated that while parameter settings did influence annotation outcomes, no universal patterns of performance changes existed. Model capabilities had stronger effects on performance than parameter adjustments.

4.1.2 Prompt tuning

Except for models not applicable to parameter adjustment, model temperature and Top-p were uniformly set at 0.7 and 0.8 for Tasks 1 and 2, respectively. Figure 3 demonstrates the impacts of prompt tuning on LLM performance for Task 1. Deepseek-r1 with P3 achieved the highest macro accuracy (0.955) and F1-score of 0.955 and 0.833, respectively, indicating satisfactory performance. Comparing P1 and P2, improving system roles had limited and model-dependent effects. Growth rates for macro accuracy and F1-score ranged from −0.87% to 1.57% and −0.95% to 10.28%, respectively, averaging 0.22% and 3.19%. Assigning a professional persona did not consistently enhance performance in objective annotation tasks and could even cause adverse effects such as Deepseek-r1, Deepseek-v3, Ernie-x1-turbo-32k and Ernie-4.5-turbo-128. On the other hand, growth rates of macro accuracy and F1-scores for P3 to P5 achieved −1.78% to 4.29% and −15.14% to 21.99%, respectively, averaging 1.86% and 7.89%. Except for Qwen-plus, clarifying task definitions or providing examples effectively strengthened LLMs’ domain-specific comprehension, consistent with the conclusions by Luo et al. [21]. Their effects were robustly higher than adjusting system roles. Furthermore, label definitions contributed more remarkably to performance than few-shots, likely because they directly supplemented domain-specific knowledge that was absent in pre-training corpora. In contrast, providing only one example per label might be insufficient for models to grasp intrinsic label meanings, leading to performance degradation in P4. Notably, excessive prompting did not guarantee further improvement and could introduce adverse effects, as observed with Deepseek-r1, Ernie-x1-turbo-32k, Qwq-plus and Gemini-2.5-flash under P5.

For comparability, sample reviews were consistently split into review segments using Deepseek-r1 with default parameters and P3 to conduct Task 2. The 600 complete reviews were segmented into 2882 comment clips, including 1893 positive, 497 neutral and 492 negative segments based on manual annotation. Figure 4 presents the tuning results of the four prompts. Compared to the baseline, growth rates of macro accuracy and F1-scores reached −1.79% to 4.24% and −3.79% to 3.96%, respectively, which were smaller than the results of Task 1. It indicated that prompt tuning exhibited higher sensitivity in improving performance of more specialized and complex tasks. Among the eight tested models, Qwq-plus (P4) demonstrated the strongest adaptability for three-class sentimental polarity annotation, achieving macro accuracy of 0.939 and an F1-score of 0.880. For most models, few-shot learning outperformed the strategy of providing detailed label definitions, different from the results of Task 1. This divergence could be attributed to the more universal nature of sentiment analysis, which relied less on domain-specific knowledge and more on general linguistic patterns in pre-training data. Additionally, sentimental polarity was a more abstract concept and its definition might be imprecise to affect the understanding of LLMs. The phenomenon of excessive prompting, especially imprecise definitions, interfering with model performance was also observed in Task 2.

Figure 5 shows the efficiency rates of prompt tuning for the two tasks. Although P2 obtained lower absolute performance gains, it achieved the highest efficiency to improve the annotation results. Furthermore, despite higher tokens, P3 still demonstrated the strongest efficiency in enhancing both performance metrics among the three prompts for Task 1. In contrast, P5 offered the lowest cost-effectiveness. In Task 2, P4 exhibited higher improvement efficiency, whereas that of P3 was considerably lower than the other two prompts.

4.1.3 Low-rank adaptation fine-tuning

For fine-tuning, a separate set of 5200 comment segments, including 3135 positive, 1107 neutral and 958 negative clips, was manually annotated and used to apply LoRA to the six Qwen2.5 models with size from 0.5 to 32 billion. Figure 6 compares the performance of original and fine-tuned models on the 2882 samples. For most of the models and prompts, model fine-tuning significantly enhanced its performance. The growth rates for accuracy and F1-scores ranged from −0.99% to 62.33% and −19.27% to 36.48%, respectively, with average rates of 12.04% and 8.77%. Excluding the abnormally low baseline performance of the Qwen2.5-3b, which led to an exceptionally high growth rate after fine-tuning, performance improvements generally showed a trend of first decreasing and then rebounding as model size increased. The models with larger size achieved better performance. The fine-tuned Qwen2.5-32b obtained the highest macro-accuracy of 0.946 (under P2) and the best F1-score of 0.884 (under P1). Additionally, fine-tuning reduced performance’s reliance on prompt design. P1 achieved the best performance for most of the fine-tuned models, compared to the fact that P4 or P5 were better for the original models. However, model fine-tuning was only applicable to locally deployed open-source models and required substantial local computer resources, a large amount of high-quality sample data, and higher technical expertise. Therefore, prompt tuning should be prioritized over model fine-tuning to achieve the desired performance targets with smaller costs.

4.1.4 Model selection

Figure 7 compares the performance of reasoning models and general-purpose models. Compared to general-purpose models, the reasoning models showed an increase in average macro accuracy and F1-scores by 1.10%–5.43% and 2.52%–25.74% across two tasks under different prompts, respectively. It demonstrated reasoning LLMs’ strength in domain-specific or logic-intensive tasks. In Task 1, the accuracy gap between reasoning and general-purpose models was smaller than in Task 2. This may be due to class imbalance in Task 1, resulting in an overestimate of accuracy. The lower inherent difficulty of Task 2 might have compressed the performance gap between the two model types. In contrast, macro-F1 scores more reliably captured their applicability for tasks of varying difficulty and specialization. Reasoning models showed a greater advantage in complex tasks, with macro F1-score gaps of 5.92% to 25.74% in Task 1 compared to 2.52% to 4.48% in Task 2 against general-purpose models. The results were also consistent with the findings of Shojaee et al. [62]. Furthermore, according to Figs. 4(i), 4(j), 5(i), and 5(j), reasoning models showed higher sensitivity to prompt tuning in Task 1, but lower in Task 2. The findings suggested that detailed prompts better stimulated deep reasoning capabilities of reasoning models, leading to superior performance in complex and professional tasks. For simple and general tasks, elaborate prompts effectively compensated for the weaker inherent reasoning of general-purpose models, optimizing their output. However, for reasoning models that already possessed chain-of-thought mechanisms, prompt tuning offered smaller marginal gains. This further indicated that task difficulty was also a factor in model selections.

Figures 6(a) and 6(b) plot the effects of model size of Qwen2.5 on annotation performance for Task 2. Regardless of the prompt templates, the original Qwen2.5-3b exhibited consistently and abnormally low performance, which may be attributed to its inherent characteristics. For the other models, model performance improved with larger model size, though with diminishing marginal gains. Qwen2.5-14b achieved the highest accuracy of 0.914 (under P5) and best F1-score of 0.846 (under P4), which was only slightly lower than Qwq-plus, and surpassed Deepseek-r1 with a size of 671 B across various prompts. Impacts of LLM size on task performance were not linear. Larger models did not always achieve superior results. It was feasible to employ smaller models for domain-specific analysis by balancing computer resources and performance.

To determine appropriate model deployment strategies, API fees and local computer resource costs were compared. For demo performance verification, the total cost of API calls reached approximately CNY 2500. If API-based Deepseek-r1 was used for full-sample analysis of both tasks, the added cost would be estimated at around CNY 1000. Although the API expenses in this study were significantly lower than the cost of GPU configuration, which was around CNY 40000, scaling to millions of samples with more expensive models could result in extremely high costs and low output speed. Therefore, model deployment strategies should be determined based on data confidentiality requirements, response speed, data volume and researchers’ available computing resources and funding.

4.2 Post-occupancy evaluation case study

4.2.1 Perception factors and sentiments

Based on the results in Subsection 4.1, Deepseek-r1 under P3 was used to annotate perception factors. From 57429 social media reviews, a total of 234993 perception factors were identified, with an additional 5442 labels classified as ‘Others’. Detailed information for each UPS can be checked in Fig. D1 of Appendix D in Electronic Supplementary Material Factors related to UPS functions, design and management were most frequently mentioned, accounting for 35.2%, 22.1%, and 18.0% of all perceived factors, respectively. In contrast, users put less emphasis on facilities, atmosphere and environmental quality, which comprised 4.9%, 7.9%, and 11.9% of the total mentions.

As shown in Fig. 8, the relative frequency distributions of perception factors are highly consistent across the four cities, with Pearson correlation coefficients ranging between 0.819 and 0.981. Given that 79.2% of the reviews were contributed by local residents, these results suggested a shared cognitive framework for evaluating UPS, despite regional variations in culture and lifestyle. However, Shenzhen exhibited more distinct perception patterns, with notably higher mentions of connectivity (A22) and catering (A32), and fewer references to landscape (A12), parking (A56) and other public facilities (A57). These differences may be attributed to Shenzhen’s status as a newly developed immigrant city, characterized by a younger demographic and faster-paced lifestyle, which heightened residents’ emphasis on commuting efficiency and basic functional fulfillment.

Figure 9 further reveals distinct perception patterns across UPS types. Underground streets and malls showed similar factor distributions, both diverging significantly from underground service space. Compared with the former two types, underground service space induced more attention to landscape (A12), acoustics (A14), leisure (A35) and operation (A62), while receiving fewer mentions of retail (A31), catering (A32) and commerce formats (A64). Cases of Digua Community (B6, an underground community center) and Tower Shadow Space (S4, an underground cultural gallery) stimulated stronger user expectations for social and recreational quality, thereby elevating concerns over landscape aesthetics, acoustic comfort and operational standards. Although both underground streets and service space were perceived as public in nature, users did not significantly distinguish between underground streets and malls. This cognitive ambiguity was likely due to the common integration of commercial functions into underground streets, which was a strategy to offset construction costs. It made them functionally similar to underground malls from the perspective of the public. Nevertheless, underground streets still received more frequent mentions of connectivity (A22), accessibility (A23) and commuting (A34), reflecting a user profile dominated by commuters and transitional visitors, in contrast to the shopping-oriented patterns typical of underground malls.

The fine-tuned Qwen2.5-32b was selected to analyze sentiments of perception factors. Sentimental scores for each factor across UPS cases are proposed in Fig. D2 of Appendix D in Electronic Supplementary Material. As summarized in Fig. 10, 67.3 % of the sentiments were positive, surpassing neutral (17.0%) and negative (15.7%) reviews, with an average sentimental score of 0.673 across all factors. Users expressed pronounced dissatisfaction with certain public facilities and management-related factors, whereas aspects linked to space environments, design and functions received comparatively higher ratings.

4.2.2 Underground public space comprehensive satisfaction

Figure 11 shows the assessment results of comprehensive satisfaction for each UPS. Among the 34 cases, Pacific Fresh City Underground Street (S5), City Hub Underground Street (Z4), and PO PARK (G2) performed best according to user perception, while CITIC Metro Square (Z8), Huasheng Underground Street (S9) and D-Mall (S1) achieved the lowest scores. A Pearson correlation between comprehensive scores and average Dianping ratings from user reviews indicated a significantly positive relationship. It demonstrated that the multidimensional perception factors and their emotional scores effectively characterized the users’ complex and integrated subjective satisfaction with the space.

4.2.3 Importance-performance analysis results

Figure 12 presents the quadrant distribution of perception factors across the four cities, with detailed case-level IPA results provided in Fig. D3 of Appendix D in Electronic Supplementary Material. The high similarity in IPA outcomes suggested consistent perception patterns and space satisfaction expectations among users from different cultural and regional backgrounds within China. Critically, factors such as spatial form and wayfinding (A21), atmosphere perception (A41) and operation (A62) were located in the second quadrant, characterized by high frequency of mention but low sentimental scores, implying higher priority for targeted improvement. Comparatively, factors including landscape (A12), connectivity (A22), accessibility (A23), retail (A31), catering (A32), culture and entertainment (A33), leisure (A35) and commerce formats (A64) were widely recognized and positively evaluated, reflecting the importance of refined interior design, strong spatial connectivity and appropriate commercial positioning in shaping successful UPS environments.

As illustrated in Fig. 13, analysis on UPS types revealed distinct perceptual profiles. Although underground streets and malls exhibited similar factor distributions, reflecting their shared commerce-dominated functions, connectivity (A22) emerged as more critical factors for underground streets. It highlighted the need for seamless integration with metro systems and adjacent underground networks. In contrast, underground service space showed unique challenges in accessibility (A23) and spatial scale (A24), pointing to overlooked demands for community-oriented access and adequate spatial dimensions in non-commercial UPS planning and design.

5 Discussion

5.1 Generalization assessments

To evaluate the generalization capability of LLMs, we examined their annotation performance on review texts from Beijing, Shanghai, Guangzhou and Shenzhen, representing regions dominated by Mandarin, Wu dialect, Cantonese and mixed dialects, respectively. As shown in Fig. 14, under consistent prompt settings, CV of macro accuracy reached 0.23%–2.87% and 0.52%–2.84% for Tasks 1 and 2, respectively, and macro F1-scores reached 1.42%–11.41% and 0.46%–2.60%. Since each city had only 150 complete reviews while there were 28 perception factors, it was highly potential to creating unbalanced sample problems across different cities, resulting in significant variations in F1-scores. However, the other CV in the two tasks were all lower than 3%. Additionally, non-native Chinese-oriented LLMs (o4-mini, Gemini-2.5-flash) exhibited competitive performance in Chinese text annotation. The findings confirmed the robustness and generalization capacity of LLMs in handling geographically and linguistically diverse urban texts.

5.2 Consistency assessments

Consistency of LLM outputs were gradually focused by scholars [40,41]. To assess annotation consistency, three rounds of independent tests for Task 1 were conducted using eight selected LLMs and five prompt templates. Notably, all results in Subsection 4.1 corresponded to the first of these three repeated tests. Consistency was measured using three indicators: 1) CV of macro accuracy and F1-scores across three runs; 2) proportions of entirely consistent labels and entirely inconsistent labels; and 3) Jaccard similarity among the three sets of extracted perception factors. Figure 15 presents some contradictory findings. From a global perspective, the average CV for macro accuracy and F1-score were 0.13% and 1.35% across all models and prompts, respectively. Except for Gemini-2.5-flash, the other models under majority of prompts exhibited extremely tiny fluctuations over three trials. However, Jaccard similarity and consistency proportions revealed non-negligible instability in LLM-based annotations from the perspective of individual outputs. The proportion of entirely consistent labels ranged from 9.0% to 65.7%, while entirely inconsistent labels accounted for 4.2% to 65.0% across tests. Notably, reasoning models, despite outperforming general-purpose models, showed lower consistency, with an average inconsistent rate of 45.2%, compared to 17.5% for general-purpose models. Jaccard similarity followed a similar trend, with an average of 0.599 for reasoning models and 0.775 for general-purpose models.

The results demonstrated that LLMs actually simulated intelligent behavior through probabilistic prediction rather than actual understanding. This mechanism clarified the coexistence of high and consistent overall performance and low consistency in individual outputs. Furthermore, reasoning models, incorporating chain-of-thought mechanisms, excelled at complex tasks but generated diverse reasoning paths and potential over-reasoning, leading to different final answers [53]. These findings reflected that model selections should not rely solely on performance indicators. Output consistency was also important for specific tasks. Although temperature adjustment had limited impacts on annotation performance, it may affect the consistency theoretically. To improve consistency while preserving accuracy, repeated sampling followed by averaging or majority voting was feasible [41]. Due to the black-box nature of LLMs, the inconsistent outputs limited their application in scientific research. For tasks with high consistency requirements, traditional natural language processing methods were more stable and lower-risk than LLMs.

5.3 Framework for large language model using

Although this study optimized LLM performance for UPS POEs, the framework and findings can be applied to other textual analysis in UUS fields. These include text classification, information retrieval, and content comparison. Figure 16 presents a basic workflow for using LLMs in such tasks. The process begins by deconstructing the analysis problem into specific and well-defined tasks, followed by a decision on whether to use an LLM. According to de Kok [19], LLMs are suitable when high-accuracy training data is scarce, manual annotation is infeasible, tasks require reasoning and strong contextual understanding, clear rules are difficult to define, and the team has sufficient budgets. A key step is to conduct iterative testing, using demo validations that compare LLM outputs against manual annotations, to select the optimal model and tuning method. Notably, performance targets vary across tasks. Besides traditional indicators such as accuracy and F1-scores, evaluating output consistency through multiple trials is recommended. Furthermore, model selection should follow a minimum-sufficient principle to avoid unnecessary costs, as larger and higher-performing models typically incur greater API expenses or GPU requirements. The model upgrade path should prioritize size first, then series, and finally type. Prompt tuning should always be attempted before fine-tuning. It is advisable to start prompt tuning with a zero-shot prompt as a baseline, then test whether adding detailed definitions or few-shot examples significantly improves performance while monitoring the increase in token usage. Full-sample analysis should proceed only after the demo test meets the targeted performance criteria.

5.4 Limitations and future work

This study optimized LLM-based methods to POEs of UPS from a user perception perspective. Some limitations remain for future researches.

1) This study only tested a limited range of hyperparameter (temperature, Top-p) and did not explore their effects on output consistency. Furthermore, the impact of example quantity in few-shot prompts was not systematically analyzed. Whether increasing examples can effectively boost model performance, similar to the findings by Zhang et al. [48], and even exceed the effects of definitions, remains to be studied.

2) The study mainly focused on LoRA for fine-tuning and did not evaluate other efficient methods (e.g., P-tuning, Adapter). The potential of RAG to provide contextual knowledge was also not explored. Future research should compare various tuning strategies and integrate RAG to assess improvements in accuracy and domain adaptation.

3) The complexity difference between the two core tasks was not quantitatively measured. This limited precise interpretation of model performance gaps. Future work should adopt quantifiable task difficulty metrics and expand the framework to other UUS analysis domains to verify its broader utility.

4) The generalization assessment did not strictly follow the principle of controlled variables. Theoretically, reviews identical in content but written in different dialects should have been used for each city. Nevertheless, our study still reflected the models’ good generalization abilities. Observed performance differences across cities more likely represented normal random variations.

5) The analysis relied solely on text reviews from a single platform (Dianping), which may not fully represent diverse user groups or cross-platform perceptions. The inherent demographic and self-reporting biases of social media data further limit generalizability (e.g., age, gender, a tendency toward extreme expressions) [54]. Future studies should incorporate multi-source and multimodal data (e.g., images, videos), and apply cross-platform validation to build more robust POE frameworks.

6 Conclusions

POEs provide a critical evidence basis for human-centered UPS design and renovation. Rapid development of LLMs offers a novel method to conduct human-centric POEs by extracting user perceptions from social media data. This study developed and tested a systematic framework for optimizing LLM performance in two core tasks of multi-label perception factor classification (Task 1) and three-category sentiment analysis (Task 2). Through a comparative analysis of 14 popular LLMs applied to 34 UPS cases across four major Chinese cities, the effects of key optimization strategies were evaluated, including model selections, parameter settings, prompt engineering and LoRA fine-tuning. The main findings are summarized as follows.

1) The results demonstrated that universal LLMs could deliver satisfactory performance on UUS tasks, with maximal macro accuracy approximately 0.95 and F1-scores above 0.83. Reasoning models showed performance gains of 1.10%–5.43% in macro accuracy and 2.52%–25.74% in F1-scores over general-purpose models, demonstrating a consistent advantage, particularly in complex tasks. However, they showed lower output consistency across repeated trials. For simpler and more general tasks, general-purpose models by prompt tuning could match the performance of reasoning models. Model performance increased with size but with diminishing marginal returns, indicating that the largest model was not always necessary. Model selection should balance task complexity, peak performance, output consistency, data confidentiality, response speed, data volume, and available computational resources and budget.

2) Adjustments to temperature and Top-p parameters showed limited and inconsistent effects on performance. The observed CV ranged from 0.09% to 0.40% for macro accuracy and from 0.85% to 1.61% for macro F1-score across different combinations. Default configurations were generally acceptable. Optimization efforts should focus on more decisive methods such as prompt tuning and model selection, though lower parameter values might potentially improve output consistency.

3) Effects of prompt tuning depended strongly on task nature. For professional and complex tasks (Task 1), providing clear definitions and label descriptions increased macro accuracy and F1-scores by up to 4.29% and 21.99%, respectively, compared to the zero-shot baseline. Models were also highly sensitive to changes in these prompts. For more universal and simpler tasks (Task 2), few-shot examples proved more effective by improving performance up to 4.24% and 3.96% in macro accuracy and F1-scores. Altering system roles produced unstable outcomes, though it achieved the highest efficiency for improving performance while increasing tokens. Moreover, excessively long prompts could even degrade performance.

4) Applying LoRA fine-tuning with a small annotated dataset significantly enhanced performance and reduced model dependence on prompt design. A fine-tuned medium-sized general-purpose model (e.g., Qwen2.5-32b) could even surpass a much larger non-fine-tuned model (e.g., Qwq-plus, Deepseek-r1), achieving a good balance between computer resources and effectiveness. Given the resource intensity and technical expertise required for fine-tuning, prompt tuning should be prioritized as a first step toward meeting performance targets.

5) The POEs revealed that user perceptions of UPS were most influenced by space functions, design and management, while factors related to public facilities, atmosphere and environmental quality received less attention. Perceptions were consistent across different cities but varied significantly among UPS types, reflecting their distinct functional roles. IPA results highlighted spatial forms, wayfinding, atmosphere perceptions and operational management as high-priority improvement across all cities. Specific UPS types presented unique optimization needs. Connectivity was critical for underground streets, while accessibility and spatial scales were key concerns for underground service space.

References

[1] Peng F L , Qiao Y K , Sabri S , Atazadeh B , Rajabifard A . A collaborative approach for urban underground space development toward sustainable development goals: Critical dimensions and future directions. Frontiers of Structural and Civil Engineering, 2021, 15(1): 20–45

[2] Ma C X , Peng F L , Qiao Y K , Li H . Evaluation of spatial performance of metro-led urban underground public space: A case study in Shanghai. Tunnelling and Underground Space Technology, 2022, 124: 104484

[3] Jia X M , Wang J Y , Yan B . Evaluation of existing underground public space supply based on the node-place-experience model. Tunnelling and Underground Space Technology, 2024, 144: 105517

[4] Hong X C , Li S C , Sun W H , Ji X . Assessment and spatial variation analysis of the collaborative development potential between underground public spaces and urban rail transit in China’s urban central areas. Tunnelling and Underground Space Technology, 2025, 158: 106415

[5] Peng F L , Wang W X , Qiao Y K , Ma C X , Dong Y H . Review on data-informed planning for underground space. Underground Space, 2026, 26: 257–281

[6] Ma J X , Shen Z W , Liang P P , Pan L Y. . Spatial element perception measurement and optimization of urban underground complex based on Kano model: A case study of Chengdu SKP. New Architecture, 2024, (5): 80–86

[7] Zhou X H , Shen S L . Assessment of living quality in Guangdong: A hybrid knowledge-based and data-driven approach. Ecological Informatics, 2024, 82: 102745

[8] Makki M , Mathers J , Matthews L , Melsom J , Biloria N , Raymond B , Cheung J , Ricafort K . A data-driven multi-objective approach to improving the quality of the urban space: The character of Sydney’s public domain. Cities, 2025, 167: 105369

[9] Zeng R S , Shen Z W , Luo J . Post-occupancy evaluation of user satisfaction: Case studies of 10 urban underground complexes in China. Tunnelling and Underground Space Technology, 2024, 143: 105500

[10] Wang X , Shen L Z , Shi S Y . Evaluation of underground space perception: A user-perspective investigation. Tunnelling and Underground Space Technology, 2023, 131: 104822

[11] Pang X X , Jia M Q , Zhang Y . Campus landscape perception decoded by surveys combined with eye tracking: Visual and cultural perception for user well-being. Building and Environment, 2025, 283: 113341

[12] Zhang Y B , Xie Z Q , Zhao X Q , Pu J W , Wang Y N , Xiong B . UPUCS: Spatial evaluation of user-perceived comfort in underground commercial space. Tunnelling and Underground Space Technology, 2025, 155: 106169

[13] Shen C , Li P . Green housing on social media in China: A text mining analysis. Building and Environment, 2023, 237: 110338

[14] Xu R X , Sun Y F , Ren M J , Guo S G , Pan R T , Lin H Y , Sun L , Han X P . AI for social science and social science of AI: A survey. Information Processing & Management, 2024, 61(3): 103665

[15] Chen L Y , Darko A , Zhang F , Chan A P C , Yang Q . Can large language models replace human experts? Effectiveness and limitations in building energy retrofit challenges assessment. Building and Environment, 2025, 276: 112891

[16] Alizadeh M , Kubli M , Samei Z , Dehghani S , Zahedivafa M , Bermeo J D , Korobeynikova M , Gilardi F . Open-source LLMs for text annotation: A practical guide for model setting and fine-tuning. Journal of Computational Social Science, 2025, 8(1): 17

[17] Li J , Gao J , Zhang Z H , Fu J , Shao G F , Zhao Z Y , Yang P P . Insights into citizens’ experiences of cultural ecosystem services in urban green spaces based on social media analytics. Landscape and Urban Planning, 2024, 244: 104999

[18] Raiaan M A K , Mukta M S H , Fatema K , Fahad N M , Sakib S , Mim M M J , Ahmad J , Ali M E , Azam S . A review on large language models: Architectures, applications, taxonomies, open issues and challenges. IEEE Access, 2024, 12: 26839–26874

[19] de Kok T . ChatGPT for textual analysis? How to use generative LLMs in accounting research. Management Science, 2025, 71(9): 7888–7906

[20] Pratap S , Aranha A R , Kumar D , Malhotra G , Iyer A P N , Shylaja S S . The fine art of fine-tuning: A structured review of advanced LLM fine-tuning techniques. Natural Language Processing Journal, 2025, 11: 100144

[21] Luo H Y , Zhang Z D , Zhu Q , Houda Ben Ameur N E , Liu X , Ding F , Cai Y L . Using large language models to investigate cultural ecosystem services perceptions: A few-shot and prompt method. Landscape and Urban Planning, 2025, 258: 105323

[22] Sun Y , Shao Y W . Measuring visitor satisfaction toward peri-urban green and open spaces based on social media data. Urban Forestry & Urban Greening, 2020, 53: 126709

[23] Zacharias J , Wang B . Willingness to walk in underground space-evidence from Singapore. IOP Conference Series: Earth and Environmental Science, 2021, 703(1): 012002

[24] Chan I Y S , Dong Z , Chen H . Impacts of connections to the outside on underground space occupants’ psychophysiological health: A virtual reality-based experimental approach. Tunnelling and Underground Space Technology, 2024, 147: 105675

[25] Li L Y , Hu S H , Dai Y P , Deng M , Momeni P , Laverghetta G , Fan L Z , Ma Z H , Wang X , Ma S Y . et al. Toward satisfactory public accessibility: A crowdsourcing approach through online reviews to inclusive urban design. Computers, Environment and Urban Systems, 2025, 122: 102329

[26] Lin S S , Shen S L , Zhou A N , Chen X S . Smart techniques promoting sustainability in construction engineering and management. Engineering, 2025, 45: 262–282

[27] Zhou B , Gui Y B , Xie X Y , Li W S , Li Q . A multi-category intelligent method for the evaluation of visual comfort in underground space. Tunnelling and Underground Space Technology, 2022, 124: 104488

[28] Kandt J , Batty M . Smart cities, big data and urban policy: Towards urban analytics for the long run. Cities, 2021, 109: 102992

[29] Lu S , Huang J Y , Wu J . Multi-dimensional urban flooding impact assessment leveraging social media data: A case study of the 2020 Guangzhou rainstorm. Water, 2023, 15(24): 4296

[30] Yang L Y , Duarte C M , Ciriquián P M . Quantifying the relationship between public sentiment and urban environment in Barcelona. Cities, 2022, 130: 103977

[31] Mehra P . Unexpected surprise: Emotion analysis and aspect based sentiment analysis (ABSA) of user generated comments to study behavioral intentions of tourists. Tourism Management Perspectives, 2023, 45: 101063

[32] Liu R X , Xiao J . Factors affecting users’ satisfaction with urban parks through online comments data: Evidence from Shenzhen, China. International Journal of Environmental Research and Public Health, 2021, 18(1): 253

[33] Zhang B , Song Y , Liu D Y , Zeng Z Z , Guo S Y , Yang Q Y , Wen Y H , Wang W J , Shen X W . Descriptive and network post-occupancy evaluation of the urban public space through social media: A case study of Bryant Park, NY. Land, 2023, 12(7): 1403

[34] Fan L , Cui X . Analysis of the factors influencing the comfort of different behaviors in underground commercial space: A case study of track connection to underground space. Iranian Journal of Science and Technology, Transactions of Civil Engineering, 2023, 47(4): 2485–2495

[35] Zhao X X , Guo D J , Chen Y L , Wu Y H , Zhu X P , Du C H , Chen Z L . Sustainable comfort design in underground shopping malls: A user-centric analysis of spatial features. Sustainability, 2025, 17(6): 2717

[36] Liu M , Zhang K , Zhu J H , Hao Z W. . Post-occupancy evaluation of underground urban link space based on semantic differential method: A case study of Shanghai Wujiaochang. New Architecture, 2019, (6): 15–20

[37] Wu Y T, Weng J. Research on the improvement of urban underground public space vitality based on post occupancy evaluation: A case study of underground public space of Linjiangmen in Chongqing. In: Proceedings of 2019 (14th) Urban Development and Planning Conference. Zhengzhou: Chinese Society for Urban Studies (CSUS), Zhengzhou Municipal People’s Government, Henan Provincial Department of Natural Resources, Henan Province Department of Housing and Urban Rural Construction, 2019, 2178–2184 (in Chinese)

[38] Dahmen J , Kayaalp M E , Ollivier M , Pareek A , Hirschmann M T , Karlsson J , Winkler P W . Artificial intelligence bot ChatGPT in medical research: The potential game changer as a double-edged sword. Knee Surgery, Sports Traumatology, Arthroscopy, 2023, 31(4): 1187–1189

[39] Reiss M V. Testing the reliability of ChatGPT for text annotation and classification: A cautionary remark. 2023, arXiv: 2304.11085

[40] Freire Y , Laorden A S , Pérez J O , Sánchez M G , García V D F , Suárez A . ChatGPT performance in prosthodontics: Assessment of accuracy and repeatability in answer generation. The Journal of Prosthetic Dentistry, 2024, 131(4): 659.e1–659.e6

[41] Lee NHong JThorne J. Evaluating the consistency of LLM evaluators. In: Proceedings of the 31st International Conference on Computational Linguistics. Abu Dhabi: Association for Computational Linguistics, 2025, 10650–10659

[42] Niu Y M , Wu J , Jiang S Y , Jiang Z B . The bullwhip effect in servitized manufacturers. Management Science, 2025, 71(1): 1–20

[43] Gan B L , Zhang D M , Huang Z K , Zheng F Y , Zhu R , Zhang W . Ontology-driven knowledge graph for decision-making in resilience enhancement of underground structures: Framework and application. Tunnelling and Underground Space Technology, 2025, 163: 106739

[44] Yu G , Ju R B , Sugumaran V , Liu H Z . Lightweight multimodal LLM-empowered dual-agent collaboration for reliable defect detection and maintenance recommendation in tunnel infrastructure. Reliability Engineering & System Safety, 2026, 268: 111973

[45] Atangana Njock P G , Yin Z Y , Xu H R , Zhang N . Structural failure risk assessment of shield tunnel using large language model. Tunnelling and Underground Space Technology, 2025, 165: 106882

[46] Törnberg P. ChatGPT-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning. 2023, arXiv: 2304.06588

[47] Zheng M QPei J XLogeswaran ML DLee Jurgens . When “a helpful assistant” is not really helpful: Personas in system prompts do not improve performances of large language models. In: Proceedings of the Association for Computational Linguistics: EMNLP 2024. Miami: Association for Computational Linguistics, 2024, 15126–15154

[48] Zhang W XDeng YLiu BPan SBing L D. Sentiment analysis in the era of large language models: A reality check. In: Proceedings of the Association for Computational Linguistics: NAACL 2024. Mexico City: Association for Computational Linguistics, 2024, 3881–3906

[49] Cobbina K A, Zhou T Y. Where to show demos in your prompt: A positional bias of in-context learning. In: Proceedings of 2025 Conference on Empirical Methods in Natural Language Processing. Suzhou: Association for Computational Linguistics, 2025, 29560–29593

[50] Xia Y C, Kim J, Chen Y H, Ye H J, Kundu S, Hao C, Talati N. Understanding the performance and estimating the cost of LLM fine-tuning. 2024, arXiv: 2408.04693

[51] Hu E JShen Y LWallis PZhu Z ALi Y ZWang S AWang LChen W Z. LoRA: Low-rank adaptation of large language models. In: Proceedings of the 10th International Conference on Learning Representations. OpenReview.net, 2022

[52] Dai D X , Chen Y X. . An evaluation of the perception of cultural services of urban historical park renovation based on sentiment computing method: A case study of Luxun Park.. Journal of Tongji University (Social Science Edition), 2022, 33(3): 81–90

[53] Wang X ZWei JSchuurmans DLe Q VChi E HNarang SChowdhery AZhou D. Self-consistency improves chain of thought reasoning in language models. In: Proceedings of the 11th International Conference on Learning Representations. Kigali: OpenReview.net, 2023

[54] Cai T LLi Y HGeng Z YPeng H WLee J DChen DDao T. Medusa: Simple LLM inference acceleration framework with multiple decoding heads. In: Proceedings of the 41st International Conference on Machine Learning. Vienna: PMLR, 2024, 5209–5235

[55] Törnberg P . Best practices for text annotation with large language models. Sociologica, 2024, 18(2): 67–85

[56] Yu D N , Li L Y , Su H , Fuoli M . Assessing the potential of LLM-assisted annotation for corpus-based pragmatics and discourse analysis. International Journal of Corpus Linguistics, 2024, 29(4): 534–561

[57] Gao D H , Ma Y F , Liu S , Song M F , Jin L B , Jiang W , Wang X , Ning W , Yu S Q , Xuan Q . et al. FashionGPT: LLM instruction fine-tuning with multiple LoRA-adapter fusion. Knowledge-based Systems, 2024, 299: 112043

[58] Zhang J Y , Luo Y , Cao H J , Su S L . Scrutinizing the cultural ecosystem services of Chinese classical gardens: A novel deep learning approach based on online reviews from a multisensory perspective. Applied Geography, 2024, 172: 103404

[59] Sokolova M , Lapalme G . A systematic analysis of performance measures for classification tasks. Information Processing & Management, 2009, 45(4): 427–437

[60] Martilla J A , James J C . Importance-performance analysis. Journal of Marketing, 1977, 41(1): 77–79

[61] Gai S J , Fu J M , Rong X , Dai L L . Importance-performance analysis and improvement of an urban park’s cultural ecosystem services based on users’ perspectives: A Beijing case study. Journal of Asian Architecture and Building Engineering, 2023, 22(2): 726–739

[62] Shojaee PMirzadeh IAlizadeh-Vahid KHorton MBengio SFarajtabar M. The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity. In: Proceedings of the 39th International Conference on Neural Information Processing Systems. San Diago, CA: Curran Associates Inc., 2025

RIGHTS & PERMISSIONS

The Author(s). This article is published with open access at link.springer.com and journal.hep.com.cn