Study on the Influence of Emotion and Fatigue on Cognitive Function During Simulated Flight Based on ERP Technology

Abstract

Background:

Pilots often experience mental fatigue during task performance, accompanied by fluctuations in positive (e.g., joy) and negative (e.g., tension) emotions. Both mental fatigue and emotional changes significantly contribute to aviation accidents, yet few studies have considered their interplay. This study had three primary objectives. First, it examined the changes in positive and negative emotions following mental fatigue. Second, it investigated how these emotions influence the recovery from fatigue. Finally, it developed a comprehensive evaluation model integrating mental fatigue and emotional states.

Methods:

Two task sets were created using the visual search paradigm, incorporating simulated flight tasks with positive and negative emotional stimuli. Data were collected from 30 participants using electroencephalogram (EEG), eye-tracking, electrocardiogram (ECG), and behavioral performance metrics.

Results:

Participants showed mental fatigue after the simulated flight task, with reduced arousal for both positive and negative emotions; positive images had stronger effects. ERP showed decreased N1, P3, and LPP amplitudes. A Support Vector Machine (SVM) classifier achieved over 93% accuracy for fatigue but about 70% for emotion recognition.

Conclusions:

The task effectively induced fatigue and indicated that positive stimuli may aid recovery. Multimodal features support accurate fatigue detection, though emotion classification needs improvement

Clinical Trial Registration:

No: ChiCTR2500104961. https://www.chictr.org.cn/showproj.html?proj=267844.

Graphical abstract

Keywords

simulated flight / mental fatigue / emotional arousal / SVM

1. Introduction

Pilots face dual challenges of mental fatigue and emotional fluctuations during mission execution [1, 2, 3]. Mental fatigue is one of the most critical factors leading to a decline in pilot performance, posing a significant threat to aviation safety [1, 4, 5]. It manifests across various dimensions, including cognitive, psychological, emotional, and behavioral [6]. Psychologically, fatigue appears as tiredness, headaches, and palpitations, while behaviorally, it is characterized by distractibility and drowsiness [7, 8, 9]. Emotionally, it often leads to anxiety, reduced self-confidence, and irritability [10]. When pilots experience mental fatigue, their levels of alertness and attention drop significantly [11, 12]. For example, 72% of Army pilots flying under sleep-deprived conditions are at risk of falling asleep during flight [13]. A survey of 162 experienced U.S. Air Force pilots and navigators revealed that fatigue-induced performance degradation is common across cockpits, often resulting in reduced situational awareness and delayed reaction times [14].

Meanwhile, emotions are also closely related to aviation safety [15, 16]. Positive emotions, such as happiness and alertness, correlate with optimal performance, whereas negative emotions, such as anxiety, often emerge during suboptimal performance [17]. Emotionally demanding tasks, particularly those involving complex working memory, have been shown to impair executive functions under emotional stress [18]. During prolonged cognitive tasks, individuals often report increased fatigue and negative emotions while experiencing a marked decline in attention and positive emotions [19]. In simulated unsafe landing scenarios, negative emotional outcomes associated with go-around decisions have been found to temporarily impair decision-making processes, increasing aviation risks [20]. Beyond flight tasks, pilots’ emotional changes driven by social pressures also impact flight safety. Prolonged exposure to emotionally stressful work environments can lead to negative workplace behaviors and diminished performance [21]. In a simulated flight experiment where task difficulty was a secondary factor, it was found that social stressors and cognitive workload manipulation influenced emotional responses. Specifically, social pressure sources triggered emotional reactions that enhanced secondary task engagement and performance [22].

Fatigue and emotional states in pilots are typically assessed through induced scenarios in simulated or real-flight environments, followed by analyzing performance and physiological data. Measurement tools include subjective questionnaires, physiological monitoring, and performance evaluations [23]. Among these, physiological metrics are particularly valued for their objectivity and sensitivity. Methods include collecting and analyzing electroencephalogram (EEG), electrocardiogram (ECG), eye-tracking, electromyogram (EMG), and blood oxygen saturation data. In EEG-based measurements, event-related potential (ERP) is commonly used to assess fatigue and emotional responses. Key ERP components include N170, P300, and late positive potential (LPP). The N170 component, peaking between 130–200 ms, is primarily associated with emotional processing and is most prominent in the right hemisphere [24]. The P300 component peaks around 300–600 ms during emotional picture tasks, with maximal activity in the central-parietal region, reflecting both emotional and non-emotional processes [25]. The LPP component, starting around 400 ms, is more sensitive to arousal levels and emotional regulation, persisting throughout stimulus presentation [26]. For mental fatigue, researchers focus on ERP components such as P1, N1, P2, N2, and P300, as changes in their latency and amplitude can reflect alterations in attention post-fatigue [27]. EEG frequency bands are also significant, with beta activity positively correlated with alertness and alpha/theta activity negatively correlated with it, serving as indicators of mental fatigue [28]. Similarly, heart rate (HR) and heart rate variability (HRV) metrics are commonly used as a physiological marker of autonomic nervous system function, particularly reflecting the balance between the parasympathetic and sympathetic branches. The high-frequency component of HRV is associated with parasympathetic activity, while the low-frequency component is considered a marker of both sympathetic and parasympathetic influences [29]. Time-on-task effects on HRV are linked to a decrease in parasympathetic activity, with time-domain indices providing more reliable monitoring of autonomic changes during mental fatigue induction. In contrast, frequency-domain indices are more closely related to psychological symptoms of mental fatigue [30]. Mental fatigue may involve two opposing processes: task disengagement and compensatory effort, both of which are influenced by the sympathetic and parasympathetic nervous systems. Therefore, a significant study has explored the relationship between HRV and mental fatigue [31]. Meanwhile, eye-tracking features like blink frequency and pupil diameter are widely used for fatigue detection [32, 33].

Recent studies leverage machine learning to classify fatigue and emotional states with high accuracy. For instance, Chen et al. [34] used graph neural networks to classify emotional states in simulated driving scenarios with 75.26% accuracy. Lee et al. [35] applied a CNN-based multi-feature block algorithm to classify pilots’ mental states (normal, fatigued, workload-induced, and distracted) with high precision. Yu et al. [36] combined eye-tracking and EEG data to build a convolutional neural network-long short-term memory model, accurately identifying pilots’ fatigue levels. Similarly, Qin et al. [37] achieved 91.8% accuracy in mental fatigue detection using ECG and eye-tracking data.

Despite these advancements, the interaction between fatigue and emotions remains poorly understood. Specifically, the reciprocal effects of fatigue on emotional arousal and how positive or negative emotions either amplify or mitigate fatigue are unclear. For instance, Ghanbari et al. [38] used emotional imagery as a stimulus to evaluate brain arousal post-fatigue but found no significant results. Conversely, Fan et al. [39] demonstrated that mindfulness meditation could counteract the negative associations between fatigue and emotional neural activation by maintaining LPP amplitudes during emotional stimuli.

In this study, experiments were designed to induce both positive and negative emotions as well as mental fatigue in simulated flight tasks, with the scenarios based on a visual search paradigm. The data collected during the experiments included EEG, ECG, eye-tracking metrics, work performance, and subjective questionnaires. The study aimed to address three key objectives: (1) Does the simulated flight task successfully induce mental fatigue, and what physiological indicators exhibit significant changes? (2) How do positive and negative emotions change after mental fatigue, and what is the arousal effect of these emotions on mitigating mental fatigue? (3) Considering the coupling between mental fatigue and emotional changes during flight tasks, is it possible to classify these two states effectively?

2. Materials and Methods

2.1 Participants

To minimize the influence of factors such as educational background and age, the study recruited 30 healthy male volunteers, all right-handed, aged 22 to 26, from the postgraduate student population of Beihang University. Before the experiment, participants were instructed to maintain adequate sleep and a normal diet for two days and to abstain from consuming stimulants such as strong tea or coffee for 24 hours. The sample size was close to the optimal sample size calculated in G*Power for ANOVA, f = 0.25, a = 0.05, 1-beta = 0.80, Number of groups = 1, Number of measurements = 4, resulting in N = 24.

During the testing sessions, participants were required to stay focused and minimize actions like speaking, swallowing, chewing, or making unnecessary movements. Upon completing the experiment, volunteers were compensated for their participation. This study was performed according to the Declaration of Helsinki and was approved by the Biological and Medical Ethics Committee of Beihang University (approval: BM20250004). All adult participants provided written informed consent to participate in this study.

2.2 Stimulus Materials

The experiment employed two paradigms: an emotional image recognition task (emotional experimental paradigm) and a visual search task (visual search paradigm). For the emotional image recognition task, 120 pictures (40 positive, 40 negative, and 40 neutral) were selected from the Chinese Affective Picture System (CAPS) as emotional stimuli. The CAPS database demonstrates high internal consistency (Cronbach’s \alpha> 0.85) and strong test-retest reliability (r > 0.75) [40].

For the visual search task, a total of 40 dashboard images were used as stimuli, including 20 target images and 20 non-target images. Each image was arranged in a 3 \times 3 grid, featuring nine dials with pointers oriented in various directions. The target stimuli were defined as dials with pointers fixed at a 45° angle (northeast direction), while the distractor stimuli consisted of dials pointing in the remaining seven directions (0°, 90°, 135°, 180°, 225°, 270°, and 315°).

2.3 Experimental Design

The experiment comprised two sub-experiments: a positive emotion condition and a negative emotion condition, both of which were completed by each participant. A 20-minute break was provided between the two sub-experiments to allow participants to recover. Each sub-experiment included three emotional task blocks and five visual search task blocks. The visual search tasks were designed to simulate cockpit instrument monitoring by pilots, while the emotional image tasks intended to elicit either positive or negative emotional states.

In the positive emotion condition, participants first completed a positive emotional image task, followed by five consecutive visual search tasks, and concluded with another positive emotional image task. The negative emotion condition adopted an identical structure. The visual search tasks positioned between the two emotional tasks were collectively referred to as fatigue-inducing tasks, aimed at eliciting mental fatigue in participants. Additionally, participants were required to complete a visual analogue scale (VAS) before and after the experiment to subjectively assess their mental state. The VAS included four dimensions: alertness, anxiety, vitality, and sleepy.

In the positive emotional task, participants were presented with a random sequence of images drawn from two categories: positive and neutral, each comprising 50% of the stimuli. Each image was displayed for 1000 ms, followed by an inter-stimulus interval of 1000 ms. The total task duration was 160 s. The negative emotional task followed the same design, but featured negative and neutral images instead. All parameters (image count, duration, and interval) were consistent across both conditions. Participants were instructed to click the left mouse button when viewing a positive or negative image and the right mouse button when viewing a neutral image. Each emotional block lasted approximately 160 s. Behavioral performance in each block was assessed via reaction time (RT) and accuracy (Acc). The total duration of the experiment was approximately 3 hours. The overall experimental procedure is illustrated in Fig. 1.

2.4 Experimental Equipment

In this experiment, EEG, ECG, eye-tracking, and behavioral data were collected from participants. The EEG signals were recorded using the NeuroScan 40 system (Compumedics NeuroScan, Charlotte, NC, USA) with a sampling rate of 1000 Hz. ECG data were acquired using the MP150 Data Acquisition System (BIOPAC system, Inc., Goleta, CA, USA), also with a sampling rate of 1000 Hz. Eye-tracking data were collected using the desktop eye-tracker (SensoMotoric Instruments, Germany). The specific experimental setup is shown in Fig. 2.

2.5 Data Processing

The data preprocessing of EEG signals was analyzed using the EEGLAB toolbox (https://sccn.ucsd.edu/eeglab/index.php) in the MATLAB 2023b (MathWorks, Inc., Torrance, CA, USA) environment. Four electrooculogram (EOG) channels (‘HEOL’, ‘HEOR’, ‘VEOU’, ‘VEOL’) were excluded from acquisition due to irrelevance to the analysis objectives. Additionally, electrodes ‘FT9’, ‘FT10’, ‘PO1’, and ‘PO2’ were removed post-acquisition due to poor signal quality, resulting in 32 retained channels. All preprocessing steps were performed using the EEGLAB toolbox running under MATLAB 2023b. Initially, EEG datasets were re-referenced to the average of the mastoid electrodes, ‘A1’ and ‘A2’, to establish a common reference. Subsequently, the continuous EEG data were filtered using a zero-phase Butterworth bandpass filter with cutoff frequencies of 1 Hz and 40 Hz to remove slow drifts and high-frequency noise. A notch filter centered at 50 Hz was also applied to eliminate line noise contamination. Following filtering, the continuous data were segmented into 1.2 s epochs (–0.2 to 1.0 s relative to stimulus onset), with the pre-stimulus interval (–0.2–0 s) used for baseline correction via mean voltage subtractions. To address ocular and other non-brain artifacts, independent component analysis (ICA) was employed. An experienced researcher visually inspected the ICA decomposition results, and artifactual components (typically 1–5 components per EEG dataset) were manually identified and removed.

An ERP analysis was performed on the EEG data collected from the frontal (FZ), central (CZ), and parietal (PZ) electrodes during emotional tasks for both the positive and negative groups. These locations overlie key cortical areas, including the prefrontal cortex, central parietal regions, and parieto-occipital cortex. Consequently, they are considered crucial recording sites in ERP studies for examining broadly distributed cognitive processes such as attention, inhibitory control, and working memory [39, 41, 42]. FZ: Positioned over the prefrontal cortex, primarily associated with emotional regulation and task control. CZ: Located over the central cortex, involved in sensory processing. PZ: Situated over the parietal-occipital region, playing a crucial role in visual processing. Specific time windows for key ERP components were defined to analyze neural activity during the tasks, including the N1 (100–150 ms), P2 (150–250 ms), N2 (200–350 ms), P3 (300–500 ms), and LPP (500–1000 ms). These intervals were carefully selected to capture neural responses critical for attention allocation, cognitive processing, and sustained emotional evaluation, aligning with the expected timing of these ERP components in the context of emotional and fatigue-related tasks.

Wavelet packet energy analysis was employed to evaluate EEG data collected before and after the fatigue tasks. For each experimental condition, we extracted the relative energy features from all channels. Specifically, after performing wavelet decomposition to extract the delta (\delta, 0.5–4 Hz), theta (\theta, 4–8 Hz), alpha (\alpha, 8–12 Hz), and beta (\beta, 12–30 Hz), the relative power ratios were calculated by dividing each energy by the sum of the four energy bands.

ECG signals were processed with the NeuroKit2 toolbox (https://neuropsychology.github.io/NeuroKit) in Python 3.9.20 (Python Software Foundation, Wilmington, DE, USA), involving preprocessing that included applying a high-pass filter at 0.5 Hz and notch filtering to eliminate 50 Hz power line interference. For feature extraction, a 60-second sliding window with a 30-second step size was used to derive HR and HRV. HRV time-domain metrics included mean normal-to-normal interval (NN), standard deviation of NN intervals (SDNN), root mean square of successive differences in NN intervals (RMSSD), percentage of NN intervals differing by more than 50 ms (pNN50), and triangular interpolation of NN interval distribution (TINN), while HRV frequency-domain metrics comprised total power (TP) and LF/HF (the low-frequency (LF, 0.04–0.15 Hz) to high-frequency (HF, 0.15–0.4 Hz) power ratio). The specific formulas for each indicator are as follows:

(1)\text{MeanNN}=\frac{1}{N}R{R}_i

(2)\mathrm{SDNN}=\sqrt{\frac{1}{N-1}\sum _{i=1}^N{\left(R{R}_i-\overline{RR}\right)}^2}

(3)\mathrm{RMSSD}=\sqrt{\frac{1}{N-1}\sum _{i=1}^N{\left(R{R}_{i+1}-R{R}_i\right)}^2}

(4)\text{pNN50}=\frac{\text{Number of} \left|R{R}_i-R{R}_{i-1}\right|>50\mathrm{m}\mathrm{s}}{N-1}\times 100\%

The eye movement data analysis was performed using the BeGaze software (SensoMotoric Instruments, Warthestraße, Teltow, Germany) included with the device for processing. The features collected from the eye movement data included pupil diameter, blink duration, blink frequency, saccade amplitude, and saccade velocity. In the positive emotion task, eye movement data from two emotion stimulus blocks (before and after the fatigue task) were used for analysis, with the data duration set at 160 s. The extracted feature data were averaged for each dimension across all participants using Python 3.9.

2.6 Statistical Analysis

To facilitate statistical analysis, we operationalized two categorical variables: “Positive Group” and “Negative Group”, which distinguished participants performing positive vs. negative emotion experimental conditions. Each emotional condition adopted a 2 \times 2 within-subject design, where participants’ physiological state during Block 1 (pre-fatigue) was defined as the “Awake State”, whereas Block 8 (post-fatigue) was designated as the “Fatigue State”. The variable “State” thus indexed the pre/post-fatigue contrast. Within each emotional block, distinct stimulus types were presented. The variable “Condition” was therefore defined to denote the valence of stimuli (Positive vs. Neutral or Negative vs. Neutral). All statistical analyses were performed using the Pingouin package (https://pingouin-stats.org/build/html/index.html) in Python 3.9.

For subjective scales, ECG features, and eye-tracking data, paired t-tests were used to compare the differences in each feature before and after the fatigue task. The significance level was set at p 0.05.

For ERP data, the image-evoked components N1, P2, N2, P3, and LPP were analyzed using a 2 \times 2 repeated-measures ANOVA with factors State (Awake vs. Fatigue) and Condition (Positive/Negative vs. Neutral). False discovery rate (FDR) correction was applied for multiple comparisons. For wavelet packet energy features, the relative power ratios of image-evoked delta \delta, \theta, \alpha, and \beta frequency bands were analyzed using a 2 \times 2 repeated-measures ANOVA with the same State and Condition factors. FDR correction was applied for multiple comparisons.

For behavioral data, reaction time was defined as the time elapsed between stimulus onset and correct keypress. Acc was defined as the proportion of correctly identified stimuli. RT and Acc data were subjected to the same 2 \times 2 repeated-measures ANOVA (State \times Condition), with FDR correction for pairwise comparisons.

3. Results

Eye movement, ECG, and subjective questionnaire data were used to verify that the designed simulated flight experiment successfully induced mental fatigue in the participants. Meanwhile, EEG and behavioral data simultaneously characterized the mutual influence between mental fatigue and emotion.

3.1 Mental Fatigue Results

3.1.1 Subjective Scales

As illustrated in Fig. 3, participants’ alertness significantly decreased after completing the task, while anxiety levels significantly increased. Similarly, vigor showed a significant decline, and sleepy markedly increased.

3.1.2 Eye Movement Feature Analysis

We analyzed several indicators, including pupil diameter, blink frequency, blink duration, saccade amplitude, and saccade velocity before and after the fatigue task, as shown in Table 1. After experiencing the fatigue task, participants exhibited a decrease in pupil diameter (Before: 5.09 \pm 0.83 mm; After: 4.51 \pm 0.81 mm; p = 0.0004), an increase in blink duration, and a relative reduction in blink frequency. While there was little change in the saccade velocity during target search, the saccade amplitude tended to decrease. However, no significant differences were observed in those indicators.

3.1.3 Analysis of ECG Signal Characteristics

Paired t-test results for before and after task data are presented in Table 2. After completing the fatigue task, HR significantly decreased (Before: 75.1 \pm 1.72 bpm; After: 72.58 \pm 3.22 bpm; p 0.0001) and multiple HRV metrics significantly increased including NN (Before: 759.45 \pm 97.33 ms vs. After: 793.08 \pm 126.11 ms; p = 0.018), SDNN (Before: 42.71 \pm 19.86 ms vs. After: 51.21 \pm 18.53 ms; p = 0.02), TP (Before: 869.16 \pm 736.99 vs. After: 1189.1 \pm 869.18; p = 0.03), TINN (Before: 103.61 \pm 30.5 ms vs. After: 130.62 \pm 39.8 ms; p 0.0001), and RMSSD (Before: 28.69 \pm 14.11 ms vs. After: 33.71 \pm 17.48 ms; p = 0.05), while pNN50 (Before: 9.84 \pm 11.09% vs. After: 14.04 \pm 16.34%; p = 0.14) and the LF/HF ratio (Before: 2.19 \pm 1.74 vs. After: 2.45 \pm 1.86; p = 0.46) showed no statistically significant changes before and after the task.

3.2 Mutual Influence Between Mental Fatigue and Emotion

3.2.1 ERP

The independent variables in our analysis were emotional type (positive vs. neutral, negative vs. neutral) and fatigue type (awake, fatigue), while the dependent variable was the amplitude of ERP components.

As shown in Fig. 4, emotional stimulus pictures elicited distinct N1, P2, N2, P3, and LPP components. The P2 component displayed a prominent large-amplitude peak around 200 ms, the P3 component showed high amplitude between 300–500 ms, the N1 component had a large amplitude around 150 ms, the N2 component exhibited high amplitude around 250 ms, and the LPP component demonstrated large amplitude with extended duration (600–800 ms).

In the positive group, as shown in Fig. 4a–c, after the fatigue task, both positive and neutral stimuli showed fatigue-related changes, including a reduction in amplitude for the N1, N2, P2, P3, and LPP components. The results of the two-way ANOVA for the positive group (Table 3) showed significant effects for the State factor.

After the fatigue task, the N1 component showed significantly reduced amplitudes at FZ (p 0.001), CZ (p 0.001), and PZ (p = 0.006), with no significant differences between positive and neutral image conditions across these electrodes. However, significant state \times condition interactions were observed at FZ (p 0.001) and CZ (p 0.001). For the P2 component, the main effect of state was significant only at PZ (p = 0.03), the main effect of condition was significant at PZ (p 0.001), and a significant interaction emerged at this electrode (p 0.001). In the N2 component, significant state effects were found at FZ (p 0.001), CZ (p 0.001), and PZ (p 0.001), with no significant condition effects, but significant interactions were present at FZ (p 0.001), CZ (p = 0.002), and PZ (p = 0.04). For the P3 component, state effects were significant at FZ (p 0.001) and CZ (p 0.001), condition effects were significant at FZ (p 0.001), CZ (p 0.001), and PZ (p 0.001), and interactions were significant at FZ (p = 0.002) and CZ (p = 0.02). Finally, for the LPP component, state effects were significant at FZ (p 0.001) and CZ (p 0.001), condition effects were highly significant across all electrodes (p 0.001), and interactions were significant at all electrodes (p 0.001).

In the negative group, as shown in Fig. 4d–f, negative stimuli reduced amplitudes across multiple ERP components compared to neutral stimuli, indicating mental fatigue following the fatigue task. After the fatigue task, differences between negative and neutral stimuli diminished across components. Moreover, fatigue had a more pronounced weakening effect on negative stimuli than on neutral stimuli. The results of the two-way ANOVA for the Negative Group (Table 4) showed significant effects for the State factor.

For the negative group, the N1 component showed significant state differences at FZ (p 0.001), CZ (p 0.001), and PZ (p 0.001), indicating reduced N1 amplitude in the central-frontal region after the fatigue task, while negative vs. neutral images differed significantly at FZ (p 0.001) and CZ (p 0.001) but not at PZ, with no significant interactions across electrodes. For the P2 component, only the PZ electrode showed a significant state difference (p = 0.01), and no interactions were detected. In the N2 component, significant state differences emerged at FZ (p = 0.002) and PZ (p = 0.002), negative images induced stronger N2 responses at FZ (p = 0.03) and marginally at CZ (p = 0.004), and a significant state \times condition interaction was observed at FZ (p = 0.03). For the P3 component, state differences were significant at FZ (p 0.001) and PZ (p = 0.002), negative images elicited stronger P3 responses at FZ (p = 0.002) and PZ (p 0.001), and no interactions were found. For the LPP component, a significant state difference occurred at FZ (p 0.001), condition effects were significant at all electrodes (p 0.001), and significant state \times condition interactions were observed across all electrodes (p 0.001).

When participants reach a state of mental fatigue, the arousal effects of positive and negative emotional stimuli differ. Assuming that neutral emotion stimuli images have the same impact on participants’ amplitude changes, we used the amplitude values of neutral emotion images from the visual search task in both groups as a baseline. We then compared the amplitude differences between positive, negative, and neutral emotion images, as shown in the Fig. 5.

In the Positive group, the amplitude difference is calculated by subtracting the amplitude induced by neutral emotion images from the amplitude induced by positive emotion images after mental fatigue. Similarly, in the Negative group, the amplitude difference is obtained by subtracting the amplitude induced by neutral emotion images from the amplitude induced by negative emotion images after mental fatigue.

As shown in Table 5, except for the N1 component, the amplitude changes induced by positive emotion images are larger than those induced by negative emotion images at the FZ, CZ, and PZ electrodes. After mental fatigue, participants seem to experience better fatigue awakening effects when exposed to positive emotion stimuli.

3.2.2 EEG Wavelet Energy Analysis

The wavelet energy distribution for the positive and negative groups in each frequency band is shown in Fig. 6a,b, respectively. After completing the visual search tasks, the relative energy in the \theta band decreased, while that in the \alpha band increased for both the positive and negative groups.

The two-way ANOVA results are presented in Table 6. In the positive emotion group, the \delta band showed no significant group, condition, or interaction effects. The \theta band exhibited a significant state effect, a significant condition effect (p 0.001), and a significant interaction effect (p = 0.006). The \alpha band showed no significant state effect (p = 0.346) but had a significant condition effect (p 0.001) and a significant interaction effect (p = 0.005). The \beta band had no significant effects for any factor.

In the negative emotion group, the \delta band showed a highly significant state effect (p = 0.006) and interaction effect (p 0.001), but no significant condition effect. The \theta band had no significant group effect but exhibited a highly significant condition effect (p 0.001) and a significant interaction effect (p = 0.006). The \alpha band showed no significant state effect but had significant condition (p = 0.002) and interaction effects (p = 0.007). The \beta band had a significant state effect (p = 0.001) and a significant interaction effect (p = 0.037), with no significant condition effect.

3.2.3 Behavioral Performance Analysis

Performance data on RT and Acc for the positive group are illustrated in Fig. 7a,b, respectively. After the visual search task, participants exhibited increased mean reaction times and decreased mean accuracy across all conditions, indicating signs of fatigue.

Two-way ANOVA results (Table 7) reveal that reaction times significantly increased following the visual search task, reflecting a clear decline in performance. Participants found it easier to distinguish between negative and neutral stimuli. However, no significant interaction effects were observed between image type and group for either the positive or negative groups. For accuracy, significant differences emerged only between positive and neutral image types. While a downward trend in accuracy was observed after the task, the decrease did not reach statistical significance. Furthermore, no significant interaction effects between image type and group were detected in the accuracy data. This suggests that while fatigue influenced reaction times, its impact on accuracy was less pronounced.

3.3 Emotion and Fatigue Classification

To better assess pilots’ emotional and fatigue states during simulated flight, we accounted for significant individual differences, as each participant’s classification thresholds for fatigue and emotion vary. We applied an Support Vector Machine (SVM) model to classify emotional and fatigue states using EEG, ECG, eye movement, and behavioral features. A radial basis function was used as the kernel for our classifier, and data normalization to the [0,1] range was performed using Z-scores. The dataset was split into training and testing sets with a 9:1 ratio, and the classification performance was evaluated using ten-fold cross-validation. In the fatigue classification task, labels are defined by inter-group conditions, with “before the visual search task” and “after the visual search task” serving as the two labels. In the emotion classification task, labels are defined by intra-group conditions, which are determined by the type of images presented. We assign label 1 to positive/negative images and label 0 to neutral images, to distinguish between positive/neutral or negative/neutral conditions.

We assessed the fatigue state and emotional image recognition in both the positive and negative groups during the emotional task module. The specific evaluation results are presented in the table. “Fatigue-po/ne” and “Fatigue-neutral” represent the classification of fatigue states when recognizing positive/negative and neutral images before and after the visual search task. “Emotion-pre” and “Emotion-after” denote the classification accuracy for distinguishing emotional images in each group.

Our evaluation model achieved excellent performance in classifying participants’ fatigue states, with accuracy exceeding 0.93. However, the model performed less effectively in distinguishing emotional states, achieving an accuracy of approximately 0.7. The specific results are shown in Table 8. Paired t-tests on emotion classification: p = 0.13 (positive group) and p = 0.69 (negative group) for positive/neutral images before/after fatigue task, no significant differences.

4. Discussion

The study investigated whether the simulated flight task successfully induced mental fatigue and examined changes in participants’ positive and negative emotional states following fatigue. It also explored the effects of fatigue on various physiological measures. To achieve these goals, firstly, we designed a simulated flight task based on visual search paradigms. Visual search tasks are used to simulate instrument scanning behaviors during flight. This scanning constitutes a core element of pilot workload during instrument flight, with studies showing that pilots allocate significant attentional resources to visual scanning under instrument flight rules [43, 44]. Visual search paradigms are also widely adopted in aviation human factors research to evaluate cockpit layout optimization [44], display complexity [45], and task load [46]. Furthermore, cross-domain validation in transportation research demonstrates the effectiveness of visual search tasks for evaluating performance and interface design, such as driver attention study under static and dynamic visual conditions [47]. Secondly, the study applied a range of analytical methods, including ERP, EEG wavelet energy analysis, eye-tracking features, ECG characteristics, and behavioral performance. Additionally, an SVM model was developed to classify participants’ emotional and fatigue states based on these measures.

Subjective scales confirmed that the visual search task effectively induced fatigue, evidenced by significant decreases in alertness and vigor and increases in anxiety and drowsiness [48]. These results align with previous findings that prolonged cognitive effort in attention-demanding tasks leads to subjective fatigue and emotional distress [49].

Behavioral performance showed significant increases in reaction time, consistent with typical indicators of mental fatigue [37, 42]. Interestingly, while accuracy showed a decreasing trend, it did not reach statistical significance, suggesting that fatigue primarily affects reaction time, while accuracy may be more resilient, possibly influenced by participants’ learning effects. Paired t-test results showed significant differences in pupil diameter, indicating that the visual search task successfully induced mental fatigue in participants [50, 51].

After the task, the heart rate significantly decreased, while TP and TINN significantly increased, reflecting reduced sympathetic activity and enhanced parasympathetic activity. The NN increased, along with increases in SDNN and RMSSD, indicating greater fluctuations in autonomic nervous system activity [44]. Although pNN50 and LF/HF also showed an increasing trend, they did not reach statistical significance. These changes in physiological indicators suggest that participants entered a state of mental fatigue, consistent with the findings from the eye-movement indicators, the subjective questionnaire responses, and EEG-related measures in this study [29, 30].

At the ERP level, N1 is an early perceptual component that reflects the initial allocation of attention to stimuli, while the N2 component is associated with conflict detection and attentional control. P2 is linked to the early evaluation of emotional information and enhanced attention, P3 reflects the late allocation of attentional resources, and LPP (Late Positive Potential) indicates sustained emotional evaluation and memory encoding [27, 39, 52].

In the positive emotion group, both positive and neutral stimuli exhibited fatigue-related changes after the visual search task, including reduced amplitudes of the N1, P3, and LPP components at the FZ and CZ electrodes. In the negative emotion group, the amplitudes of the N1 and N2 components at FZ, N1 at CZ, and N1 and P2 at PZ were all diminished. The changes in the N1 component reflect a decline in the brain’s sensitivity to positive emotional picture stimuli, reduced allocation of early attentional resources, and slower information processing speed [52]. The reduction in the P3 component suggests that after prolonged visual search tasks, participants allocated fewer attentional resources to emotional stimulus judgment, indicating potential difficulties in effectively processing emotional stimuli or regulating emotions [27, 52]. The alterations in the LPP component indicate blunted responses to emotional stimuli, impaired deep evaluation of emotional stimuli, and diminished emotional regulatory effects of positive emotional pictures [27, 39]. These findings align with previous research highlighting that fatigue impairs cognitive resources, leading to declines in attention and processing efficiency [53, 54].

Regarding condition effects, in the positive emotion group, image type significantly influenced the LPP and P3 components at FZ and CZ, as well as the P2, N2, LPP, and P3 components at PZ, suggesting that positive image stimuli had significantly stronger effects on cognitive processing and emotional stimulus intensity than neutral stimuli. In the negative emotion group, negative image stimuli exerted significantly stronger effects on cognitive processing and emotional stimulus intensity than neutral stimuli in the N2 component at FZ and CZ, and the P3 component at PZ. Notably, for the N1 and LPP components, neutral picture stimuli elicited larger amplitudes than negative picture stimuli in the negative emotion group, which differed from the positive emotion group. This phenomenon may be related to participants’ intentional inhibition of emotional responses to unpleasant pictures, which significantly reduced LPP amplitude, with this effect persisting throughout the LPP window [55].

To compare the arousal effects of positive and negative emotional stimuli on fatigue, a cross-group comparable emotional arousal index (\mathrm{\Delta }Amplitude = emotional stimulus amplitude – neutral stimulus amplitude) was constructed, using the amplitude induced by neutral emotional stimuli as the baseline. This approach provides a dynamic quantitative measure of fatigue-emotion interaction effects. Post-visual search task, amplitude changes induced by positive emotional images were larger than those induced by negative emotional images across the N2, P2, P3, and LPP components at the FZ, CZ, and PZ electrodes. This suggests that, following mental fatigue, positive emotional stimuli may have a stronger awakening effect on mental fatigue.

It has been confirmed in previous study that \alpha wave activity increases with the rise in mental fatigue [8], suggesting a slowdown in brain activity after the visual search task. In the negative group, the relative energy in the \delta band increased after the visual search task. This band has been positively correlated with mental fatigue in monotonous tasks [56]. The relative energy in the \beta band decreased, indicating a reduction in arousal levels [57]. Significant main effects of picture type were observed in the \theta- and \alpha-bands for both groups, reflecting different arousal effects elicited by positive, negative, and neutral stimuli. For the positive group, significant group effects were observed in the \theta-band, while for the negative group, these effects were significant in the \delta- and \beta-bands. These findings indicate that the visual search task led to a decrease in brain excitability among the subjects, resulting in symptoms of sleepy. Meanwhile, positive and negative groups exhibit significant interactions in the \theta- and \alpha-bands, indicating that there is an interaction between the changes in wavelet packet energy caused by the visual search task and those caused by emotional picture stimulation.

The SVM-based classification model demonstrated excellent performance in distinguishing fatigue states, with accuracy exceeding 93%, consistent with the observed physiological and behavioral changes. However, its ability to classify emotional states was less effective, with an accuracy of approximately 70%. The accuracy of emotion classification was lower than that of fatigue classification. In our experiment, we only used time-domain features (such as amplitude) of EEG and frequency-domain features from wavelet decomposition, which may not be sufficient to reflect the key neural mechanisms of emotional changes. This is fewer features than those extracted in previous study on emotion classification [58]. Additionally, deep learning is currently the dominant algorithm for emotion classification, and traditional SVM has less advantage in classification accuracy. The similar classification results of positive/negative and neutral emotional stimuli before and after the fatigue task may be because the decline in emotional regulation ability due to fatigue is holistic, but it does not affect emotional responses themselves [27, 59]. In real-world scenarios such as long-duration driving, where fatigue and emotion interact, the risk of accidents is elevated, and understanding the underlying mechanisms can offer methods for algorithm optimization [60].

5. Limitation

While this study provides valuable insights, several limitations should be acknowledged. First, the sample size of 30 participants may limit the generalizability of the findings, particularly to professional pilots who operate in highly dynamic environments. Second, the simulated flight task focused on instrument reading, which simplifies the complexities of real-world aviation scenarios (e.g., environmental stressors, multitasking demands). Third, although emotional induction tasks were effective, categorizing emotions into only three valence categories (positive, negative, and neutral) may not fully capture the multidimensional nature of emotional states. Additionally, the predefined stimuli might not reflect individual variability in emotional responses, and the short duration of emotional task blocks could have constrained the induction of sustained emotional states. Fourth, while the interaction between fatigue and emotions was explored, the specific effects of distinct emotional subtypes (e.g., anger vs. anxiety) on fatigue arousal remain unclear and warrant further investigation. Lastly, although the SVM model was effective in classifying fatigue, its accuracy in detecting emotional states under fatigue conditions was lower, highlighting the need for further research to improve classification precision in such scenarios.

6. Conclusions

In this study, a simulated flight task based on the visual search paradigm was designed. Emotional image stimuli were used to evoke participants’ positive and negative emotional states, while the visual search task was employed to induce mental fatigue. The changes following positive and negative emotional stimuli were analyzed based on EEG, ECG, eye-tracking metrics, and task performance. Additionally, the impact of these emotional states on fatigue recovery was evaluated. Finally, an SVM classifier was used to assess emotional stimuli and fatigue tasks.

The results demonstrated that the simulated flight task effectively induced mental fatigue. Following the task, participants exhibited reduced pupil diameter, prolonged reaction times, decreased HR, and increased values of NN, SDNN, and RMSSD. Neurophysiological measurements revealed that fatigue impaired attentional resource allocation and emotional processing. ERP components N1, P3, and LPP decreased, indicating weakened early attention, late cognitive evaluation, and emotional regulation capabilities. Additionally, the elevated relative energy of \alpha waves further suggested a decline in cerebral arousal levels. Notably, using the ERP amplitudes elicited by neutral images as a baseline, positive emotional stimuli induced stronger arousal effects than negative stimuli after fatigue, evidenced by larger amplitude differences (\mathrm{\Delta }N2, \mathrm{\Delta }P3, \mathrm{\Delta }LPP) across the frontal, central, and parietal regions. The SVM classifier demonstrated high accuracy in identifying fatigue states, but challenges remained in classifying emotional states under fatigue. These findings highlight the complex interaction between fatigue and emotional states and provide valuable insights for improving pilot safety and performance.

References

[1] Bendak S, Rashid HS. Fatigue in aviation: A systematic review of the literature. International Journal of Industrial Ergonomics. 2020; 76: 102928. https://doi.org/10.1016/j.ergon.2020.102928.

[2] Rosa E, Lyskov E, Grönkvist M, Kölegård R, Dahlström N, Knez I, et al. Cognitive performance, fatigue, emotional, and physiological strains in simulated long-duration flight missions. Military Psychology: the Official Journal of the Division of Military Psychology, American Psychological Association. 2022; 34: 224–236. https://doi.org/10.1080/08995605.2021.1989236.

[3] Chand HV, Karthikeyan J. CNN Based Driver Drowsiness Detection System Using Emotion Analysis. Intelligent Automation & Soft Computing. 2022; 31: 717–728. https://doi.org/10.32604/iasc.2022.020008.

[4] Hu X, Lodewijks G. Detecting fatigue in car drivers and aircraft pilots by using non-invasive measures: The value of differentiation of sleepiness and mental fatigue. Journal of Safety Research. 2020; 72: 173–187. https://doi.org/10.1016/j.jsr.2019.12.015.

[5] Arsintescu L, Pradhan S, Chachad RG, Gregory KB, Mulligan JB, Flynn-Evans EE. Early starts and late finishes both reduce alertness and performance among short-haul airline pilots. Journal of Sleep Research. 2022; 31: e13521. https://doi.org/10.1111/jsr.13521.

[6] Li Y, He J. A review of strategies to detect fatigue and sleep problems in aviation: Insights from artificial intelligence. Archives of Computational Methods in Engineering. 2024; 31: 4655–4672. https://doi.org/10.1007/s11831-024-10123-5.

[7] Magnuson JR, Doesburg SM, McNeil CJ. Development and recovery time of mental fatigue and its impact on motor function. Biological Psychology. 2021; 161: 108076. https://doi.org/10.1016/j.biopsycho.2021.108076.

[8] Tran Y, Craig A, Craig R, Chai R, Nguyen H. The influence of mental fatigue on brain activity: Evidence from a systematic review with meta-analyses. Psychophysiology. 2020; 57: e13554. https://doi.org/10.1111/psyp.13554.

[9] Wascher E, Getzmann S. Rapid mental fatigue amplifies age-related attentional deficits. Journal of Psychophysiology. 2014; 28: 215–224. https://doi.org/10.1027/0269-8803/a000127.

[10] Wang D, Xie X, Tian H, Wu T, Liu C, Huang K, et al. Mental fatigue and negative emotion among nurses during the COVID-19 pandemic. Curr Psychol. 2022; 41: 8123–8131.

[11] Alreshidi I, Moulitsas I, Jenkins KW. Advancing aviation safety through machine learning and psychophysiological data: a systematic review. IEEE Access. 2024; 12: 5132–5150. https://doi.org/10.1109/ACCESS.2024.3349495.

[12] Wingelaar-Jagt YQ, Wingelaar TT, Riedel WJ, Ramaekers JG. Fatigue in Aviation: Safety Risks, Preventive Strategies and Pharmacological Interventions. Frontiers in Physiology. 2021; 12: 712628. https://doi.org/10.3389/fphys.2021.712628.

[13] Caldwell JA, Gilreath SR. A survey of aircrew fatigue in a sample of U.S. Army aviation personnel. Aviation, Space, and Environmental Medicine. 2002; 73: 472–480.

[14] Miller JC, Melfi ML. Causes and effects of fatigue in experienced military aircrew. Technical Report. Brooks City-Base (TX): Air Force Research Laboratory. 2006. Report No.: AFRL-HE-BR-TR-2006-0071. Available at: February 15, 2025, https://apps.dtic.mil/sti/pdfs/ADA462989.pdf.

[15] Luciani F, Veneziani G, Ciacchella C, Rocchi G, Reho M, Gennaro A, et al. Safety at high altitude: the importance of emotional dysregulation on pilots’ risk attitudes during flight. Frontiers in Psychology. 2022; 13: 1042283. https://doi.org/10.3389/fpsyg.2022.1042283.

[16] Jeon A. The effect of pre-flight attendants’ emotional intelligence, emotional labor, and emotional exhaustion on commitment to customer service. Service Business. 2016; 10: 345–367. https://doi.org/10.1007/s11628-015-0273-2.

[17] Tichon JG, Wallis G, Riek S, Mavin T. Physiological measurement of anxiety to evaluate performance in simulation training. Cognition, Technology & Work. 2014; 16: 203–210. https://doi.org/10.1007/s10111-013-0257-8.

[18] Schoofs D, Wolf OT, Smeets T. Cold pressor stress impairs performance on working memory tasks requiring executive functions in healthy young men. Behavioral Neuroscience. 2009; 123: 1066–1075. https://doi.org/10.1037/a0016980.

[19] Li J, Song G, Miao D. Effect of mental fatigue on nonattention: a visual mismatch negativity study. Neuroreport. 2016; 27: 1323–1330. https://doi.org/10.1097/WNR.0000000000000694.

[20] Causse M, Dehais F, Péran P, Sabatini U, Pastor J. The effects of emotion on pilot decision-making: A neuroergonomic approach to aviation safety. Transportation Research Part C: Emerging Technologies. 2013; 33: 272–281. https://doi.org/10.1016/j.trc.2012.04.005.

[21] Chang KY, Chang WH, Yeh YC. The effect of role stress and emotional exhaustion on well-being among pilots: the moderating effect of leisure participation. The International Journal of Aerospace Psychology. 2024; 34: 59–72. https://doi.org/10.1080/24721840.2023.2242382.

[22] Hidalgo-Muñoz AR, Mouratille D, Matton N, Causse M, Rouillard Y, El-Yagoubi R. Cardiovascular correlates of emotional state, cognitive workload and time-on-task effect during a realistic flight simulation. International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology. 2018; 128: 62–69. https://doi.org/10.1016/j.ijpsycho.2018.04.002.

[23] Wang P, Houghton R, Majumdar A. Detecting and Predicting Pilot Mental Workload Using Heart Rate Variability: A Systematic Review. Sensors (Basel, Switzerland). 2024; 24: 3723. https://doi.org/10.3390/s24123723.

[24] Joyce C, Rossion B. The face-sensitive N170 and VPP components manifest the same brain processes: the effect of reference electrode site. Clinical Neurophysiology: Official Journal of the International Federation of Clinical Neurophysiology. 2005; 116: 2613–2631. https://doi.org/10.1016/j.clinph.2005.07.005.

[25] Weinberg A, Hilgard J, Bartholow BD, Hajcak G. Emotional targets: evaluative categorization as a function of context and content. International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology. 2012; 84: 149–154. https://doi.org/10.1016/j.ijpsycho.2012.01.023.

[26] MacNamara A, Joyner K, Klawohn J. Event-related potential studies of emotion regulation: A review of recent progress and future directions. International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology. 2022; 176: 73–88. https://doi.org/10.1016/j.ijpsycho.2022.03.008.

[27] Boksem MAS, Meijman TF, Lorist MM. Effects of mental fatigue on attention: an ERP study. Cognitive Brain Research. 2005; 25: 107–116. https://doi.org/10.1016/j.cogbrainres.2005.04.011.

[28] Ray WJ, Cole HW. EEG alpha activity reflects attentional demands, and beta activity reflects emotional and cognitive processes. Science (New York, N.Y.). 1985; 228: 750–752. https://doi.org/10.1126/science.3992243.

[29] Thayer JF, Lane RD. The role of vagal function in the risk for cardiovascular disease and mortality. Biological Psychology. 2007; 74: 224–242. https://doi.org/10.1016/j.biopsycho.2005.11.013.

[30] Melo HM, Nascimento LM, Takase E. Mental fatigue and heart rate variability (HRV): The time-on-task effect. Psychology & Neuroscience. 2017; 10: 428–436. https://doi.org/10.1037/pne0000110.

[31] Csathó Á Van der Linden D, Matuz A. Change in heart rate variability with increasing time-on-task as a marker for mental fatigue: A systematic review. Biological Psychology. 2024; 185: 108727. https://doi.org/10.1016/j.biopsycho.2023.108727.

[32] Cheng S, Sun J, Ma J, Dang W, Tang M, Hui D, et al. Posturographic Balance’s Validity in Mental and Physical Fatigue Assessment Among Cadet Pilots. Aerospace Medicine and Human Performance. 2018; 89: 961–966. https://doi.org/10.3357/AMHP.5128.2018.

[33] Diaz-Piedra C, Rieiro H, Suárez J, Rios-Tejada F, Catena A, Di Stasi LL. Fatigue in the military: towards a fatigue detection test based on the saccadic velocity. Physiological Measurement. 2016; 37: N62. https://doi.org/10.1088/0967-3334/37/9/N62.

[34] Chen J, Lin X, Ma W, Wang Y, Tang W. EEG-based emotion recognition for road accidents in a simulated driving environment. Biomedical Signal Processing and Control. 2024; 87: 105411. https://doi.org/10.1016/j.bspc.2023.105411.

[35] Lee DH, Jeong JH, Kim K, Yu BW, Lee SW. Continuous EEG decoding of pilots’ mental states using multiple feature block-based convolutional neural network. IEEE Access. 2020; 8: 121929–121941. https://doi.org/10.1109/ACCESS.2020.3006907.

[36] Yu X, Chen CH, Yang H. Air traffic controllers’ mental fatigue recognition: A multi-sensor information fusion-based deep learning approach. Advanced Engineering Informatics. 2023; 57: 102123. https://doi.org/10.1016/j.aei.2023.102123.

[37] Qin H, Zhou X, Ou X, Liu Y, Xue C. Detection of mental fatigue state using heart rate variability and eye metrics during simulated flight. Human Factors and Ergonomics in Manufacturing & Service Industries. 2021; 31: 637–651. https://doi.org/10.1002/hfm.20927.

[38] Ghanbari Z, Choobineh A, Zakerian SA, Gharagozlou F, Nami M. Emotionally Loaded Visual Stimuli to Alter Brain Arousal: A Flicker Fusion Study. Health Scope. 2023; 12: e132613. https://doi.org/10.5812/jhealthscope-132613.

[39] Fan J, Li W, Lin M, Li X, Deng X. Effects of mindfulness and fatigue on emotional processing: an event-related potentials study. Frontiers in Behavioral Neuroscience. 2023; 17: 1175067. https://doi.org/10.3389/fnbeh.2023.1175067.

[40] Lu B, Hui MA, Yu-Xia H. The Development of Native Chinese Affective Picture System–A pretest in 46 College Students. Chinese Mental Health Journal. 2005; 19: 719–722.

[41] Liu J, Zhu Y, Cong F, Björkman A, Malesevic N, Antfolk C. Analysis of modulations of mental fatigue on intra-individual variability from single-trial event related potentials. Journal of Neuroscience Methods. 2024; 406: 110110. https://doi.org/10.1016/j.jneumeth.2024.110110.

[42] Pauletti C, Mannarelli D, Pauri F, Petritis A, Maffucci A, Currà A, et al. The role of fatigue in attentional processing in multiple sclerosis: data from event-related potentials. Journal of Neural Transmission (Vienna, Austria: 1996). 2025; 132: 105–115. https://doi.org/10.1007/s00702-024-02827-6.

[43] Wang Y, Guo X, Liu Q, Yang X, Bai Y, Du J, et al. Eye movement characteristics research on pilots of different experience background during aircraft cockpit display image visual search task. In Man–Machine–Environment System Engineering International Conference on Man-Machine-Environment System Engineering (pp. 85–94). Springer: Singapore. 2018.

[44] Allsop J, Gray R, Bulthoff HH, Chuang L. Effects of anxiety and cognitive load on instrument scanning behavior in a flight simulation. In 2016 IEEE Second Workshop on Eye Tracking and Visualization (ETVIS) (pp. 55–59). 2016.

[45] Lazaro MJ, Kang Y, Yun MH, Kim S. The effects of visual complexity and decluttering methods on visual search and target detection in cockpit displays. International Journal of Human–Computer Interaction. 2021; 37: 588–600. https://doi.org/10.1080/10447318.2021.1890491.

[46] Zhou Y, Chen DK, Tan XX, Zhao M. Study on Visual Search Performance of Cockpit Interfaces Under Time Pressures. Journal of Northeastern University (Natural Science). 2023; 44: 1514. (In Chinese)

[47] Chu Y, Qin H, Ran L. Influence of information layout of vehicle central control screen on visual search performance. Science Technology and Engineering Journal. 2022; 22: 13521–13526.

[48] Shahid A, Wilkinson K, Marcu S, Shapiro CM. Visual analogue scale to evaluate fatigue severity (VAS-F). In STOP, THAT and One Hundred Other Sleep Scales (pp. 399–402). Springer: New York, NY. 2011.

[49] Möckel T, Beste C, Wascher E. The Effects of Time on Task in Response Selection–An ERP Study of Mental Fatigue. Scientific Reports. 2015; 5: 10113. https://doi.org/10.1038/srep10113.

[50] Renata V, Li F, Lee CH, Chen CH. Investigation on the correlation between eye movement and reaction time under mental fatigue influence. In 2018 International Conference on Cyberworlds (CW) (pp. 207–213). IEEE. 2018.

[51] Bafna T, Hansen JP. Mental fatigue measurement using eye metrics: A systematic literature review. Psychophysiology. 2021; 58: e13828. https://doi.org/10.1111/psyp.13828.

[52] Ghani U, Signal N, Niazi IK, Taylor D. ERP based measures of cognitive workload: A review. Neuroscience and Biobehavioral Reviews. 2020; 118: 18–26. https://doi.org/10.1016/j.neubiorev.2020.07.020.

[53] Shigihara Y, Tanaka M, Ishii A, Tajima S, Kanai E, Funakura M, et al. Two different types of mental fatigue produce different styles of task performance. Neurology, Psychiatry and Brain Research. 2013; 19: 5–11. https://doi.org/10.1016/j.npbr.2012.07.002.

[54] Tanaka M, Mizuno K, Tajima S, Sasabe T, Watanabe Y. Central nervous system fatigue alters autonomic nerve activity. Life Sciences. 2009; 84: 235–239. https://doi.org/10.1016/j.lfs.2008.12.004.

[55] Moser JS, Hajcak G, Bukay E, Simons RF. Intentional modulation of emotional responding to unpleasant pictures: an ERP study. Psychophysiology. 2006; 43: 292–296. https://doi.org/10.1111/j.1469-8986.2006.00402.x.

[56] Lal SKL, Craig A. Driver fatigue: electroencephalography and psychological assessment. Psychophysiology. 2002; 39: 313–321. https://doi.org/10.1017/s0048577201393095.

[57] Zhao C, Zhao M, Liu J, Zheng C. Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. Accident; Analysis and Prevention. 2012; 45: 83–90. https://doi.org/10.1016/j.aap.2011.11.019.

[58] Shen P, Changjun Z, Chen X. Automatic speech emotion recognition using support vector machine. In Proceedings of 2011 International Conference on Electronic & Mechanical Engineering and information Technology. IEEE. 2011; 2: 621–625. https://doi.org/10.1109/EMEIT.2011.6023178.

[59] Grillon C, Quispe-Escudero D, Mathur A, Ernst M. Mental fatigue impairs emotion regulation. Emotion (Washington, D.C.). 2015; 15: 383–389. https://doi.org/10.1037/emo0000058.

[60] Ji L, Yi L, Li H, Han W, Zhang N. Detection of Pilots’ Psychological Workload during Turning Phases Using EEG Characteristics. Sensors (Basel, Switzerland). 2024; 24: 5176. https://doi.org/10.3390/s24165176.