Cognitive Neural Differentiation and Integration of Multimodal Metaphors: Influencing Factors and Processing Mechanisms

Abstract

Metaphors are a core category of cognitive linguistics and an important mode of human thinking. They concretize abstract concepts through cross-domain mapping and build a bridge between cognition and understanding in verbal communication and interpersonal communication. Metaphor research has shifted from a pure linguistic perspective to multidisciplinary and multimodal research. However, there has yet been no systematic review of how the brain processes the differentiation and integration mechanism of verbal and non-verbal modal metaphorical information, as well as the main influencing factors. In particular, a weak area in current research is how special groups achieve compensation of metaphorical understanding through neuroplasticity. This review systematically describes the relevant achievements in cognitive neuroscience in recent years, with the aim of revealing the main influencing factors of multimodal metaphor processing and the process of neural differentiation and cross-modal integration. This review also focuses on the compensatory mechanisms in autism, aphasia, and deafness, and describes how they achieve effective metaphorical understanding through the reconstruction of neuroplasticity. Moreover, it provides an integrated perspective for understanding the neural basis of metaphorical cognition, as well as a theoretical basis and practical guidance for advancing multimodal metaphor research and applications in rehabilitation. Future research should combine temporal neurodynamic technology with ecological interventions designed to further promote advancement in this field.

Keywords

multimodal metaphor / neural differentiation / cross-modal integration / neuroplasticity / compensatory mechanisms

1. Introduction

Metaphor refers to the systematic representation of concepts within one cognitive category [1]. It is not only a linguistic phenomenon or rhetorical device, but also a cognitive-communicative tool, the core of which is to transform abstract and complex concepts (such as “life”) into concrete, perceptible experiences (such as “journeys”) through cross-domain mapping, thus achieving efficient information transfer, emotional resonance, and social interaction [2, 3, 4]. For example, a series of metaphorical languages with daily experience, such as “Dark Age”, “Personality Like Pepper”, “Mood Like a Thunderstorm”, and “Feelings Enter the Autumn Stage”, achieve the perceptualization of abstract experience through concrete analogies [5, 6]. In fact, concepts such as age, character, mood, and emotion do not have characteristics such as brightness, food, and weather, but people will still often use metaphors to describe things [7, 8].

With the rise of cognitive science and multimodal language analysis, metaphor research has progressed from viewing metaphors as rhetorical decorations at the linguistic level, to understanding them as the basic way in which human abstract thinking and conceptual systems are constructed. Researchers have discovered that during cross-subjective information transmission, the construction and understanding of metaphors has moved beyond the scope of language. Metaphorical meaning can be co-transmitted through two or more symbolic modalities (e.g., visual images, gestures, spatial design, and other non-verbal metaphors) and is now developing towards multimodal metaphors. This helps individuals to more accurately understand the subjective experience of the other party, including semantic connotation, communicative intention, and the emotional dimension [9, 10, 11, 12]. For example, the image metaphor of “dead trees symbolizing ecological crisis” in public service advertisements, the synergy between the phonetic metaphor of “indulgence in silkiness” and the silk floating animation in chocolate advertisements, as well as the spatial coding of gestures through the metaphor of “ascending action metaphor progress” all indicate the activation of cross-domain associations between verbal and non-verbal modalities in metaphor construction [13, 14, 15, 16]. It is therefore important to identify the main factors affecting the dynamic comprehension process of individuals when they receive verbal or non-verbal metaphorical information from others.

The processing efficiency of multimodal metaphors is closely related to the complex interaction of social and cultural practices. However, based on its core role in constructing the basic framework of metaphor understanding, regulating the real-time processing process, and revealing the differentiation of understanding paths, the core influencing factors can be summarized into three dimensions: knowledge experience, contextual information, and individual differences [17, 18, 19]. Relevant neurological evidence suggests that in multimodal interfaces, the cross-modal integration coding of metaphors (visual symbol topological mapping, auditory prosodic emotion marking, tactile texture embodied simulation) can induce dynamic functional reorganization of distributed brain networks. The specific neural mechanisms are reflected in the enhancement of cross-brain functional coupling (e.g., the semantic binding of the angular gyrus-fusiform circuit pathway supports the integration of images and texts), and in the improved synchronization of high-frequency neural oscillations (the real-time integration efficiency of Gamma band activity reflecting multimodal information) [20, 21, 22].

Worth noting is that this mechanism has unique compensatory value in special group communication. For example, patients with autism are able to reconstruct semantic networks through visual-tactile multimodal channels (e.g., flowcharts, metaphors, social rules, vibrational feedback, and emotional intensity). Moreover, their neuroplasticity is manifested by functional reorganization of the mirror system and enhanced activation of the right temporoparietal joint area [23, 24]. Patients with aphasia rely on compensatory activation of right-brain non-verbal pathways (e.g., the right superior temporal sulcus is involved in metaphor comprehension) to achieve partial recovery of language function through multimodal integration [25, 26, 27]. The deaf-mute group relies on the spatial coding network of the right parietal lobe, as well as the motor intention decoding function of the visual cortex through the spatial metaphor of sign language (e.g., “time is like flowing water” simulates the flow of time through gesture direction) [28, 29]. It is also worth noting that the direction and degree of neuroplasticity are highly dependent on the joint effects of innate endowment and acquired intervention. The practical efficacy of these compensatory mechanisms, such as universality, efficiency and optimal inducing conditions, remains subject to individual differences, cognitive load and technology transformation. Therefore, the main direction and key aim of future research is how to translate neural advantages found in the laboratory, such as enhanced activation of the right temporoparietal symphysis region and compensation of the right brain pathway, into practical applications.

In summary, this paper focuses on the study of metaphors in cognitive linguistics. The aim is to systematically elucidate how individuals receive and understand the information transmitted by others through verbal and non-verbal modalities in daily communication situations. What are the main factors that regulate this process? During the process of information reception, how does the neural mechanism of the brain realize the differentiation and processing of verbal and non-verbal modalities and cross-modal integration? For special groups with specific communication disorders (e.g., autism spectrum disorder, aphasia, deaf and mute people), how does their nervous system form a compensatory mechanism through plasticity restructuring to achieve effective communication? Therefore, this review will first cover the influence of knowledge experience, contextual information, and individual differences on the understanding process of multimodal metaphors, based on the current and relevant theories of metaphors. Next, we discuss neural function differentiation between verbal and non-verbal modalities and the brain network synergy, and neuroplasticity of cross-modal metaphors. The neural coding mechanism of multimodal metaphors is also systematically explained. Finally, this thesis examines future research directions on multimodal metaphors. It aims to promote interdisciplinary integration and further in-depth research into multimodal metaphors. This should expand our understanding of the nature of metaphors, optimize social communication and technological innovation, and provide theoretical guidance and a scientific basis for development prospects in education, medical care, artificial intelligence and other application fields.

2. The Main Influencing Factors for Metaphor Processing

Metaphor processing is affected by the interaction of a variety of complex factors. Although there are many influencing factors, this study focuses on the three key dimensions of knowledge experience, contextual information, and individual differences. These are based on above core role in constructing the basic framework of metaphor understanding, the regulation of real-time processing, and revealing the differentiation of understanding paths.

2.1 Knowledge and Experience

The Conceptual Metaphor Theory (CMT) proposed by Lakoff and Johnson (1980) [2] has had a significant impact in the field of cognitive linguistics. It challenges the traditional linguistic view of a metaphor as a rhetorical ornament, arguing instead that it is essentially a cognitive mechanism of cross-domain mapping. In other words, from the concrete source domain (e.g., journey) through to the abstract target domain (e.g., life) to carry out systematic cross-domain mapping. The formation of this mapping is not arbitrary, but strongly dependent on the knowledge and experience accumulated by the individual, i.e., the individual’s cognition of things and their interrelationships acquired through education or hands-on practice [30]. The core sources of individual knowledge experience include both culturally acquired knowledge and embodied experiential knowledge [2, 31].

Culturally acquired knowledge stems from shared metaphorical schemas that are passed down from generation to generation by specific cultural groups through education, media dissemination, and social practice. These schemas shape the default framework for group members to understand abstract concepts and internalize them into individual reserves of cultural knowledge. For example, the difference between the knowledge that “dragon” symbolizes authority and auspiciousness in Chinese culture, and the knowledge that “dragon” in Western culture is a metaphor for danger and evil [31]. This stems from their unique historical narratives and collective experiences, and is solidified through idioms (e.g., Wang Zi Cheng Long: 望子成龙), proverbs and other linguistic forms. Knowledge of cultural norms profoundly affects the ability of individuals to decode metaphorical information in cross-cultural communication. Embodied experiential knowledge, on the other hand, is based on the bodily experience accumulated by humans through interaction of the perceptual-motor system with the physical and social environment [32]. Such experiences form an indispensable, a priori basis for understanding abstract metaphors. Due to the finite nature of cognitive resources (e.g., working memory) [18], the grasp of abstract concepts often relies on the embodied anchoring of bodily experience. For example, the “warm” tactile sensation associates “friendly” emotions through connection between the somatosensory cortex and the limbic system. This process involves a wealth of tactile experiences, and the emotional feelings evoked directly affect the individual’s psychological and behavioral judgments [33]. Somatosensory experiential knowledge is based on multimodal visual, tactile, kinesthetic and other modalities, and enables individuals to use “somatosensory analogies” to quickly activate and understand abstract semantic networks [34, 35].

Therefore, the knowledge of cultural acquisition and the knowledge of embodied experience together constitute the “knowledge and experience base” required for an individual’s understanding of metaphors. It is the synergy of these two aspects of knowledge that enables people to process and interpret metaphorical information easily, quickly, and with relative accuracy when understanding the physical world or communicating with people.

2.2 Contextual Information

Contextual information refers to the specific environment in which language occurs, including verbal cues (e.g., context, intonation) and non-verbal cues (e.g., physical scenes, participant relationships, accompanying facial expressions, shared cultural backgrounds, etc.). It is not a static background, but a core cognitive variable that dynamically shapes the construction of metaphorical meaning, provides immediate and constrained clues for the encoding and decoding of metaphors, and profoundly affects the generation and interpretation of metaphorical meanings [36, 37]. The core of the influence mechanism lies in the ability of contextual information to pre-activate and filter relevant knowledge and experience. A specific communicative context acts as a “cognitive sieve”, preferentially activating specific parts of the individual’s knowledge and experience base that are highly adapted to the current situation, while suppressing irrelevant parts. This provides the most appropriate source domain options and target domain interpretation frameworks for metaphor mapping [38, 39]. For example, strict social norms in diplomatic settings will activate metaphorical symbols such as “handshake” and “bridge” that meet the requirements of power relations and etiquette to express cooperation. The context of commercial advertising tends to activate consumer cultural symbols, such as “diamonds symbolize eternal love”, to fit the logic of capital, reflecting the context’s directional invocation of internalized knowledge of social norms [9, 13].

Further, contextual information builds the semantic expectation framework needed to understand metaphors by providing rich cues, especially non-verbal and multimodal information. These guide processors to predict and integrate metaphorical expressions that are about to emerge or have already appeared [40]. In strong contexts that share a lot of background information (e.g., family member conversations), a vague reference (e.g., “see you in the old place”) can be quickly understood because it is highly dependent on context-activated shared spatial memory, and the framework construction is rapid and efficient. Conversely, in weak contexts (e.g., cross-cultural negotiations) that lack a common context, a single verbal metaphor is often insufficient to carry abstract concepts, and the role of contextual information is more prominent. In such cases it is necessary to rely on multimodal cues such as images, gestures, and expressions to “compensate” and work together to construct a complete semantic framework for understanding [41, 42]. The richness and consistency of contextual cues directly determine the clarity of the expected framework, which in turn affects the smoothness of metaphor processing.

In particular, when the metaphorical expression itself is ambiguous or conflicts with the context (e.g., ironic language or metaphorical actions), contextual information becomes a key driver to disambiguate and reconstruct the speaker’s true intention [43, 44]. For example, when the literal meaning of words that appear to be praise is actually sarcastic and clashes with the context (e.g., the speaker’s sarcastic expression, tone of voice, tensions, or the context of the event). Understanders must rely heavily on these conflicting contextual cues, invoke cognitive flexibility to inhibit automated literal interpretation priorities, actively adjust comprehension strategies, and finally reconstruct metaphorical intentions that fit the overall information of the context [45, 46]. This process reveals the core role of contextual information in guiding comprehension beyond the literal surface and into the level of metaphorical intent, while presenting high requirements for the cognitive regulation ability of processors.

In summary, metaphor processing relies heavily on contextual information for dynamic and real-time adjustment. It uses long-term accumulated knowledge and experience as the cognitive basis to support top-down processing. In this process, individuals use existing knowledge and experience, as well as expectations and cognitive frameworks, to actively guide information processing by a concept-driven processing method [47, 48]. Metaphor processing also relies on the immediacy and constraint clues provided by contextual information to drive bottom-up processing. This refers to the direct triggering of information processing by the physical characteristics of external stimuli, which is a data-driven processing method [49, 50].

Contextual information finely regulates the generation and understanding of metaphorical meaning by activating and screening relevant knowledge, constructing a semantic expectation framework, and resolving conflict-inducing intent reconstruction. To a large extent, the degree of individual cognitive flexibility determines whether it can effectively integrate “long-term schema” provided by knowledge and experience, with “real-time navigation” provided by contextual cues. By coordinating the interaction between the two, an accurate grasp and efficient processing of metaphorical information can be realized.

2.3 Individual Differences

Individual differences profoundly shape the cognitive pathways and specificity of processing strategies for metaphor processing. Firstly, the core of cognitive style differences (such as field dependence and field independence) lies in the differently weighted distribution of an individual’s dependence on “environmental cues” and “internal representations”. This is mainly manifested in the fact that field-dependent individuals are highly anchored to the immediate context and group consensus. Such individuals give priority to integrating external cues (e.g., the cultural symbol of “red = festive” in advertising), and their metaphorical interpretation is easily guided by situational cues and shared knowledge, which are closely related to the efficiency of their social-emotional information processing network. On the other hand, field-independent individuals tend to rely on internal knowledge structure and autonomous analytical ability. They can actively strip away contextual constraints and extract atypical associations from their own experience base for creative mapping (e.g., reconstructing the visual features of calligraphy “Feibai” into a philosophical metaphor of “blank life”), thus reflecting a stronger neural mechanism of cognitive inhibition and conceptual reconstruction [51].

Secondly, differences in group identity (e.g., occupational groups, subcultural circles) stem from the dominant role of domain-specific embodied schemas internalized in long-term professional practice on concept mapping. Repeated bodily experiences and conceptual manipulations in specific domains (e.g., tactical collaboration of athletes or esports players) shape highly specialized perceptual-motor patterns and knowledge frameworks, which then become the default source domain for metaphor generation [52]. For example, doctors use the metaphor of “immune system natural defense” to describe physiological mechanisms, which stems from the deep neural binding of military concepts and immune function in their professional training. E-sports players use “push tower” to refer to team goal achievement, which is based on the embodied association reinforcement of their manipulation actions and tactical success [17, 53]. This identity-driven metaphor preference is essentially the embodiment of professional cognitive frameworks at the neural representation level, which can improve the efficiency of metaphor processing in the domain. However, it may also limit the flexible processing of metaphors across domains.

It is worth noting that differences in neurophysiological or sensory channels trigger cross-modal functional compensation and neuroplastic reorganization, altering the perceptual entry and processing pathways of metaphors [54, 55]. For example, due to hearing loss, the deaf-mute group has developed a synergistic reinforcement mechanism of visual-spatial-tactile channels. This is manifested as orientation metaphors in spatial sign language (e.g., gesture trajectories simulate time flow), while tactile vibrations are encoded as emotional intensity symbols, and abstract concept understanding is achieved by relying on neural remodeling in the visual cortex, somatosensory cortex, and posterior parietal cross-modal integration area [28, 56]. Individuals on the autism spectrum may experience challenges in linguistic metaphor processing due to differences in Theory of Mind networks or semantic integration pathways, but their cognitive systems are often compensated through visual regularization and concrete sensory anchoring. They can also use flowcharts to deconstruct social metaphors, or transform emotional fluctuations into physical representations of tactile temperature changes (cold/hot), highlighting their neural superiority in rule-driven and sensory figurative processing [57].

Therefore, the influence of individual differences on metaphor processing is actually based on the adaptive reconstruction of cognitive style, group identity, and neural basis. It determines that information extraction is biased towards contextual cues, internal schema inhibition, or compensatory sensory channels. This shifts the mapping mechanism towards conventional consensus, independent innovation, or cross-modal transformation, and shapes the differentiated activation patterns of social brain network, executive control network, and sensorimotor integration network. Therefore, systematic cognitive reconstruction reveals that metaphor processing is not a unified path, but rather an adaptive product of differences in individual neurocognitive systems and their interaction with environment.

3. Cognitive Neural Differentiation and Integration Mechanisms of Multimodal Metaphors

3.1 Neurological Differentiation of Verbal and Non-Verbal Modalities

The core neural mechanism of language metaphor processing needs to play the cross-domain mapping function of the left hemisphere language region, while also depending on the semantic integration ability of the right hemisphere association network [58]. Using functional magnetic resonance imaging (fMRI), Benedek et al. (2014) [59] found that the activation intensity of the left middle temporal gyrus (MTG) and angular gyrus (AG) in novel metaphor generation tasks was significantly greater than that of literal language processing, suggesting they play a central role in topological associations between abstract and concrete concepts [60]. Specifically, during the processing of linguistic metaphors, the MTG is mainly responsible for extracting the semantic features of abstract concepts (e.g., time), while the AG is responsible for integrating the association between concrete experiences (e.g., knife) and abstract concepts (e.g., time) to complete cross-domain semantic mapping [58]. A cross-cultural study further show that English metaphor processing exhibits stronger leftization activity in the brain compared to Chinese metaphors. Given the visual characteristics of the Chinese writing system and the dominant role of the right hemisphere in visuospatial processing, there is evidence that Chinese metaphorical comprehension occupies more resources in the right hemisphere [61]. One possible explanation is that, although the core MTG and AG mechanisms underpin metaphorical mapping across languages, the specific neural pathways that support pre-mapping language symbol processing vary among writing systems. For example, the processing of two-character metaphors (e.g., Xin Hai: 心海) in Chinese invokes the fusiform gyrus (FG) to process the unique visual morphology of Chinese characters and perform semantic encoding. In contrast, alphabetic scripts such as English rely more on the superior temporal sulcus (STS) of the left hemisphere for phonological-semantic binding [62, 63]. When a particular writing system processes literal and figurative meaning, its neural processes dynamically construct the necessary linguistic representations. This complex construction process is moderated by factors such as linguistic characteristics, context, reasoning difficulty, and semantic prominence [64], and significantly affects the asymmetry of brain processing. Eventually, these conditioned representations are integrated into the core cross-domain mapping mechanism regulated by MTG and AG. However, caution must be exercised when interpreting neuroimaging findings from this type of cross-linguistic comparison. The neural differences in metaphor processing between Chinese and English, although closely related to the characteristics of writing systems (such as logographic vs phonetic scripts), many current studies have not systematically and rigorously controlled a range of potential confounding variables in the design of stimulus materials, such as orthographic complexity, semantic transparency, word frequency, familiarity, and context dependence [45, 64, 65]. Specifically, due to the visual complexity and higher semantic transparency of Chinese characters, there may be a greater reliance on resources from the right hemisphere in the early processing stages; whereas phonetic scripts like English tend to rely more on the left hemisphere’s phonological-semantic pathway [62, 63]. If these variables are not sufficiently matched in cross-language comparisons, then the observed differences in brain region activation are difficult to clearly define in terms of how much they reflect differences in the deep metaphor processing mechanisms themselves versus and how much they stem from differences in these basic perceptual or lexical attributes. Therefore, although existing evidence suggests a preliminary trend that there may be lateralized differences in metaphor processing between Chinese and English, it is premature to regard this as a definitive and universal conclusion about neural mechanisms. Future research needs to rigorously match these confounding variables through carefully designed experiments in order to more robustly isolate the independent contributions of script system characteristics to metaphor neural representation, thereby revealing more clearly the commonalities and specificities of neural mechanisms underlying cross-language metaphor processing.

It should be noted that some studies have shown that during brain processing, attention components in the anterior cingulate gyrus and prefrontal regions monitor and filter relevant aspects of the context and appropriate meaning. N400 components are often observed in the early stages of linguistic metaphorical processing, and their amplitude is widely thought to reflect the difficulty of semantic integration. In this process, the prefrontal cortex (PFC) can be used to help suppress literal meaning interference and regulate the contextual adaptation of metaphors. Afterwards, a late positive component (LPC) or a component known as P600 typically appears. Although they may overlap in latency and scalp distribution, their cognitive functions still have certain emphases. P600 is usually closely associated with reanalysis of syntactic or semantic-syntactic interfaces, conflict resolution, or cognitive control processes [66], while LPC is more related to late cognitive processing such as emotional evaluation, fine semantic integration, situational model updating, or memory encoding [67]. In metaphor comprehension, the occurrence of P600/LPC typically reflects the secondary processing, reconstruction, or evaluation of metaphorical meaning based on contextual information. Specifically, when metaphor comprehension involves reanalyzing the conflict between literal and metaphorical meanings, it may more easily elicit P600; whereas when the comprehension process focuses on the emotional resonance of the metaphor or integrating it into a broader situational model, it may manifest as LPC [68, 69, 70]. Moreover, the activation strength of this component increases with decreased familiarity with the metaphor, thereby optimizing the processing efficiency and accuracy of understanding language metaphors [71, 72]. More importantly, the performance and functional interpretation of these event-related potentials (ERP) components highly depend on experimental tasks and contextual factors. For example, the effect size of the N400 may vary due to the novelty of the metaphor and the strength of contextual support, while the amplitude and distribution of the LPC/P600 may also differ depending on whether the task requires semantic consistency judgment or emotional resonance evaluation [73, 74]. These results suggest the processing of linguistic metaphors in the brain is a dynamic and cognitive regulation process, and the higher the novelty of the metaphor, the greater the consumption of prefrontal neural resources. This pattern of cognitive cost can effectively resolve the resolution accuracy of unconventional metaphors [75].

Compared with linguistic metaphors, non-verbal metaphors (such as vision, gesture, and touch) convey metaphorical meanings through distributed brain networks. Their neural mechanisms are mainly reflected in embodied simulation and spatial coding across sensory channels. Forceville (2009) [13] reported that during the processing of visual metaphors, the object recognition function of the FG (e.g., dead tree) is activated through the occipitotemporal symphysis area. The semantic integration function of the AG is then used to bind it to abstract concepts (e.g., ecological crisis). The affective valence of visual metaphors (e.g., sense of urgency) can enhance coupling between the amygdala and the medial PFC, support emotional-semantic meaning binding, and endow individuals with emotional experiences through visual metaphors [21, 63].

A study has shown that the neural basis of gesture metaphor is closely related to the motor cortex and mirror neuron system [76]. Gestures simulate abstract concepts (e.g., ascending gesture metaphor progression) through spatial movement, activating the premotor cortex and inferior parietal lobule, and forming a cross-domain mapping of motion-semantics [77]. The synergistic activity between the right superior temporal sulcus (rSTS) and the mirror neuron system (MNS) is enhanced when individuals observe gesture metaphors, suggesting that gestures are simulated through action to facilitate metaphor understanding and support cross-subject empathy [78, 79]. The processing of tactile metaphors (e.g., silk texture metaphor silky experience) relies on the cross-modal integration of somatosensory cortex and insula [21]. The association between tactile stimuli and abstract concepts has been found to activate the texture representation area of the somatosensory cortex, with the insula converting tactile valence into emotional meaning through interoception [80, 81]. The above-mentioned research on the neural mechanism of non-verbal metaphors reveals that human understanding of abstract thinking, such as bodily action and perceptual experience, does not rely solely on the transmission of literal information, but also builds abstract concepts through cross-domain simulation of sensorimotor systems [82] to form more efficient embodied cognitive shortcuts. This metaphorical path of “body before language” may be neural evidence that human beings had the ability to think abstractly before the birth of language.

Studies have revealed that metaphors, as the core carrier of human abstract thinking, have a common architecture and channel-specific division of labor in their neural mechanism. Whether it is the left hemisphere semantic network (MTG-AG’s cross-domain mapping) or the distributed sensory system called by non-verbal metaphors (visual integration of occipitotemporal symphysis-angular gyrus, gesture simulation of motor cortex-mirror neurons, and tactile transformation of somatosensory cortex-insula), the processing process shows a similar cognitive neural logic. First, it relies on the embodied experiential activation/simulation of perceptual or motor systems, then it extracts and abstracts the core semantic features, and finally it realizes the dynamic mapping and meaning reconstruction between different conceptual domains. This commonality is mainly reflected in two aspects. First, the central pivotal role of prefrontal executive control, which manifests in the emotional valence of PFC inhibition of literal interference in linguistic metaphors and the integration of medial prefrontal cortex (mPFC) in non-verbal metaphors [21, 71]. This suggests that higher-order cognitive regulation is a necessary condition for cross-channel metaphor comprehension. Second, the cross-modal plasticity of brain region functions, such as AG, binds abstract concepts in linguistic metaphors, while integrating image symbolism in nonverbal visual metaphors [58, 63]. This highlights the universal integration mechanism of multimodal semantic hubs. However, channel differences between different modalities shape the specific division of labor of metaphorical processing paths. Specifically, language metaphors rely more on hierarchical semantic networks, such as accurate mapping dominated by language regions in the left hemisphere. Moreover, their ERP characteristics usually manifest as obvious N400 effects (reflecting the difficulty of initial semantic integration) and subsequent LPC/P600 (reflecting higher-order integration/refactoring) under specific conditions (e.g., novel metaphors, weak contexts) [71]. Nonverbal metaphors tend to be spatial-emotional parallel processing, that is, conceptual metaphors are activated through spatial coding in the motor cortex, while emotional meaning is transformed through insula sensations [76, 81]. Although the processing of nonverbal metaphors may be more dependent on sensorimotor channels, this does not imply a complete absence or general weakening of the semantic integration monitoring link (similar to N400). Indeed, numerous studies have shown that under certain conditions (e.g., when nonverbal symbols conflict with their intended or conventional symbolism, or tasks that require explicit semantic judgment), the semantic integration of nonverbal stimuli such as visual images and gestures can induce negative wave components similar to linguistic N400, usually in the temporoparietal region [83, 84, 85]. This suggests that cross-modal semantic integration processes may share certain mechanisms at the neuroelectrophysiological level.

3.2 Brain Network Synergy and Neuroplasticity Across Modal Metaphors

Efficient processing of multimodal metaphors depends on the dynamic coordination of distributed brain networks. The mechanisms include multimodal hub integration, real-time synchronization of high-frequency neural oscillations (Gamma band), and conflict regulation and resource allocation of PFC. These show significant group adaptability and compensatory performance in neural integration.

The resting-state functional connectivity study showed that the AG, as the core hub of multimodal semantic integration, can perform topological mapping of cross-modal information by connecting the visual, auditory and somatosensory cortices. Furthermore, its functional connectivity strength is positively correlated with the metaphor comprehension efficiency [20]. A neural oscillation study provide temporal dynamic evidence for the integration of specific types of multimodal information. For example, He et al. (2018) [86] used synchronous electroencephalogram (EEG)-fMRI technology to show the gamma band activity of the parietal-temporal cortex network was significantly enhanced when understanding metaphorical gestures and verbal synergistic expressions involving abstract concepts. This synchronization enhancement of high-frequency neural oscillations is thought to be a key neural mechanism that supports real-time binding and integration of cross-modal metaphorical information, such as gesture space coding and abstract semantics [86]. However, it should be noted that most current research on neural oscillations (including the cited literature) primarily focuses on the integration within or between consistent and complementary modalities (such as matching gesture-speech pairs). Whether and how gamma oscillations play a similar binding role in more diverse and complex contexts of multimodal metaphors (such as image-text metaphors, or tone-text metaphors with conflicting information) still needs to be explored in depth. In addition, metaphor comprehension relies on the dynamic adjustment of contextual information, and thus requires dynamic coordination between the left hemisphere language region (e.g., MTG and AG) and the right hemisphere associative network (e.g., temporoparietal joint area). A recent study has shown that synchronization enhancement of Gamma oscillations during metaphor generation is significantly and negatively correlated with the novelty of metaphors [87]. Although this discovery originates from a single-modal (language) task, the high-frequency neural oscillations that it reveals support a cross-domain mapping mechanism, providing theoretical support for the pivotal role of AG in multimodal metaphorical integration. Further work is required to verify whether Gamma activities perform similar functions in multi-channel information binding through cross-modal paradigms, such as audio-visual metaphorical conflict design.

This dynamic synergy mechanism has also been further validated in cross-cultural neural mechanism research, with the functional connectivity strength of default mode network (DMN, precuneus [PCUN], inferior parietal lobule [IPL]) and dorsal attention network (DAN, intraparietal sulcus [IPS]) being significantly higher in East Asian subjects than in Western subjects [61, 88]. This result suggests cultural experience is also involved in neural representation processes that modulate brain network synergistic patterns to shape metaphorical understanding. Specifically, the neural representation of context-dependent metaphor processing (e.g., using water as a metaphor), characterized by a DMN and the DAN, supports global semantic association. The DMN is the main brain region, including the medial prefrontal cortex, posterior cingulate cortex, anterior cuneineus, and inferior parietal lobules. Goal-oriented metaphor processing (e.g., time is money) activates the ventral attention network (VAN) and left prefrontal lobe [22, 89, 90, 91].

This synergistic integration mechanism plays an important role in the processing of metaphors in normal groups. However, due to cognitive or physiological deficits, special populations are unable to invoke the same integration mechanism to process metaphors in a similar manner to ordinary people. These populations use neuroplasticity to achieve the compensatory integration of other brain functional tissues in order to understand and process metaphors. For example, the metaphor processing mechanism of patients with aphasia was found to have enhanced compensatory activation on the right side of the rSTS and inferior frontal gyrus (IFG) [92], suggesting that non-verbal channels can reconstruct semantic associations through the right hemisphere network. This conclusion is also supported by the study of autism groups. Pierno et al. (2006) [93] found that autistic patients rely on visual-tactile channels to reconstruct semantic networks due to deficits in linguistic metaphor comprehension. Activation of the right parietal cortex of these individuals was enhanced when they graphed social rules through tactile flow. The processing of metaphorical information also relies on the compensatory integration of non-verbal channels. Recent research has shown that compensatory efficiency in children with autism is regulated by metaphorical significance. Specifically, a highly dominant metaphor can induce early cross-modal coupling in the right temporo-parietal symphysis region while significantly reducing the N400 amplitude, whereas low-dominant metaphors continue to induce abnormal late positive components (LPCs), reflecting the group’s integration barriers to abstract semantics [94]. The deaf-mute group activates the right parietal lobe spatial network through spatial metaphors in sign language, such as the gestural direction of “time flow” [28]. When congenitally deaf and mute people process sign language metaphors, the activity intensity of the rSTS and bilateral fusiform gyrus (bFG) is significantly stronger than that of people with normal hearing, and the compensatory visual-motor pathway is also more advantageous [95]. In addition, haptic-vibrating devices can convert speech prosody into tactile rhythm (e.g., high-frequency vibrations are used to express excitement) and activate the left somatosensory cortex and supplementary motor area (SMA) in deaf-mutes, inducing cross-modal plasticity recombination [96].

These compensatory integration mechanisms based on special populations complement and enrich our understanding of the brain network synergy mechanism of cross-modal metaphors. The studies also show the human brain can achieve cross-modal integration of metaphor information through dynamic reorganization of brain networks, such as right parietal lobe compensation and somatosensory-motor pathway enhancement. Furthermore, the compensatory strategies of different groups are specific, such as bFG dominance in deaf-mutes. The discovery and description of this mechanism has expanded the connotation of embodied cognition theory, while also providing a possible neuroscientific basis for the development of educational interventions and rehabilitation technologies (e.g., tactile and vibration devices) for special groups.

4. Summary and Future Research Prospects

This review has systematically described the neural processing mechanisms and main influencing factors of multimodal metaphorical information in daily communication situations. In particular, it outlines the neural differentiation and cross-modal integration mechanisms of verbal and non-verbal modalities, and explains how special groups (e.g., autism, aphasia, deaf and mute) achieve compensatory metaphorical processing through neuroplasticity. Some correspondences between brain structures and functions related to metaphor processing are shown in Table 1 (Ref. [13, 20, 21, 23, 24, 25, 27, 58, 59, 62, 63, 71, 75, 77, 78, 80, 81, 93, 94]). By analyzing the neural basis of cross-modal dynamic integration in metaphor processing, such as the embodied simulation mechanism of mirror neuron system and the semantic association function of default pattern network, researchers can systematically elucidate the neural enhancement path of language comprehension efficacy in interpersonal interactions. Through the synergistic suggestion of verbal and non-verbal modalities, the two sides of the dialogue can activate the shared mirror neuron system and the insula-prefrontal emotional circuit. This realizes the deduction of thinking patterns, empathy of psychological states, and the docking of cultural schemas, thus significantly improving the communication efficiency and quality of social interaction [97]. The current review not only elucidates the neural mechanism by which humans receive and process information from others, but also deepens our understanding of multimodal metaphor processing by special groups. Although significant progress has been made in understanding the cognitive neural mechanisms of multimodal metaphors, many challenges remain in its theoretical construction and practical application. In the future, it is necessary to further promote interdisciplinary integration exploration in the following directions.

First, the temporal neural mechanism of dynamic metaphor processing requires further clarification. In future studies, virtual reality (VR) and high-precision timing technology (such as cross-band oscillation analysis) could be combined to further explore how the neural resource competition mechanism of cross-channel metaphors (such as interhemispheric inhibition during language-gesture integration) can reshape the neural coding efficiency of multi-channel metaphors. This may also reveal the integration law of multimodal information (such as the spatiotemporal trajectory of gestures, the prosodic fluctuation of speech, and the real-time feedback of haptics) within millisecond-level time windows, as well as how the phase synchronization of Gamma band oscillation supports rapid binding of cross-modal information [86]. Predictive coding mechanisms in metaphor processing, such as how the brain preactivates possible metaphorical meanings based on context, and their interactions with mirror neuron systems, may also be pivotal to understanding dynamic comprehension processes [97]. Therefore, future research should begin from the perspective of “hierarchical metaphor processing”, and then delve into metaphor processing mechanisms from multiple dimensions (e.g., temporal dynamics, spatial network, and cognitive complexity), thus promoting theoretical integration and application innovation.

Secondly, the intervention strategy for multimodal metaphor processing needs to be extended from the laboratory to the ecological context. At present, exploration of the compensatory mechanisms for autism, aphasia, and deaf-mute groups is mostly limited to control tasks, with multimodal metaphor processing (such as gesture-language synergy in dialogue) in actual social situations requiring further investigation. In the future, ecological intervention technologies based on neuroplasticity should be developed. These could include simulation of complex communication scenarios through multimodal virtual social platforms, developing real-time neurofeedback systems combined with brain-computer interface technology, and training patients to reconstruct metaphor networks using non-verbal channels (e.g., visual flow diagrams, tactile vibration cues) [96, 98], thereby achieving individualized adaptation of the intervention program. Additionally, longitudinal follow-up studies of individuals at different ages are needed to reveal causal associations between the intervention and brain network reorganization (e.g., enhancement of the right temporoparietal symphysis), and to evaluate its long-term effects [99]. Therefore, future research should focus on the multi-group adaptive and individualized neural map of compensatory neural mechanisms, develop multimodal intervention strategies with low cognitive load and high social situation adaptability for special groups of different ages and different characteristics, and explore their roles in advanced social cognitive functions (e.g., emotional resonance, intention inference, and negotiated decision-making). Such research is essential to proceed beyond the current limitations, unleash the full potential of compensatory mechanisms, and ultimately enable educational innovation, rehabilitation practice, and human-computer intelligent interaction.

Finally, technology convergence and computational modeling will provide a new paradigm for multimodal metaphor research. On the one hand, artificial intelligence can draw on the cross-modal integration mechanism of the human brain (such as the semantic binding function of the AG) to develop a multimodal metaphor generation model based on deep learning. This should improve the ability of natural language processing systems to parse graphic, textual and phonetic metaphors [97]. On the other hand, emotional metaphors establish a cognitive bridge between abstract emotions and concrete experiences through embodied emotional representations (e.g., “heart like a knife” and “burning in anger”), and their processing is deeply coupled with the neural basis of empathy (e.g., mirror neuron system, prefrontal-limbic circuit) [21]. Therefore, affective computing can be combined with compensatory mechanisms of the tactile-visual channel (e.g., vibrational feedback in the deaf-mute group) to design more empathetic human-computer interfaces [28]. At the theoretical level, further studies are needed to integrate embodied cognition, extended cognition, and predictive coding frameworks, as well as the construction of a metaphor processing model at the level of “perception-simulation-prediction” to explain the neural mechanism of the whole process, from low-order sensory input to higher-order semantic linkage [100].

In short, research on multimodal metaphors needs to strengthen interdisciplinary collaboration, combining neuroscience, linguistics, computational modeling and technical engineering to further uncover the hierarchical coding mechanisms of metaphors in the brain. Future research should aim to convert theoretical achievements into practical applications for educational innovation, clinical rehabilitation, and human-computer collaboration, thereby truly serving the needs of human cognition and social development.

References

[1] Johnson-Laird PN. Mental models in cognitive science. Cognitive Science. 1980; 4: 71–115. https://doi.org/10.1016/S0364-0213(81)80005-5.

[2] Lakoff G, Johnson M. The metaphorical structure of the human conceptual system. Cognitive Science. 1980; 4: 195–208. https://doi.org/10.1016/S0364-0213(80)80017-6.

[3] Wilson-Mendenhall CD, Barrett LF, Simmons WK, Barsalou LW. Grounding emotion in situated conceptualization. Neuropsychologia. 2011; 49: 1105–1127. https://doi.org/10.1016/j.neuropsychologia.2010.12.032.

[4] Gibbs Jr RW. Metaphor wars: Conceptual metaphors in human life. Cambridge University Press: Cambridge, United Kingdom. 2017. https://doi.org/10.1017/9781107762350.

[5] Hutchinson S, Louwerse M. Extracting social networks from language statistics. Discourse Processes. 2018; 55: 607–618. https://doi.org/10.1080/0163853X.2017.1332446.

[6] Desai RH. Are metaphors embodied? The neural evidence. Psychological Research. 2022; 86: 2417–2433. https://doi.org/10.1007/s00426-021-01604-4.

[7] Veale T, Shutova E, Klebanov BB. Metaphor: A computational perspective. Springer Nature: Cham, Switzerland. 2022.

[8] Zhao Q, Ahrens K, Huang C-R. Linguistic synesthesia is metaphorical: a lexical-conceptual account. Cognitive Linguistics. 2022; 33: 553–583. https://doi.org/10.1515/cog-2021-0098.

[9] Forceville C. Pictorial metaphor in advertisements. Metaphor and Symbolic Activity. 1994; 9: 1–29. https://doi.org/10.1207/s15327868ms0901_1.

[10] McGregor KK, Rohlfing KJ, Bean A, Marschner E. Gesture as a support for word learning: the case of under. Journal of Child Language. 2009; 36: 807–828. https://doi.org/10.1017/S0305000908009173.

[11] Kita S, Emmorey K. Gesture links language and cognition for spoken and signed languages. Nature Reviews Psychology. 2023; 2: 407–420. https://doi.org/10.1038/s44159-023-00186-9.

[12] Alibali MW, Nathan MJ. Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures. Journal of the Learning Sciences. 2012; 21: 247–286. https://doi.org/10.1080/10508406.2011.611446.

[13] Forceville C. The role of non-verbal sound and music in multimodal metaphor. In Multimodal Metaphor (pp. 383–400). De Gruyter Mouton: Berlin, Germany. 2009.

[14] Kou G, Liang Y. A comparative study of multi-modal metaphors in food advertisements. Semiotica. 2022; 2022: 275–291. https://doi.org/10.1515/sem-2020-0117.

[15] Jahameh H, Zibin A. The use of monomodal and multimodal metaphors in advertising Jordanian and American food products on Facebook: A comparative study. Heliyon. 2023; 9: e15178. https://doi.org/10.1016/j.heliyon.2023.e15178.

[16] Stöckl H. Detecting generic patterns in multimodal argumentation: A corpus-based study of environmental protection print-ads. Journal of Argumentation in Context. 2024; 13: 260–291. https://doi.org/10.1075/jaic.00030.sto.

[17] Eckert P. Communities of practice. Encyclopedia of Language and Linguistics. Elsevier Perspective, USA Empowerment in Organizations. 2006; 6: 177–186.

[18] Günther F, Rinaldi L, Marelli M. Vector-Space Models of Semantic Representation From a Cognitive Perspective: A Discussion of Common Misconceptions. Perspectives on Psychological Science: a Journal of the Association for Psychological Science. 2019; 14: 1006–1033. https://doi.org/10.1177/1745691619861372.

[19] Pelkey J. Embodiment and language. Wiley Interdisciplinary Reviews. Cognitive Science. 2023; 14: e1649. https://doi.org/10.1002/wcs.1649.

[20] Binder JR, Desai RH, Graves WW, Conant LL. Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cerebral Cortex (New York, N.Y.: 1991). 2009; 19: 2767–2796. https://doi.org/10.1093/cercor/bhp055.

[21] Lacey S, Stilla R, Sathian K. Metaphorically feeling: comprehending textural metaphors activates somatosensory cortex. Brain and Language. 2012; 120: 416–421. https://doi.org/10.1016/j.bandl.2011.12.016.

[22] Holyoak KJ, Stamenković D. Metaphor comprehension: A critical review of theories and evidence. Psychological Bulletin. 2018; 144: 641–671. https://doi.org/10.1037/bul0000145.

[23] Hamilton AFDC. Reflecting on the mirror neuron system in autism: a systematic review of current theories. Developmental Cognitive Neuroscience. 2013; 3: 91–105. https://doi.org/10.1016/j.dcn.2012.09.008.

[24] Yenkoyan K, Grigoryan A, Fereshetyan K, Yepremyan D. Advances in understanding the pathophysiology of autism spectrum disorders. Behavioural Brain Research. 2017; 331: 92–101. https://doi.org/10.1016/j.bbr.2017.04.038.

[25] Hope TMH, Leff AP, Prejawa S, Bruce R, Haigh Z, Lim L, et al. Right hemisphere structural adaptation and changing language skills years after left hemisphere stroke. Brain: a Journal of Neurology. 2017; 140: 1718–1728. https://doi.org/10.1093/brain/awx086.

[26] Gajardo-Vidal A, Lorca-Puls DL, Hope TMH, Parker Jones O, Seghier ML, Prejawa S, et al. How right hemisphere damage after stroke can impair speech comprehension. Brain: a Journal of Neurology. 2018; 141: 3389–3404. https://doi.org/10.1093/brain/awy270.

[27] Kourtidou E, Kasselimis D, Angelopoulou G, Karavasilis E, Velonakis G, Kelekis N, et al. The Role of the Right Hemisphere White Matter Tracts in Chronic Aphasic Patients After Damage of the Language Tracts in the Left Hemisphere. Frontiers in Human Neuroscience. 2021; 15: 635750. https://doi.org/10.3389/fnhum.2021.635750.

[28] Emmorey K, McCullough S, Brentari D. Categorical perception in American sign language. Language and Cognitive Processes. 2003; 18: 21–45. https://doi.org/10.1080/01690960143000416.

[29] Poeppel D, Emmorey K, Hickok G, Pylkkänen L. Towards a new neurobiology of language. The Journal of Neuroscience: the Official Journal of the Society for Neuroscience. 2012; 32: 14125–14131. https://doi.org/10.1523/JNEUROSCI.3244-12.2012.

[30] Sohoglu E, Peelle JE, Carlyon RP, Davis MH. Predictive top-down integration of prior knowledge during speech perception. The Journal of Neuroscience: the Official Journal of the Society for Neuroscience. 2012; 32: 8443–8453. https://doi.org/10.1523/JNEUROSCI.5069-11.2012.

[31] Kövecses Z. Metaphor in culture: Universality and variation. Cambridge university press: Cambridge, United Kingdom. 2005.

[32] Gallese V, Lakoff G. The Brain’s concepts: the role of the Sensory-motor system in conceptual knowledge. Cognitive Neuropsychology. 2005; 22: 455–479. https://doi.org/10.1080/02643290442000310.

[33] Zhong K, Wang Y, Wang H. Sense hardness: Effect of haptic perception on consumer attitudes towards brand extension. Journal of Consumer Behaviour. 2021; 20: 535–549. https://doi.org/10.1002/cb.1883.

[34] Hinojosa JA, Moreno EM, Ferré P. Affective neurolinguistics: towards a framework for reconciling language and emotion. Language, Cognition and Neuroscience. 2020; 35: 813–839. https://doi.org/10.1080/23273798.2019.1620957.

[35] Tribot A-S, Blanc N, Brassac T, Guilhaumon F, Casajus N, Mouquet N. What makes a teddy bear comforting? A participatory study reveals the prevalence of sensory characteristics and emotional bonds in the perception of comforting teddy bears. The Journal of Positive Psychology. 2024; 19: 379–392. https://doi.org/10.1080/17439760.2023.2170273.

[36] Hartung F, Kenett YN, Cardillo ER, Humphries S, Klooster N, Chatterjee A. Context matters: Novel metaphors in supportive and non-supportive contexts. NeuroImage. 2020; 212: 116645. https://doi.org/10.1016/j.neuroimage.2020.116645.

[37] Diaz MT, Hogstrom LJ. The influence of context on hemispheric recruitment during metaphor processing. Journal of Cognitive Neuroscience. 2011; 23: 3586–3597. https://doi.org/10.1162/jocn_a_00053.

[38] McNally L. Semantics and pragmatics. Wiley Interdisciplinary Reviews. Cognitive Science. 2013; 4: 285–297. https://doi.org/10.1002/wcs.1227.

[39] Broeker L, Ewolds H, de Oliveira RF, Künzell S, Raab M. Additive Effects of Prior Knowledge and Predictive Visual Information in Improving Continuous Tracking Performance. Journal of Cognition. 2020; 3: 40. https://doi.org/10.5334/joc.130.

[40] Chwilla DJ. Context effects in language comprehension: The role of emotional state and attention on semantic and syntactic processing. Frontiers in Human Neuroscience. 2022; 16: 1014547. https://doi.org/10.3389/fnhum.2022.1014547.

[41] Bambini V, Bertini C, Schaeken W, Stella A, Di Russo F. Disentangling Metaphor from Context: An ERP Study. Frontiers in Psychology. 2016; 7: 559. https://doi.org/10.3389/fpsyg.2016.00559.

[42] Stróżak P, Abedzadeh D, Curran T. Separating the FN400 and N400 potentials across recognition memory experiments. Brain Research. 2016; 1635: 41–60. https://doi.org/10.1016/j.brainres.2016.01.015.

[43] Spotorno N, Koun E, Prado J, Van Der Henst JB, Noveck IA. Neural evidence that utterance-processing entails mentalizing: the case of irony. NeuroImage. 2012; 63: 25–39. https://doi.org/10.1016/j.neuroimage.2012.06.046.

[44] Akimoto Y, Sugiura M, Yomogida Y, Miyauchi CM, Miyazawa S, Kawashima R. Irony comprehension: social conceptual knowledge and emotional response. Human Brain Mapping. 2014; 35: 1167–1178. https://doi.org/10.1002/hbm.22242.

[45] Bohrn IC, Altmann U, Jacobs AM. Looking at the brains behind figurative language–a quantitative meta-analysis of neuroimaging studies on metaphor, idiom, and irony processing. Neuropsychologia. 2012; 50: 2669–2683. https://doi.org/10.1016/j.neuropsychologia.2012.07.021.

[46] Chuikova ZV, Filatov AA, Faber AY, Arsalidou M. Mapping common and distinct brain correlates among cognitive flexibility tasks: concordant evidence from meta-analyses. Brain Imaging and Behavior. 2025; 19: 50–71. https://doi.org/10.1007/s11682-024-00921-7.

[47] Bar M. The proactive brain: using analogies and associations to generate predictions. Trends in Cognitive Sciences. 2007; 11: 280–289. https://doi.org/10.1016/j.tics.2007.05.005.

[48] Ufer C, Blank H. Multivariate analysis of brain activity patterns as a tool to understand predictive processes in speech perception. Language, Cognition and Neuroscience. 2024; 39: 1117–1133. https://doi.org/10.1080/23273798.2023.2166679.

[49] Federmeier KD, Wlotko EW, De Ochoa-Dewald E, Kutas M. Multiple effects of sentential constraint on word processing. Brain Research. 2007; 1146: 75–84. https://doi.org/10.1016/j.brainres.2006.06.101.

[50] Grant A, Grey S, van Hell JG. Male fashionistas and female football fans: Gender stereotypes affect neurophysiological correlates of semantic processing during speech comprehension. Journal of Neurolinguistics. 2020; 53: 100876. https://doi.org/10.1016/j.jneuroling.2019.100876.

[51] Kozhevnikov M, Evans C, Kosslyn SM. Cognitive Style as Environmentally Sensitive Individual Differences in Cognition: A Modern Synthesis and Applications in Education, Business, and Management. Psychological Science in the Public Interest: a Journal of the American Psychological Society. 2014; 15: 3–33. https://doi.org/10.1177/1529100614525555.

[52] Aglioti SM, Cesari P, Romani M, Urgesi C. Action anticipation and motor resonance in elite basketball players. Nature Neuroscience. 2008; 11: 1109–1116. https://doi.org/10.1038/nn.2182.

[53] Pérez-Sobrino P. Multimodal metaphor and metonymy in advertising: A corpus-based account. Metaphor and Symbol. 2016; 31: 73–90. https://doi.org/10.1080/10926488.2016.1150759.

[54] Amedi A, Merabet LB, Bermpohl F, Pascual-Leone A. The occipital cortex in the blind: Lessons about plasticity and vision. Current Directions in Psychological Science. 2005; 14: 306–311. https://doi.org/10.1111/j.0963-7214.2005.00387.x.

[55] Striem-Amit E, Wang X, Bi Y, Caramazza A. Neural representation of visual concepts in people born blind. Nature Communications. 2018; 9: 5250. https://doi.org/10.1038/s41467-018-07574-3.

[56] Goldin-Meadow S. Hearing gesture: How our hands help us think. Harvard University Press: Cambridge, MA. 2005.

[57] Grandin T. How does visual thinking work in the mind of a person with autism? A personal account. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences. 2009; 364: 1437–1442. https://doi.org/10.1098/rstb.2008.0297.

[58] Huang Y, Huang J, Li L, Lin T, Zou L. Neural network of metaphor comprehension: an ALE meta-analysis and MACM analysis. Cerebral Cortex (New York, N.Y.: 1991). 2023; 33: 10918–10930. https://doi.org/10.1093/cercor/bhad337.

[59] Benedek M, Beaty R, Jauk E, Koschutnig K, Fink A, Silvia PJ, et al. Creating metaphors: the neural basis of figurative language production. NeuroImage. 2014; 90: 99–106. https://doi.org/10.1016/j.neuroimage.2013.12.046.

[60] He Y, Steines M, Sommer J, Gebhardt H, Nagels A, Sammer G, et al. Spatial-temporal dynamics of gesture-speech integration: a simultaneous EEG-fMRI study. Brain Structure & Function. 2018; 223: 3073–3089. https://doi.org/10.1007/s00429-018-1674-5.

[61] Wang Q. Neural mechanism and representation of English and Chinese metaphors of bilinguals with different second language proficiency: An ERP study. Chinese Journal of Applied Linguistics. 2018; 41: 67–83. https://doi.org/10.1515/cjal-2018-0004.

[62] Han S, Northoff G. Culture-sensitive neural substrates of human cognition: a transcultural neuroimaging approach. Nature Reviews. Neuroscience. 2008; 9: 646–654. https://doi.org/10.1038/nrn2456.

[63] Duque ACM, Cuesta TAC, Melo ADS, Maldonado IL. Right hemisphere and metaphor comprehension: A connectionist perspective. Neuropsychologia. 2023; 187: 108618. https://doi.org/10.1016/j.neuropsychologia.2023.108618.

[64] Shen L, Li X, Huang S, Huang Y, Gao X, You Z, et al. Comprehending scientific metaphors in the bilingual brain: Evidence from event-related potentials. Frontiers in Psychology. 2022; 13: 1037525. https://doi.org/10.3389/fpsyg.2022.1037525.

[65] Yang J, Wang X, Shu H, Zevin JD. Task by stimulus interactions in brain responses during Chinese character processing. NeuroImage. 2012; 60: 979–990. https://doi.org/10.1016/j.neuroimage.2012.01.036.

[66] Brouwer H, Fitz H, Hoeks J. Getting real about semantic illusions: rethinking the functional role of the P600 in language comprehension. Brain Research. 2012; 1446: 127–143. https://doi.org/10.1016/j.brainres.2012.01.055.

[67] Citron FMM, Goldberg AE. Metaphorical sentences are more emotionally engaging than their literal counterparts. Journal of Cognitive Neuroscience. 2014; 26: 2585–2595. https://doi.org/10.1162/jocn_a_00654.

[68] Bayer M, Sommer W, Schacht A. Reading emotional words within sentences: the impact of arousal and valence on event-related potentials. International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology. 2010; 78: 299–307. https://doi.org/10.1016/j.ijpsycho.2010.09.004.

[69] Shen W, Fiori-Duharcourt N, Isel F. Functional significance of the semantic P600: evidence from the event-related brain potential source localization. Neuroreport. 2016; 27: 548–558. https://doi.org/10.1097/WNR.0000000000000583.

[70] Zheng X, Lemhöfer K. The “semantic P600” in second language processing: When syntax conflicts with semantics. Neuropsychologia. 2019; 127: 131–147. https://doi.org/10.1016/j.neuropsychologia.2019.02.010.

[71] Bambini V, Gentili C, Ricciardi E, Bertinetto PM, Pietrini P. Decomposing metaphor processing at the cognitive and neural level through functional magnetic resonance imaging. Brain Research Bulletin. 2011; 86: 203–216. https://doi.org/10.1016/j.brainresbull.2011.07.015.

[72] Cardillo ER, Watson CE, Schmidt GL, Kranjec A, Chatterjee A. From novel to familiar: tuning the brain for metaphors. NeuroImage. 2012; 59: 3212–3221. https://doi.org/10.1016/j.neuroimage.2011.11.079.

[73] Bakker I, Takashima A, van Hell JG, Janzen G, McQueen JM. Tracking lexical consolidation with ERPs: Lexical and semantic-priming effects on N400 and LPC responses to newly-learned words. Neuropsychologia. 2015; 79: 33–41. https://doi.org/10.1016/j.neuropsychologia.2015.10.020.

[74] Abraham A, Rutter B, Hermann C. Conceptual expansion via novel metaphor processing: An ERP replication and extension study examining individual differences in creativity. Brain and Language. 2021; 221: 105007. https://doi.org/10.1016/j.bandl.2021.105007.

[75] Zhu J, Chen H, Cong F, Ma J. The role of executive control ability in second language metaphor comprehension: Evidence from ERPs and sLORETA. Journal of Neurolinguistics. 2024; 72: 101211. https://doi.org/10.1016/j.jneuroling.2024.101211.

[76] Gallese V. Bodily selves in relation: embodied simulation as second-person perspective on intersubjectivity. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences. 2014; 369: 20130177. https://doi.org/10.1098/rstb.2013.0177.

[77] Lopers I. What is True About the Mirror Neuron System; Reflecting on Previous Evidence Regarding Mirror Neuron Function, Empathy and Autism Spectrum Disorder [Doctoral]. University of Groningen: Groningen, The Netherlands. 2024.

[78] Rizzolatti G, Sinigaglia C. The functional role of the parieto-frontal mirror circuit: interpretations and misinterpretations. Nature Reviews. Neuroscience. 2010; 11: 264–274. https://doi.org/10.1038/nrn2805.

[79] Cacciante L, Pregnolato G, Salvalaggio S, Federico S, Kiper P, Smania N, et al. Language and gesture neural correlates: A meta-analysis of functional magnetic resonance imaging studies. International Journal of Language & Communication Disorders. 2024; 59: 902–912. https://doi.org/10.1111/1460-6984.12987.

[80] Sathian K, Lacey S, Stilla R, Gibson GO, Deshpande G, Hu X, et al. Dual pathways for haptic and visual perception of spatial and texture information. NeuroImage. 2011; 57: 462–475. https://doi.org/10.1016/j.neuroimage.2011.05.001.

[81] Giraud M, Zapparoli L, Basso G, Petilli M, Paulesu E, Nava E. Mapping the emotional homunculus with fMRI. iScience. 2024; 27: 109985. https://doi.org/10.1016/j.isci.2024.109985.

[82] Zheng WQ, Liu Y, Fu XL. Cognitive and neural mechanisms of sensory-motor system’s role in metaphor comprehension. Progress in Biochemistry and Biophysics. 2018; 45: 325–335. https://doi.org/10.16476/j.pibb.2017.0141.

[83] West WC, Holcomb PJ. Event-related potentials during discourse-level semantic integration of complex pictures. Brain Research. Cognitive Brain Research. 2002; 13: 363–375. https://doi.org/10.1016/s0926-6410(01)00129-x.

[84] Wu YC, Coulson S. Meaningful gestures: electrophysiological indices of iconic gesture comprehension. Psychophysiology. 2005; 42: 654–667. https://doi.org/10.1111/j.1469-8986.2005.00356.x.

[85] Ortiz MJ, Grima Murcia MD, Fernandez E. Brain processing of visual metaphors: An electrophysiological study. Brain and Cognition. 2017; 113: 117–124. https://doi.org/10.1016/j.bandc.2017.01.005.

[86] He Y, Nagels A, Schlesewsky M, Straube B. The Role of Gamma Oscillations During Integration of Metaphoric Gestures and Abstract Speech. Frontiers in Psychology. 2018; 9: 1348. https://doi.org/10.3389/fpsyg.2018.01348.

[87] Yu Y, Krebs L, Beeman M, Lai VT. Hidden Brain States Reveal the Temporal Dynamics of Neural Oscillations During Metaphor Generation and Their Role in Verbal Creativity. Psychophysiology. 2025; 62: e70023. https://doi.org/10.1111/psyp.70023.

[88] Gutchess AH, Welsh RC, Boduroglu A, Park DC. Cultural differences in neural function associated with object processing. Cognitive, Affective & Behavioral Neuroscience. 2006; 6: 102–109. https://doi.org/10.3758/cabn.6.2.102.

[89] Jenkins LJ, Yang YJ, Goh J, Hong YY, Park DC. Cultural differences in the lateral occipital complex while viewing incongruent scenes. Social Cognitive and Affective Neuroscience. 2010; 5: 236–241. https://doi.org/10.1093/scan/nsp056.

[90] Turker S, Kuhnke P, Eickhoff SB, Caspers S, Hartwigsen G. Cortical, subcortical, and cerebellar contributions to language processing: A meta-analytic review of 403 neuroimaging experiments. Psychological Bulletin. 2023; 149: 699–723. https://doi.org/10.1037/bul0000403.

[91] Ouerchefani R, Ouerchefani N, Ben Rejeb MR, Le Gall D. Pragmatic language comprehension: Role of theory of mind, executive functions, and the prefrontal cortex. Neuropsychologia. 2024; 194: 108756. https://doi.org/10.1016/j.neuropsychologia.2023.108756.

[92] Turkeltaub PE, Coslett HB, Thomas AL, Faseyitan O, Benson J, Norise C, et al. The right hemisphere is not unitary in its role in aphasia recovery. Cortex; a Journal Devoted to the Study of the Nervous System and Behavior. 2012; 48: 1179–1186. https://doi.org/10.1016/j.cortex.2011.06.010.

[93] Pierno AC, Mari M, Glover S, Georgiou I, Castiello U. Failure to read motor intentions from gaze in children with autism. Neuropsychologia. 2006; 44: 1483–1488. https://doi.org/10.1016/j.neuropsychologia.2005.11.013.

[94] Cheng L, Wang X, Mao H, Liu Y, Yuan W, Wang P, et al. Influence of Salience on Neural Responses in Metaphor Processing of Chinese Children with Autism: Evidence from ERPs. Journal of Autism and Developmental Disorders. 2025. https://doi.org/10.1007/s10803-025-06915-8. (online ahead of print)

[95] Papanicolaou AC. Non-Invasive Mapping of the Neuronal Networks of Language. Brain Sciences. 2023; 13: 1457. https://doi.org/10.3390/brainsci13101457.

[96] Damera SR, Malone PS, Stevens BW, Klein R, Eberhardt SP, Auer ET, et al. Metamodal Coupling of Vibrotactile and Auditory Speech Processing Systems through Matched Stimulus Representations. The Journal of Neuroscience: the Official Journal of the Society for Neuroscience. 2023; 43: 4984–4996. https://doi.org/10.1523/JNEUROSCI.1710-22.2023.

[97] Schneider RM, Sullivan J, Marušič F, Žaucer R, Biswas P, Mišmaš P, et al. Do children use language structure to discover the recursive rules of counting? Cognitive Psychology. 2020; 117: 101263. https://doi.org/10.1016/j.cogpsych.2019.101263.

[98] Benetti S, Collignon O. Cross-modal integration and plasticity in the superior temporal cortex. Handbook of Clinical Neurology. 2022; 187: 127–143. https://doi.org/10.1016/B978-0-12-823493-8.00026-2.

[99] Turkeltaub PE, Messing S, Norise C, Hamilton RH. Are networks for residual language function and recovery consistent across aphasic patients? Neurology. 2011; 76: 1726–1734. https://doi.org/10.1212/WNL.0b013e31821a44c1.

[100] Lachaud CM. Conceptual metaphors and embodied cognition: EEG coherence reveals brain activity differences between primary and complex conceptual metaphors during comprehension. Cognitive Systems Research. 2013; 22: 12–26. https://doi.org/10.1016/j.cogsys.2012.08.003.

Funding

Anhui Province Universities Research Project for Distinguished Young Scholars(2023AH020041)

Key Project under the Young Faculty Development Program in Higher Education(YQZD2025041)

Quality Project of Degree and Graduate Education of Huaibei Normal University(2023jgxm005)