Verifying different sensory modality properties for concepts results in a processing cost known as the Modality-Switch Effect. It has been argued that this cognitive cost is the result of a perceptual simulation. This paper extends this argument and reports an experiment investigating whether the effect is the result of an activation of sensory information which can also be triggered by perceptual linguistically described stimuli. Participants were first exposed to a prime sentence describing a light or a sound’s perceptual property (e.g. “The light is flickering”, “The sound is echoing”), then required to perform a property-verification task on a target sentence (e.g. “Butter is yellowish”, “Leaves rustle”). The content modalities of the prime and target sentences could be compatible (i.e. in the same modality: e.g. visual-visual) or not (i.e. in different modalities). Crucially, we manipulated the stimuli’s presentation modality such that half of the participants was faced with written sentences while the other half was faced with aurally presented sentences. Results show a cost when two different modalities alternate, compared to when the same modality is repeated with both visual and aural stimuli presentations. This result supports the embodied and grounded cognition view which claims that conceptual knowledge is grounded into the perceptual system. Specifically, this evidence suggests that sensory modalities can be pre-activated through the simulation of either read or listened linguistic stimuli describing visual or acoustic perceptual properties.

The modality-switch effect: Visually and aurally presented prime sentences activate our senses

SCERRATI, ELISA;BARONI, GIULIA;GALATOLO, RENATA;LUGLI, LUISA;NICOLETTI, ROBERTO
2015

Abstract

Verifying different sensory modality properties for concepts results in a processing cost known as the Modality-Switch Effect. It has been argued that this cognitive cost is the result of a perceptual simulation. This paper extends this argument and reports an experiment investigating whether the effect is the result of an activation of sensory information which can also be triggered by perceptual linguistically described stimuli. Participants were first exposed to a prime sentence describing a light or a sound’s perceptual property (e.g. “The light is flickering”, “The sound is echoing”), then required to perform a property-verification task on a target sentence (e.g. “Butter is yellowish”, “Leaves rustle”). The content modalities of the prime and target sentences could be compatible (i.e. in the same modality: e.g. visual-visual) or not (i.e. in different modalities). Crucially, we manipulated the stimuli’s presentation modality such that half of the participants was faced with written sentences while the other half was faced with aurally presented sentences. Results show a cost when two different modalities alternate, compared to when the same modality is repeated with both visual and aural stimuli presentations. This result supports the embodied and grounded cognition view which claims that conceptual knowledge is grounded into the perceptual system. Specifically, this evidence suggests that sensory modalities can be pre-activated through the simulation of either read or listened linguistic stimuli describing visual or acoustic perceptual properties.
2015
Scerrati, Elisa; Baroni, Giulia; Borghi, Anna M.; Galatolo, Renata; Lugli, Luisa; Nicoletti, Roberto
File in questo prodotto:
File Dimensione Formato  
fpsyg-06-01668.pdf

accesso aperto

Tipo: Versione (PDF) editoriale
Licenza: Licenza per Accesso Aperto. Creative Commons Attribuzione (CCBY)
Dimensione 381.68 kB
Formato Adobe PDF
381.68 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/521685
Citazioni
  • ???jsp.display-item.citation.pmc??? 3
  • Scopus 18
  • ???jsp.display-item.citation.isi??? 14
social impact