Integrating Information From Vision and Touch: A Neural Network Modeling Study

Perception of external events often depends on integrating different sensory information. Many studies show strong evidence for visual–tactile integrations. Understanding how visual and tactile information are merged together is still a challenging problem. Here, a neural network model was used to investigate the mechanisms underlying visual–tactile interactions. It includes two unimodal areas (visual and tactile, respectively), sending feedforward connections into a downstream bimodal area. The unimodal areas influence each other via two synaptic mechanisms: feedback synapses from the bimodal area and direct reciprocal connections. The network reproduces a variety of visual–tactile interactions: 1) detection of faint tactile stimuli is facilitated by concomitant visual input; 2) tactile spatial resolution is improved by visual information; 3) cross-modal advantages are maximum when poor unisensory information is available (inverse effectiveness); and 4) conflict situations are resolved based on the more reliable sensory cue. The model identifies distinct roles for the feedback and direct synapses: the first are fundamental to improve detection of low intensity tactile stimuli in cross-modal stimulation, and the second are mostly implicated in visual enhancement of tactile spatial localization and resolution. A better comprehension of how vision and touch interact in the neural system may contribute to physiological knowledge, clinical practice, and technological applications.