Abstract
Music is understood and enjoyed in all cultures. The experience of music in deaf individuals, without acoustic information, shows that this is a human capacity that is shared across the senses. It is not clear whether neural reorganisation allows deaf individuals to use auditory brain regions to perceive music through touch and vision, or whether the brain has modality-independent mechanisms for musical processing. The technology needed to understand these multimodal aspects of music is not available. This project will develop new technologies that can deliver salient components of music through other senses, and use neuroimaging to understand modality-independent musical processing.
References
Cardin, V., Grin, K., Vinogradova, V., & Manini, B. (2020). Crossmodal reorganisation in deafness: mechanisms for functional preservation and functional change. Neuroscience & Biobehavioral Reviews. 113: 227-237.
Cardin, V., Orfanidou, E., Rönnberg, J., Capek, C.M., Rudner, M., & Woll, B. (2013). Dissociating cognitive and sensory neural plasticity in human superior temporal cortex. Nature Communications. 4: 1413.
Newbold, J., Gold, N., & Berthouze, N. (2020). Movement Sonification Expectancy Model: Leveraging Musical Expectancy Theory to Create Movement-Altering Sonifications. Journal on Multimodal User Interfaces. 14: 153-166.
Tierney, A. and Kraus, N. (2015) Neural entrainment to the rhythmic structure of music. Journal of Cognitive Neuroscience 27 (2): 400-408.
Tajadura-Jiménez, A., Cohen, H., & Berthouze, N. Bodily Sensory Inputs and Anomalous Bodily Experiences in Complex Regional Pain Syndrome: Evaluation of the Potential Effects of Sound Feedback. (2017) Front Hum Neurosci. 11:379.