CD: br-promotions.com
Pres%mulus Oscillatory Brain Ac%vity Influences the Percep%on of the McGurk-‐Effect
Julian Keil ¹ , Niklas Ihssen ² , Nathan Weisz ¹
¹University of Konstanz, ²University of Bangor
The McGurk effect is a perceptual illusion, which demonstrates an interacDon between auditory and visual sensory systems in speech percepDon. If a mismatch between the percepDon of a sound and the accompanying visual input occurs, it has been observed that the unified percepDon of both modaliDes fuses to a novel percept that neither matches the auditory nor the visual sDmulus. This effect only appears in 60-‐80% of trials containing mismatching informaDon.
The present study was designed to clarify the condiDons under which this effect occurs and to idenDfy corDcal sources associated with it. We were interested in the potenDal influence of induced ongoing brain oscillaDons associated with varying percepDon.
Trials, in which a fusion was reported were compared to trials in which the subjects responded to the visual input. Although both, congruent and incongruent trials were presented, only varying percepDon during the presentaDon of idenDcal incongruent sDmuli was analyzed.
S%muli:
* 390 videos of actor pronouncing syllables ("aba", "ada", "aga") * 2/3 of the videos dubbed with incongruent sound
* Response which syllable was perceived following each video via buYon press
* 45 minutes total duraDon Data aquisi%on:
* 148-‐channel whole-‐head MEG system (MAGNES™ 2500 WH, 4D Neuroimaging, San Diego, USA)
* 678.17 Hz sample rate
* 0.1-‐200 Hz online band pass filter
We found an increase in induced beta band acDvity before the actual percepDon of the McGurk effect in the lec Inferior Frontal Gyrus, Precuneus, and Medial Frontal Gyrus (comprising the Frontal Eye Field). This paYern of co-‐acDvaDon indicates a top-‐down modulaDon of both visual and auditory cortex. It is possible that the likelihood to perceive an unified percept – in this case the McGurk effect – depends on the state of this network as reflected in ongoing oscillatory corDcal acDvity. Further, certain expectaDons may shape the funcDonal architecture of this network to enhance the processing of perceptual features.
We also found an increase in induced beta band acDvity during the percepDon of the McGurk effect in the Precuneus and Middle Frontal Gyrus (comprising the Frontal Eye Field). This replicates earlier findings on the role of higher-‐order processing stages involved in McGurk effect.
The present results add to these findings by showing oscillatory acDvity is not merely elicited by mismatching auditory and visual informaDon but related to the actual illusory fusion of both modaliDes.
The Study was supported by the DFG (Deutsche Forschungsgemeinschac) and the Zukuncskolleg Universität Konstanz. We thank Dr. Sabine Heim for support and input.
We thank Lars Riecke, Tobias Flaisch, Ursula Lommen and ChrisDane Wolf for providing the subjects' individual MRI images.
References
Fieldtrip Matlab-‐toolbox: hYp://fieldtrip.fcdonders.nl/
McGurk, H., MacDonald, J., (1976). Hearing lips and seeing voices. Nature, 264, 746-‐748.
Ihssen, N., (2007). The costs and benefits of emoDonal processing. Diss. Universität Konstanz.
Contact: julian.keil@uni-‐konstanz.de www.uni-‐konstanz.de/obob P.O.Box D25 78457 Konstanz Germany 0049 -‐ (0)7531 -‐ 88 42 50
In tr oducD on Me th od s Re su lts Conclusions Re fer en ces
Par%cipants:
* 12 (4m/8f, mean age 25.2) right handed, healthy subjects Data analysis:
* extracDon of ± 2 second epochs around sound onset
* visual inspecDon and rejecDon of trials containing arDfacts * subtracDon of ERF-‐AcDvity from single trials
* TFR: Wavelet, 2-‐80 Hz, 40 frequency steps, DICS source localisaDon
* cluster based dependent samples T-‐test with monte-‐carlo randomisaDon
* Individual MRI images were used for source localizaDon All analysis steps performed using fieldtrip
When comparing fusion trials versus trials with visual responses, we found two effects, one before sound onset (marked green) and one thereacer (marked red):
* Between -‐250 ms and -‐150 ms before the sound onset we found an increase in beta band acDvity (15 -‐ 33 Hz) in the "Fusion"-‐trials with a lec fronto-‐temporal and posterior-‐
parietal topography. The DICS-‐beamformer idenDfied three major corDcal sources: lec Inferior Frontal Gyrus, Precuneus and Medial Frontal Gyrus.
* Between 30 ms and 140 ms acer the sound onset, we found an increase in beta band acDvity (14 -‐ 32 Hz) in the "Fusion"-‐trials with a lec parietal topography. The DICS-‐
beamformer indenDfied two corDcal sources: Precuneus and Middle Frontal Gyrus.
ABA
ADA AGA OTHER
1500 ms
Random Pause 500 ms - 1000 ms
~ 50 ms before sound onset
0 ms
response or timeout ISI
First Video Frame
Onset of mouth movement
Sound onset
Response screen
Konstanzer Online-Publikations-System (KOPS)
URN: http://nbn-resolving.de/urn:nbn:de:bsz:352-opus-120577 URL: http://kops.ub.uni-konstanz.de/volltexte/2010/12057/
