Undetected > detected contrast revealed activity differences in the right inferior frontal gyrus (MNI: x, y, z = 52 20 6, T = 4.42, kE= 149, p < 0.001 uncorrected; MNI: x, y, z
= 34 26 -6; T = 3.93, kE= 73, p < 0.001 uncorrected).
Analysis exploring the interaction between task and modality on subjective delay detection revealed no effects on the conservative FWE level. At a more liberal threshold (p
< 0.001 uncorrected, 50 voxels), a significant interaction between detection and modality was found in the left cerebellum (62.0% in left lobule VI, 9.3% in lobule V) extending to the fusiform gyrus (MNI: x, y, z = -32 -48 -26, F = 20.41, kE = 182, p < 0.053 FWE cluster corrected; see Figure 9). Contrast estimates corresponding to the respective cluster showed significant differences between detected vs. undetected trials for the bimodal (detected >
undetected, MNI: x, y, z = -30 -34 -28, T = 5.82, kE = 156, pFWE < .001), compared with the unimodal condition.
Figure 9. fMRI results showing activation of the left cerebellum with cluster extensions in the left fusiform gyrus for the interaction of delay detection and modality. Bar graph depicts contrast estimates of the respective cluster illustrate a specific activation for detected compared with undetected trials in the bimodal conditions. Error bars represent the standard error of the mean. p < 0.001 uncorrected with minimum cluster extension of 50 voxels.
detection-related BOLD responses. Accordingly, voluntary button presses performed by participants led to either unimodal visual/auditory or bimodal AV stimuli after various delays, and the task of the participants was to detect delays between the button press and the sensory outcome of a specific modality. In passive trials, participants were asked to report the number of modalities, which could be only visual, only auditory or AV, presented after a cue. Proportion of delay detection was found to be higher for bimodal compared with unimodal trials, and when the task was visual compared with when it was auditory. Analysis on fMRI data revealed BOLD suppression in somatosensory, visual and auditory processing regions when the stimuli were triggered by voluntary movements as opposed to when the stimuli were passively perceived. Moreover, BOLD suppression was found to be higher for subjectively detected compared with undetected trials. Analysis of BOLD activity associated with subjective delay detection revealed an interaction between detection and modality in the left cerebellum, suggesting that when the outcome was unimodal, activity in this area was less pronounced for detected trials than when it was bimodal, in which case, an opposite pattern emerged. This result, along with higher delay detection responses for bimodal trials point to the influence of additional modality on the perception of voluntary actions and their outcomes.
As expected, BOLD suppression was found in visual, somatosensory and auditory regions of the brain during voluntary actions. Suppression effects were also similar across modalities compared with similar sensory inputs observed passively. This result points to predictive processing of voluntary action outcomes whereby equally likely action outcomes were anticipated and subsequently suppressed (Bays et al., 2006; Voss et al., 2008).
Moreover, reduced BOLD responses in auditory and visual cortices were strongest for subjectively undetected trials. This was to be expected, as subjective detection of delays increased linearly with the inclusion of physical delays.
Existing research on multisensory processing has demonstrated that a certain stimulus modality can facilitate the processing of another modality, especially when they are temporally and/or spatially in synchrony (Diederich and Colonius, 2004; Meredith et al., 1987; Stein, 2012). Results of Study 2 were in line with this, in that additional sensory modality facilitated behavioral performance, as evidenced by increased detection responses when the action outcome was audiovisual.
Regarding detection-related BOLD activity, cerebellum was found to be involved in subjective detection of delays, and more so when the sensory outcome was bimodal. This result supports previous findings on cerebellum’s role in error-related processing of voluntary action outcomes (Blakemore et al., 2001; S. Blakemore et al., 1998; Wolpert et al., 1995). In addition, it suggests that the additional sensory modality was associated with increased error-related processing. Therefore, it seems in general that multisensory stimuli facilitated detection performance, which was accompanied by increased activity in the cerebellum. Other areas found to be involved for subjectively delayed trials were the ACC and the hippocampus. ACC has been consistently implicated in monitoring of action and behavior (Botvinick et al., 2001; Carter et al., 1998; Holroyd and Coles, 2002; Matsumoto et al., 2007). Higher involvement of this area for subjectively detected trials support the role of ACC in detecting temporal discrepancies between actions and their sensory outcomes.
Together, findings of Study 2 provide supporting evidence for BOLD suppression with regard to voluntary action outcomes, and error-related action outcome processing in the cerebellum. Importantly, the present findings extend BOLD suppression to multisensory action outcomes, suggest predictive processing of equally likely sensory outcomes linked with voluntary movements, and show modulatory effect of the additional sensory modality on error-related processing in the cerebellum.
Despite these novel findings, one limitation that needs consideration is the different task demands across voluntary and passive conditions. Whereas in the voluntary condition participants were asked to focus on a single modality and detect delays between the button press and this single modality, in the passive condition they were required to report the number of modalities presented in a trial. In this sense, the tasks required different aspects of the sensory stimuli to be attended. Moreover, stimulus presentation in the passive condition was unpredictable in time, compared with stimulus presentation in the voluntary condition which was more or less predictable by the button press. Research on action outcome processing underlies the influence of temporal predictability, predictability of the stimulus occurrence in time, which is different than prediction regarding action outcome related processing (Hughes et al., 2012). Future studies should distinguish temporal predictability of the sensory stimuli from predictive processing of action outcomes. A related limitation of Study 2 was the fact that when button presses were not modelled as a condition of no interest, results regarding BOLD suppression effects were weaker. Efference copy-
related predictive processing can be better investigated with contrasting voluntary movements with their externally-generated counterparts. This enables re-afferent feedback to be similar across different movements, and makes it possible to study the contribution of the efferent signal (Weiskrantz et al., 1971).
To conclude, Study 2 investigated neural correlates of perceiving unimodal vs.
bimodal outcomes associated with voluntary button presses as opposed to passive viewing of these sensory inputs. Findings showed BOLD suppression in sensory cortices which was independent of task and modality, along with increased BOLD activity in these areas for subjectively delayed as opposed to undelayed trials. In addition, detection of delays were coupled with increased activity in the ACC and hippocampus, which is in line with existing evidence on the role of these areas in action monitoring. Furthermore, the cerebellum was found to be modulated not only by voluntary actions, but also by whether the outcome was unimodal or bimodal, with increased activity in bimodal trials, corroborating its role in processing temporal discrepancies in the predicted action outcome relationship. Together, these findings support predictive processing of voluntary action outcomes and extend it to multisensory outcomes of voluntary movements.
5 STUDY 3: NEURAL CORRELATES OF MULTISENSORY ACTION OUTCOMES: ADDRESSING EFFERENCE COPY- RELATED PREDICTIONS AND NATURALISTIC ACTION OUTCOMES
The content of Study 3 has been submitted for a publication as:
Arikan, B.E., van Kemenade, B.M., Podranski, K., Steinsträter O., Straube, B., Kircher, T.
(under review). Neural correlates of perceiving multisensory action outcomes in an ecologically valid set-up: BOLD suppression in sensory cortices and multisensory influences.