Relevance for FRN research 121

The relevance of the three studies for FRN research was already discussed in the previous section: First, and in contrast to several findings that indicate a connection between the FRN effect and adaptive behavior (Butterfield & Mangels, 2003; Frank et al., 2005;

Hewig et al., 2010; Holroyd & Krigolson, 2007; Santesso et al., 2009; Van Der Helden et al., 2010, but see P. Li et al., 2010; Mies, van der Veen, Tulen, Birkenhäger, et al., 2011;

Philiastides et al., 2010), we found no such correlation between these measures. Apparently, the FRN effect magnitude is only predictive for behavior in studies that rely strongly on a reinforcement learning paradigm, but not in experiments where working memory is more central. Second, the finding that the FRN effect is modulated by prior information about

20 Note that compared to the ERPs in the learning phase of Studies 1 and 2, and the test phase of Study 2 (see Appendix B), the feedback-P300 was almost absent after the feedback presentation in the learning phase of Study 3 (see section II.3.4.2).

feedback validity indicates that either the ACC or the dopaminergic system can be affected by top-down influences, possibly by the prefrontal cortex and the hippocampus (Doll, Jacobs, Sanfey & Frank, 2009; Doll, Hutchinson, & Frank, 2011; Frank & Claus, 2006; Li, Delgado,

& Phelps, 2011).

III.2.2.1 Relevance for P300 research

Concerning the feedback-P300, Study 1 showed that this component, along with a frontal positivity, can be predictive for learning from feedback in an explicit learning and decision tasks. Moreover, feedback valence influenced the amplitude of the feedback-P300, with positive feedback being associated with a more pronounced amplitude compared to negative feedback. These findings can at least partially be attributed to the infrequency of positive feedback relative to negative feedback, as infrequent stimuli commonly lead to a stronger P300. However, Study 2 replicated this finding even though positive and negative feedback were equally likely. Moreover, an inspection of ERPs in the test phase of Study 2 also revealed a strong effect of positive feedback, despite positive feedback being more likely than negative feedback in this phase (see Appendix B). Even though this finding is in line with several studies that reported a comparable effect of feedback valence on the feedback-P300 (e.g., Banis & Lorist, 2012; Hajcak, Holroyd, et al., 2005; Hajcak et al., 2007; Leng &

Zhou, 2010; P. Li et al., 2010; Wu & Zhou, 2009; Zhou et al., 2010), other studies did not report such findings (Foti et al., 2011; P. Li et al., 2011) or even found an effect in the opposite direction (e.g., Chase et al., 2011; Frank et al., 2005; see Appendix A for an overview). A consequence of these inconsistent results is that no generally accepted

explanation for this effect exists, although it is certainly needed. On the one hand, considering that the feedback-P300 is tightly linked to controlled feedback processing, a theory of the origins of this effect appears to be vital for a better understanding of this feedback processing system. On the other hand, any viable theory of the P300 in general needs to account for this finding.

For instance, the feedback-P300 could indicate an affective reaction and resulting attention allocation to the feedback information. Although several studies showed that affective stimuli indeed leads to a larger P300 compared to neutral stimuli (e.g., Briggs &

Martin, 2009; Keil et al., 2007; Kliegel, Horn, & Zimmer, 2003; for a review, see Olofsson, Nordin, Sequeira, & Polich, 2008), Wu and Zhou (2009) already dismissed this explanation

reasoning that if this were the case, there should either be no difference between conditions or a more pronounced P300 after the presentation of negative information, i.e., after negative feedback, because usually negative information has a greater affective impact (Ito et al., 1998;

Tversky & Kahneman, 1981).

According to the context-updating theory (Donchin & Coles, 1988) the P300 reflects updating of a representation in working memory with the P300 amplitude correlating with the extent of the update and the unexpectedness of a stimulus (however, see Kok, 2001; Verleger, 2008). From this perspective, it is central which representation was held in working memory at the time of feedback presentation and what kind of feedback was expected. It appears to be most likely that the participants held a representation of the chosen stimulus in working memory (d'Ydewalle, 1979a; d'Ydewalle & Buchwald, 1976) and that they were inclined to believe it was correct (Gibson & Sanbonmatsu, 2004; Sharot, Riccardi, Raio, & Phelps, 2007) or else they would not have chosen this particular stimulus. Accordingly, negative feedback should have led to a larger update than positive feedback as it is dissimilar to the

representation in working memory and it violates expectations. However, the larger feedback-P300 found after positive feedback is at odds with this conclusion. The question remains why positive feedback should lead to a greater working memory update. A straightforward

explanation would be that the working memory is updated with the information about the correct response. This information is readily available when positive feedback is presented but has to be deduced when negative feedback is presented. Although plausible, this

explanation ignores central points of the context-updating theory, namely, prior assumptions and unexpectedness (e.g., Donchin, 1981; Duncan-Johnson & Donchin, 1977).

A more viable explanation could include on the LC-NA theory that links the P300, specifically the P3b, to a phasic NA signal from the locus coeruleus to the outcome of a decision process that improves subsequent processing of this outcome (Nieuwenhuis, 2011;

Nieuwenhuis, Aston-Jones, et al., 2005). In tasks used for ERN research, this decision process might relate to the detection of an error and thus this process determines whether an error has occurred or not (e.g., Nieuwenhuis et al., 2001; Steinhauser & Yeung, 2010; for reviews, see Overbeek et al., 2005; Ullsperger et al., 2010). The Pe can be considered to be a correlate of this decision process, similar to the P300. In a decision task with feedback, the feedback processing might instead involve determining whether an initial choice was correct.

According to Steinhauser and Yeung (2010), error detection can be considered to be a

decision process with the Pe amplitude reflecting the accumulated evidence that an error has occurred which is compared to a response criterion. In more detail, certain information is more central for adaptive behavior. Obviously, when a task is usually completed successfully it is important to register and be aware of the occurrence of an error because this indicates that cognitive adjustment (e.g., of a response threshold or of an attention lens) is necessary. That is, an error is a motivationally salient event. As one of the main properties of controlled processes is the ability to swiftly implement adjustments, it is vital that this information is made available to controlled processes, i.e., that one becomes aware of the error. Moreover, the more evidence that an error has occurred is available, the less likely is the correct

completion of a task, the more important is cognitive adjustment, and thus the more important is the awareness and controlled processing of this error. Awareness and unimpaired controlled processing of an error is achieved by a strong response of the attentional system. In terms of the LC-NA theory, as well as the Polich’s (2007) inhibition theory, this attentional response might be related to a phasic NA response that affects several brain regions and results in a strong P3b, or in this case, a pronounced (late) Pe. In interaction with phasic and tonic DA-level (phasic DA responses preceding the NA response might be reflected by the P3a, see J.

B. O'Connell et al., 2011; Polich, 2007), NA contributes to an optimal signal-to-noise ratio in information processing (Brennan & Arnsten, 2008; Oades, 1985; Sikström & Söderlund, 2007) and thus a phasic NA response ensures privileged processing of specific information and the suppression of erroneous activity (see also Polich, 2007).

For external feedback, a comparable process might occur; however, in this case the process that instigates the attentional response is attuned to accumulating evidence that a response was correct. Accordingly, the more evidence is accumulated for a correct response without interference, the stronger the feedback-P300. Again, this is because it is essential for adaptive behavior that controlled processes instigate swift cognitive adjustments when the direction of adjustments is clear. Of course, this eventually results in a greater working memory update as proposed by the context-updating account; however, the evidence-accumulation account has the merit of explaining the feedback valence effect and of going beyond the mere outcome of a process (i.e., the working memory update) by being more specific about the details and antecedents of this process. For instance, the ambiguous nature of an incongruent Stroop stimulus in Study 2 might have introduced evidence against a

correct choice and thus resulted in a reduced feedback-P300, even though the relevant feedback indicated a correct choice.

Interestingly, this approach bears resemblance to the information theory of the P300 (Johnson Jr., 1988a) that was picked up recently by Luque and colleagues (2012). According to Johnson, the P300 amplitude is a function of task-relevant information provided by a stimulus, information loss during the presentation of the stimulus, and subjective probability of the stimulus. Put in terms of this theory, an error is more informative for the task at hand than a correct response in an ERN paradigm (e.g., Erikson flanker task, Stroop task, etc.).

This is not only because it is rare, but also because it indicates suboptimal response selection and the need for adjustment. In a learning task, positive feedback is more informative because it readily provides information with respect to stimulus to choose in the test phase, whereas this information needs to be deduced from negative feedback, i.e. additional information needs to be added.

This information theory account of the feedback-P300 might also explain the absence of a feedback valence effect on the feedback-P300 in some experiments (e.g., Foti et al., 2011; P. Li et al., 2011). These studies tend to utilize tasks were no actual learning is required or possible (guessing task, gambling task); under these circumstances, feedback is obviously not informative for future behavior. Consequently, only the FRN amplitude should be

modulated by feedback valence (as the outcome is still better or worse than expected), but not the feedback-P300 amplitude. In fact, the feedback-P300 amplitude should be relatively small in those tasks. In contrast, in some paradigms, a general rule has to be extracted (called “open tasks” by Nuttin and Greenwald, 1968) and under these circumstances negative feedback might be as or evenmore informative than positive feedback. Indeed, one central finding of Chase and colleagues (2011) was a larger feedback-P300 following negative feedback that led to a change in response behavior in reversal learning. Here, participants should have tested for negative feedback because it confirmed the participant’s explicit hypothesis that reward contingencies had switched.

Furthermore, Tricomi and Fiez (2008, 2012) found that brain activity in the prefrontal cortex and the caudate nucleus correlated with the informational value of feedback, i.e.

whether it was presented early or late in a learning task (Tricomi & Fiez, 2008) and whether two or four response options were available (Tricomi & Fiez, 2012) in a declarative memory task which is very similar to the multiple-choice task in Study 1. However, the reported

activity in the caudate nucleus also raises the question whether the authors simply measured a correlate of the reward prediction error. Indeed, a major challenge for future studies based on the information theory account is that they need to address the differences between the

prediction error and informational value, e.g. by varying the informational value of feedback while holding the unexpectedness of the feedback constant. Another possibility would be devise a situation where feedback is unexpected, but uninformative, or expected, but

informative. Indeed, as shown by the results in the study of Case and colleagues (2011), when a hypothesis is tested, confirming evidence might be exactly what is expected, but still be very informative as indicated by the subsequent switch in response behavior.

III.2.3 The Effect of Feedback Valence on Performance

The previous line of reasoning is partly based on the assumption that positive feedback is more informative than negative feedback, and that positive feedback receives more

attention. Support for this comes from research on the effect of “right” and “wrong”

responses, i.e., the feedback valence, on learning (Buchwald, 1969; R. L. Cohen & Nilsson, 1974; d'Ydewalle, 1976; d'Ydewalle & Buchwald, 1976; Eelen & D'Ydewalle, 1979; T.

Herrmann & Stapf, 1973). A similar effect was found in Studies 1 and 2, with a correct learning phase guess leading to better test performance compared to an incorrect guess. This effect was also observed in Study 3; however, it was not encountered when feedback was invalid.

Thorndike (1932) hypothesized that positive feedback simply had a greater impact than negative feedback, but later research rejected this assumption. Studies further established that it is indeed positive feedback that leads to better performance and not negative feedback leading to worse performance (Buchwald, 1969). Moreover, a greater number of response alternatives increases the likelihood that a response associated with positive feedback will be repeated (d'Ydewalle & Buchwald, 1976; d'Ydewalle & Eelen, 1975). This is due to an increase in informational value of positive feedback with increasing number of alternatives (d'Ydewalle, 1979b). Moreover, it was shown that after feedback participants turn their attention to the correct response and that the response repetition of participants “is a function of recalling previously given responses and their outcomes” (d'Ydewalle, 1979a, p.434), i.e., response and feedback are encoded separately (d'Ydewalle & Buchwald, 1976; d'Ydewalle &

Eelen, 1975). Consequently, when the feedback, but not the response, is forgotten,

participants resort to guessing the feedback of the recalled response and are biased to assume that this response was correct (d'Ydewalle, 1979a; Eelen & D'Ydewalle, 1979).²¹

Together, these studies not only provide an explanation for some of the behavioral results reported in this dissertation, but also support the assumption that positive feedback is more informative for participants and therefore receives more attention. Moreover, they raise the question whether some of the variables that were found to modulate the impact of

feedback valence on performance – such as the number of alternatives, expectation of a second test (d'Ydewalle, 1976), or active responding versus mere observation (d'Ydewalle &

Eelen, 1975; Eelen & D'Ydewalle, 1979) – modulate the effect of feedback valence on the feedback-P300 in a similar fashion.