Behavioral Results 111

Reduced feedback reliability impaired test performance. This, however, was mainly due to the detrimental effect of invalid feedback. As expected, invalid feedback led to impaired test performance, mirroring results previous results (Mies, Van der Molen, et al., 2011; Mies, Van der Veen, Tulen, Hengeveld, et al., 2011), in which negative feedback marked as invalid led to erroneous behavioral adaptation in a small number of trials.

However, the effect of feedback validity was qualified by an interaction with feedback valence, indicating that test performance was best only when feedback was positive and subsequently confirmed to be valid. Previous research already reported that positive feedback leads to a better learning performance than negative feedback (d'Ydewalle, 1976, 1979a;

d'Ydewalle & Buchwald, 1976; Ernst & Steinhauser, 2012; T. Herrmann & Stapf, 1973; B. R.

Williams, 1972) and it has been argued that this effect is due to the participants’ tendency to turn their attention towards the correct response (d'Ydewalle, 1979a). That is, participants use the feedback to determine the correct response. Accordingly, if an additional cognitive step needed to be taken to determine the correct character in our task, learning and test

performance suffered compared to correct, valid feedback which provided this information

right away. In brief, invalid feedback might have disrupted feedback processing or rehearsal of the correct character or both.

However, there is also an alternative explanation - it is conceivable that invalid feedback was initially used for learning and the subsequent validity cue was insufficient to correct this misinformation. During the test phase, participants would have then recalled the incorrect character as being correct or they might have felt unsure about the correct character, akin to the misinformation effect (Eakin, Schreiber, & Sergent-Marshall, 2003; Greene, Flynn, & Loftus, 1982). However, if this were the case, performance after positive feedback which was subsequently marked as invalid should have been worst which was not the case.

Moreover, our FRN results indicate that less reliable feedback was processed unlike highly reliable feedback, possibly in order to avoid utilization of false information, making it less plausible that erroneous learning had a great impact on test performance.

II.3.5.2 FRN results

In line with our expectations, there was feedback valence effect on the FRN for highly reliable feedback, but not for less reliable feedback. This suggests that participants processed this highly reliable feedback similar to regular feedback as in other FRN studies. Conversely, less reliable feedback was processed relatively dissimilar to regular feedback, and rather like irrelevant or inconclusive feedback (Ernst & Steinhauser, manuscript under revision) and consequently, there was no feedback valence effect on the FRN for less reliable feedback.

This result allows for a better understanding of the FRN and basic reinforcement processes, as well as their interaction with higher cognitive processes. Specifically, a closer inspection of our results further indicated that the effect of the feedback reliability manipulation on the FRN is mainly due to a top-down process as the difference between conditions manifested itself already in early trials. The absence of an FRN effect for less reliable feedback implies that this information was not automatically utilized by the reinforcement system. This is also in line with Mies, Van der Molen, and colleagues (2011) assumption that a process, which they associated with the RAC, is responsible for the evaluation of feedback relevance and influences processing of feedback information. However, our results appear to be in contrast with Mies, Van der Veen, Tulen, Hengeveld and colleagues (2011) EEG results that showed no effect of feedback validity on the FRN. It seems plausible that in their experiment the validity information was not processed fast enough to affect the processes underlying the

FRN because of the simultaneous presentation of feedback and validity cue. In the present experiment and in a prior study (Ernst & Steinhauser, manuscript under revision) participants knew in advance about the feedback reliability or relevance and therefore the top-down process might have been able to influence the processing of the feedback more effectively.

Interestingly, Walsh and Anderson (2011) reported that when an instruction suggests certain contingencies in an upcoming decision task, response behavior is in line with this information from the very beginning, whereas the FRN effect remains unaffected reflecting unbiased reinforcement learning. The seems to be in conflict with our results; however, while in the mentioned study processing of feedback was not necessarily detrimental to performance as it did not provide false information, less reliable feedback in our experiment was, by

definition, likely to be misinformative and thus likely to be harmful learning and

performance. It appears that processing of feedback by the reinforcement learning system is suppressed only when it is considered to contain misinformation, but not when it is simply redundant.

Therefore, we conclude that our results are neither in contradiction with Mies, Van der Veen, Tulen, Hengeveld, et al.’s (2011) nor Walsh and Andersons (2011) findings, but draw a more detailed picture of the effect of feedback reliability on feedback processing by indicating that basic reinforcement learning processes do not utilize potentially invalid feedback when it might be harmful to performance and sufficient preparation time is provided. However, because the FRN is assumed to reflect a dopamine learning signal conveyed to the ACC by midbrain dopamine neurons (Holroyd & Coles, 2002), it remains open whether the absence of a feedback valence effect on the FRN was caused by top-down processes suppressing the generation the dopamine signal, or whether suppression of activity in the ACC led to the absence of the FRN effect. Recent research on the effect of instructions on learning might be informative in this regard as it supports the assumption that reinforcement learning in the basal ganglia is subject to input from the prefrontal cortex and the hippocampus that biases learning the direction of instructions held in working memory (Doll et al., 2011; Doll et al., 2009; for a review see Wolfensteller & Ruge, 2012). Finally, our results add to growing body of research indicating that although the basic reinforcement learning system and an higher, working memory-based system process feedback information independently, there is also interaction between both systems (e.g., Collins & Frank, 2012; Frank & Claus, 2006; J. Li et al., 2011).

II.3.5.3 Conclusion

To summarize, we found that feedback validity, feedback valence, and feedback reliability affected feedback processing. Our behavioral results not only showed that invalid feedback did impair learning, but also that the knowledge whether invalid feedback is likely or not is important for learning. Most importantly, our ERP results show that the effect of negative and positive feedback on the FRN can be attenuated by top-down processes that utilize prior information about the reliability of the feedback to affect basic reinforcement learning processes.

III. GENERAL DISCUSSION