Behavioral Data 105

The proportion of correct responses in the learning trials (M = 49.5%, SD = 3.10) did not differ significantly from chance performance, t(26) = 0.86, p = 0.40. This suggests that participants were merely guessing in the learning phase. The experiment investigated the effect of guessing accuracy, feedback validity and feedback reliability on test phase

performance. To this end, each item in the test phase was classified according to whether the guess was correct or incorrect, whether the initial feedback was valid or invalid, and whether this feedback was established as highly reliable (valid in 87.5% of the cases) or less reliable (valid in 62.5% of the cases) by the information presented at the beginning of the learning block.

Due to the fact that invalid feedback had to be rare in the highly reliable feedback condition there were insufficient trials for a reliable analysis of the interaction of Feedback Reliability, Feedback Validity and Feedback Valence. Accordingly, we collapsed our data across the Feedback Valence in order to investigate any effects of Feedback Reliability and Feedback Validity on test performance (Fig. 13A). The data was subjected to a two-way ANOVA with repeated measurement on the variables Feedback Reliability (less reliable, highly reliable) and Feedback Validity (valid, invalid), which revealed a significant main effect of Feedback Validity, F(1, 26) = 12.8, p < 0.001, but neither a main effect of Feedback Reliability, F(1, 26) = 1.19, p = 0.29, nor a significant interaction, F(1, 26) = 1.47, p = 0.24.

Figure 13. Average performance in the test block as a function of Feedback Reliability and Feedback Validity (A), and Feedback Valence and Feedback Validity (B).

As expected, test performance was impaired when the feedback was invalid (M = 72.0%, SD = 11.8) compared to when it was valid (M = 78.0%, SD = 6.77). Because Feedback Reliability did not affect test performance independent of Feedback Validity, we collapsed over this condition in order to investigate whether the effect of Feedback Validity depended on Feedback Valence (ref. Figure 13B). Indeed, a two-way ANOVA with repeated measurement on the variables Feedback Valence (positive, negative) and Feedback Validity (valid, invalid) revealed significant main effects of Feedback Validity, F(1, 26) = 28.0, p <

0.001, and Feedback Valence, F(1, 26) = 11.5, p < 0.005, which were however qualified by a significant interaction, F(1, 26) = 6.91, p < 0.05. Apparently, the test performance was best when the feedback was positive and valid (M = 84.6%, SD = 7.18), but impaired when it was positive, but invalid (M = 70.0%, SD = 16.9) or negative, regardless of validity (valid: M = 71.6%, SD = 9.89; invalid: M = 70.7%, SD = 12.1). Taken together, this shows that feedback characteristics in the learning phase had an effect on later test performance. When feedback processing is complicated by either invalid feedback or negative feedback, test performance suffers. Because invalid feedback was more prevalent when feedback was less reliable, overall performance was reduced in this condition (M = 74.6%, SD = 7.96) compared to when feedback highly reliable (M = 77.5%, SD = 7.80), as revealed by a one-way ANOVA with Feedback Reliability as only factor, F(1, 26) = 5.44, p < 0.05.

II.3.4.2 Feedback-Locked ERP Data

We hypothesized that information on the likelihood of a feedback being valid or invalid should affect the way it is processed by the basic reinforcement learning system and thus the FRN. Specifically, when feedback is considered to be highly reliable, then feedback should be processed normally, even though there was some chance that it would be invalid.

Accordingly, there should be an FRN effect similar to previous studies (e.g., Ernst &

Steinhauser, 2012). Conversely, less reliable feedback should not be processed like normal feedback, and consequently there should be a reduced effect of feedback valence on the FRN, or even none at all. To investigate these assumptions, we considered feedback-locked ERPs elicited by the initial feedback as a function of Feedback Valence (positive, negative) and Feedback Reliability (highly reliable, less reliable). Indeed, visual inspection of the ERPs and respective topographies (see Fig. 14) revealed that there was a valence-dependent amplitude difference at fronto-central electrode sites around 310 ms, but only when the feedback was reliable.

These observations are supported by a two-way ANOVA with repeated measurements on the variables Feedback Valence (right, wrong) and Feedback Reliability (highly reliable, less reliable) for the mean amplitude in the time window of 285 to 336 ms at the FCz electrode position. This analysis revealed a marginally significant main effect of Feedback Valence, F(1, 26) = 2.99, p < 0.10. More importantly, it was qualified by a significant

interaction between Feedback Valence and Feedback Reliability, F(1, 26) = 4.59, p < 0.05. It appears that only when initial feedback was considered to be highly reliable, the feedback valence had an effect on the FRN with a more negative average amplitude after negative feedback than after positive feedback (see Fig. 14B). In contrast, no such difference was found for less reliable feedback.

Figure 14. Grand average feedback-locked ERPs in the learning phase as a function of Feedback Valence (positive, negative) and Feedback Reliability (highly reliable, less reliable). A and B: Grand average waveforms for channels FCz and Pz, respectively. C: Topographies of the difference wave between positive and negative feedback for each Feedback Reliability condition 285-334 ms following feedback onset. D: Mean amplitudes in the 285-334 ms time window for the Feedback Reliability and Feedback Valence conditions at FCz.

However, as there was a sizable effect of feedback valence on the positive peak preceding the FRN, it is conceivable that this affected our findings. Therefore, in order to further corroborate our findings, we conducted a peak-to-peak analysis of the FRN at the FCz electrode position. An ANOVA of the peak amplitudes including the factors Feedback

Valence and Feedback Reliability showed a significant interaction of these factors, F(1, 26) = 6.91, p < 0.05, but no main effects, Fs < 1. For the condition where feedback was highly reliable we found a larger peak after negative feedback compared to positive feedback.

Conversely, the peak amplitude after positive feedback was larger after positive than after negative feedback when feedback was less reliable. Overall, both FRN analyses show that information about feedback reliability affected the FRN in the hypothesized direction: While for highly reliable feedback valence information affected the FRN amplitude in the same fashion as observed in multiple prior studies, there was no comparable FRN effect for less reliable feedback.

This effect is likely to be due to different processing of the feedback. However, from these results alone, it is not possible to tell whether these processing differences were caused by top-down influences or by the repeated invalidation of the feedback experienced by the participants. More specifically, it is conceivable that in the less reliable feedback condition, feedback was initially processed similar to the highly reliable feedback condition, but that in the course of a feedback reliability block, processing changed because the reinforcement learning system utilized the frequent validity cues to adjust itself to the fact that upcoming feedback was likely to be invalid. In contrast to this “experience-based” account, a top-down explanation would suggest that prior feedback reliability information made higher cognitive processes disengage from learning in the reinforcement learning system, resulting in feedback processing differences between the conditions already at the beginning of a block.

Accordingly, the FRN effect should be absent already in the first half of both less reliable feedback blocks, whereas for highly reliable feedback blocks it should be present at any time

Figure 15. Mean amplitudes of the FRN at the FCz electrode position with standard error of the mean as a function of feedback valence (positive, negative) and feedback reliability (reliable, less reliable). The mean amplitudes for the first half of the blocks is shown above (A), those for the second half are shown below (B).

in the block. If, however, the experience-based explanation holds true, the FRN effect should only be absent in the second half of the less reliable feedback blocks, but present in the first half.

In order to make an informed decision between both explanations, we divided the feedback reliability blocks in an early and a late half and determined average amplitudes of the feedback-locked ERP at FCz in the FRN time window for each block half, as well as for each feedback valence and feedback reliability condition. We subjected these data to a three-way ANOVA with repeated measurement of the variables Block Half (early, late), Feedback Valence (correct, incorrect) and Feedback Reliability (highly reliable, less reliable). First, there was a significant main effect of Block Half, F(1, 26) = 4.33, p < 0.05, with reduced amplitudes in late blocks. This might either be due to reduced attention to feedback at the end of a block, or increased attention to feedback at the beginning. More importantly and similar to the results of our prior analyses, we obtained a significant interaction of Feedback

Reliability and Feedback Valence, F(1, 26) = 4.39, p < 0.05 – however, there was no significant three-way interaction, F < 1. As presented in Figure 15, feedback valence did affect the FRN amplitude in neither the first, nor the second half of a block with less reliable feedback. In contrast, there was an FRN effect for highly reliable feedback, albeit it was reduced in the second half of a block. These results were also obtained when the ERP data were subjected to a peak-to-peak analysis. Again, we found a significant interaction of Feedback Reliability and Feedback Valence, F(1, 26) = 5.29, p < 0.05. However, for this measure, we also found a significant three-way interaction between Feedback Valence, Feedback Reliability and Block Half, F(1, 26) = 5.14, p < 0.05. Apparently, there was marginally significant FRN effect (positive feedback: M = 9.25 µV, SD = 4.32; negative feedback: M = 11.2 µV , SD = 4.42) for highly reliable feedback in both block halves, F(1, 26) = 3.50, p = 0.07, while there was none or even an inverse FRN effect (positive feedback:

M = 11.1 µV, SD = 3.77; negative feedback: M = 9.81 µV , SD = 4.06) for less reliable feedback, F(1, 26) = 2.22, p < 0.15. In fact, for the second half of blocks with less reliable feedback the peak-to-peak FRN amplitude was more pronounced after positive feedback (M = 11.7 µV, SD = 4.57) than for negative feedback (M = 8.72 µV , SD = 4.28), F(1, 26) = 11.5, p

< 0.005. Together, these results indicate that already in early trials, less reliable feedback was processed differently compared to highly reliable feedback. Accordingly, this favors the

assumption that top-down influence caused the FRN differences between feedback reliability conditions.

II.3.5 Discussion

In the present study, we used ERPs and test performance to elucidate the effect of feedback validity and feedback reliability on feedback processing, with a special focus on the modulation of the FRN by feedback reliability. To this end, we administered a simple

decision task where participants could maximize their pay-off in a test phase by learning from feedback provided after a guess during the prior learning phase. This initial feedback,

however, was not always valid and thus participants needed to consider a subsequent validity cue. This validity cue informed participants whether the initial feedback had been valid or invalid. Further, at the beginning of the learning phase, the participants were informed about the reliability of the initial feedback. Feedback was either highly reliable or less reliable, i.e., it was valid in 62.5% or 87.5% of the cases, respectively.