Basic Characteristics of the FRN 26

Although a negativity after negative feedback can already be observed in ERP waveforms presented in early P300 research papers (e.g., Johnson Jr. & Donchin, 1978;

Ruchkin et al., 1981), it was ERN research that led to the discovery of the FRN. Motivated by the question whether there is an ERN-like deflection when the individual can only have insight in the correctness of the response by external feedback, (Miltner et al., 1997) investigated ERPs after positive and negative feedback in a time estimation task. Like a multitude of subsequent studies (ref. Appendix A), they found a negative deflection peaking at frontocentral recording locations (Fz, FCz) 230 to 330 ms after negative feedback onset.

Such a negativity was not evident after positive feedback, and although several later studies found a positive feedback-related negativity (for an overview see Table I), it is less

pronounced than the negative feedback equivalent (this difference is the FRN effect). In addition, this first study showed that the feedback negativity is part of a generic feedback processing system, i.e., it is independent from the modality of the feedback. This conclusion is further supported by the fact that the FRN was found in a diverse array of paradigms, such as in a probabilistic learning task (e.g., Frank, Woroch, & Curran, 2005; Holroyd & Coles,

8 The naming of the component is not consistent across papers. The feedback-related negativity has also been known as the feedback ERN (fERN; e.g., Holroyd; Butler Holroyd), midline feedback negativity (MFN;

Gehring;), feedback negativity (FN; e.g., Hajcak, Holroyd, Moser, & Simons, 2005) or just N200. With regard to the latest theories on the nature of this component, I will henceforth refer in this introduction to the N200 following a feedback stimulus as FRN, while the FRN effect is the difference of the amplitudes following positive and negative feedback in the N200 time interval.

2002; Nieuwenhuis, Slagter, Von Geusau, Heslenfeld, & Holroyd, 2005), the Wisconsin Card Sorting Test (WCST; Cunillera et al., 2012), gambling tasks (Gehring & Willoughby, 2002;

Nieuwenhuis, Yeung, Holroyd, Schurger, & Cohen, 2004), and in less abstract tasks, such as a virtual T-maze task (Baker & Holroyd, 2009) or Blackjack (Hewig et al., 2008). For a reliable estimation of the FRN amplitude in a condition, Marco-Pallarés and colleagues (Marco-Pallarés, Cucurell, Münte, Strien, & Rodríguez-Fornells, 2011a) recommend at least 20 trials with young, healthy participants. A recent series of experiments provided evidence that the FRN is stable over time in individuals and thus could be used as a trait-like measure (Segalowitz et al., 2010).

It is generally accepted that the FRN and the ERN are closely related, if not the same (Holroyd & Coles, 2002). The ERN (or Ne, Falkenstein et al., 1990; Falkenstein, Hohnsbein, Hoormann, & Blanke, 1991) is a negative component that is most pronounced after an erroneous (Falkenstein et al., 1990; Falkenstein et al., 1991; Gehring et al., 1993) or too late response (Johnson, Otten, Boeck, & Coles, 1997; Luu, Flaisch, & Tucker, 2000) and peaks about 50-100 ms after response onset at medial frontal electrode positions (Falkenstein, Hoormann, Christ, & Hohnsbein, 2000; Leuthold & Sommer, 1999).The ERN is followed by a positivity, called Pe (Falkenstein et al., 1991), that peaks between 200 and 500 ms and is maximal at parietal electrode sites, similar to the P300 (Overbeek, Nieuwenhuis, &

Ridderinkhof, 2005).

However, a recent account claims that the feedback negativity and the oddball N200 both reflect the same component (Holroyd, 2004; Holroyd, Pakzad-Vaezi, et al., 2008)⁹. In fact, several studies brought forward direct (Baker & Holroyd, 2011; Eppinger, Kray, Mock,

& Mecklinger, 2008; Hewig et al., 2010; Holroyd, Krigolson, & Lee, 2011b; Holroyd, Pakzad-Vaezi, et al., 2008; Kreussel et al., 2012) or indirect support (M. X. Cohen, Elger, &

Ranganath, 2007; Foti, Weinberg, Dien, & Hajcak, 2011) for this assumption: In this view, the FRN effect is essentially a feedback-locked N200 (irrespective of its valence, see Holroyd et al., 2006) that is overlaid by a positive component (Luck, 2005), the Rew-P¹⁰, that is more pronounced after positive feedback. For instance, Baker and Holroyd (2011) showed that the Rew-P latency is sensitive to the complexity of the information communicated by the

9 For the ERN, however, Gehring and Willoughby (2002) argued that both components are not identical.

10 This positivity has also been called “feedback correct-related positivity” (Holroyd, Pakzad-Vaezi, et al., 2008) or “outcome positivity” (Hewig et al., 2011)

feedback. Because of this, the Rew-P and the N200 did no longer overlap in several of their experiments, “thereby exposing the N200” (Baker & Holroyd, 2011; p. 28). Although this new account of the FRN effect raises some questions concerning the relation between the FRN effect and the ERN effect, it is compatible with other assumptions concerning the FRN.

I.4.2.2 The Neural Origin of the FRN

The dipole solution estimated by Miltner et al. (1997) located the origin of the

difference in EEG activity between positive and negative feedback in the ACC. Later studies tried to identify more specific locations, but in general determined that the ACC is strongly related to the FRN (Müller, Möller, Rodríguez-Fornells, & Münte, 2005; Nieuwenhuis, Slagter, et al., 2005). The ERP dipole findings were further corroborated by fMRI results (Nieuwenhuis, Aston-Jones, & Cohen, 2005, but see also van Veen, Holroyd, Cohen, Stenger,

& Carter, 2004), and intracranial electrode recordings in humans (Z. M. Williams, Bush, Rauch, Cosgrove, & Eskandar, 2004) and monkeys (Amiez, Joseph, & Procyk, 2005;

Kennerley, Dahmubed, Lara, & Wallis, 2009; Kennerley & Wallis, 2009; Shima & Tanji, 1998).

In addition, a similar generator was found for the ERN. The neuronal generator of the ERN is located in the ACC according to studies using dipole source localization (Dehaene, Posner, & Tucker, 1994; M. J. Herrmann, Römmler, Ehlis, Heidrich, & Fallgatter, 2004;

Holroyd, Dien, & Coles, 1998; van Schie, Mars, Coles, & Bekkering, 2004; van Veen, Cohen, Botvinick, Stenger, & Carter, 2001) , intracerebral ERP recordings in monkeys (Gemba, Sasaki, & Brooks, 1986; Niki & Watanabe, 1979) and humans (Brázdil et al., 2002),

magnetoencephalography (MEG; Miltner et al., 2003), and event-related fMRI (Carter et al., 1998; Kiehl, Liddle, & Hopfinger, 2000; Ullsperger & Von Cramon, 2001); for an overview, see Carter & Van Veen, 2007). However, several authors pointed out that there are at least two distinct sub-regions in the ACC: the rostral ACC, which is related to affective processing, and the dorsal ACC (dACC), which is supposedly involved in purely “cognitive” operations like cognitive control (e.g., Bunge, Hazeltine, Scanlon, Rosen, & Gabrieli, 2002; Bush et al., 2000; Carter & Van Veen, 2007; Weissman, Giesbrecht, Song, Mangun, & Woldorff, 2003).

Most theories of the ERN concentrate more on the dACC. In addition to the ACC, in several of these studies activation of the lateral prefrontal cortex was also observed during error processing (Kiehl et al., 2000; Ullsperger & Von Cramon, 2001).

I.4.2.3 FRN Theories

Due to the roots of FRN in ERN research, and because both components are assumed to be strongly related, most ERN theories also try to account for the FRN. The earliest ERN theories assumed that this component reflects the activity of a general error monitoring system (Coles, Scheffers, & Holroyd, 2001; Falkenstein et al., 1991; Falkenstein et al., 2000;

Gehring et al., 1993; Scheffers & Coles, 2000; Scheffers, Coles, Bernstein, Gehring, &

Donchin, 1996). According to these accounts, the ACC achieves error detection by a comparison of an efferent copy of the implemented motor response and the motor representation of the intended, usually correct, response. A discrepancy, i.e., when both representations are dissimilar, is registered in the ACC and the resulting brain activity

manifests itself as an ERN. Because of this, the theory is also called the mismatch theory, and it includes the assumption that the degree of mismatch is related to the ERN amplitude. In addition, it is further assumed that the mismatch information, i.e., information that an error has occurred, is thus transferred to other brain areas, supposedly the frontal lobe, to inform compensatory processes aimed at the reduction of future errors. For this account to be a valid FRN theory, the concept of a representation mismatch has to be broadened to include

representations other than motor representations.

A different account, the conflict-monitoring theory, proposes that the ACC is not involved in the detection of errors by comparison of representations, but in the monitoring of conflict between responses. This conflict occurs when two responses are activated at the same time: For instance, when a task involves fast responding to a stimulus (for example, in the Eriksen flanker task (Botvinick, Cohen, & Carter, 2004; Eriksen & Eriksen, 1974) or the Stroop task (Stroop, 1935; for a review, see MacLeod, 1991), both correct and incorrect responses are activated and the response whose activation reaches an internal criterion first is enacted (see Hübner, Steinhauser, & Lehle, 2010; Ratcliff, 1978). Sometimes the incorrect response’s activation reaches the criterion first, resulting in an error. However, the activation of the correct response will more often than not surpass the activation of the incorrect

response shortly thereafter. The result of both responses being highly activated is a strong conflict, which is detected by the ACC and results in a pronounced ERN. In the case of the FRN, the conflict is supposed to occur between the expected and the actual representation (Jia et al., 2007), Like in the mismatch/error detection theory, the ACC itself is not responsible for

the implementation of attentional control necessary for future error-avoidance, but is sensitive to competition between processes and the resulting conflict that necessitate attentional control to be resolved. In other words, the ACC signals – supposedly to the prefrontal cortex (Kerns et al., 2004) – when cognitive control is needed based on conflict information.

A recently proposed update of the conflict-monitoring account further argues that response conflict reflected in ACC activity not only drives immediate adjustment of cognitive control, but also that this conflict information can serve as a teaching signal (Botvinick, 2007). This perspective aims at integrating findings about the importance of the ACC in decision-making and long-term learning with the response conflicting account.

That is, this account tries to integrate the conflict-monitoring theory with the already presented RL theory which assumes that the ERN and the FRN, respectively, are

manifestations of a reinforcement learning signal – more specifically, the temporal difference error – that is conveyed to the ACC by midbrain DA neurons. This signal is used to train the ACC and leads ultimately to adaptive behavior based on the reinforcement history. Thus, the FRN, as well as the ERN, are manifestations of basic reinforcement learning and indicate whether an outcome was better or worse than expected. Moreover, the amplitude of both components should be predictive for behavioral adaptation.

Finally it has been suggested that the FRN is also influenced by direct “affective”

signals from the ventromedial PFC to the ACC (Hewig et al., 2011), reflecting so-called somatic markers (Bechara & Damasio, 2005; Bechara, Damasio, & Damasio, 2000).

However, as it will become clear in the following discussion of variables that modulate the FRN and the FRN effect, support for the RL theory is strong and as multiple findings are in line with predictions of this account.

I.4.2.4 FRN-Modulating Variables

I.4.2.4.1 The FRN as an indicator of feedback valence

One of the central predictions of the RL theory is that the FRN should encode the valence of the feedback, i.e., whether an outcome is good or bad (Nieuwenhuis, Holroyd, Mol, & Coles, 2004; Nieuwenhuis, Slagter, et al., 2005), or, more precisely, “better or worse than expected”.

Importantly, “worse” is context-dependent, i.e., it does not necessarily mean that the individual did not receive an reward – it might be that it was just not as high as expected

(Holroyd, Larsen, & Cohen, 2004; for comparable fMRI results, see Nieuwenhuis, Heslenfeld, et al., 2005).

As presented in Appendix A, almost all FRN studies find a feedback valence effect, except under certain conditions (e.g., blocking, Baker & Holroyd, 2009). One notable exception is a study by Gehring and Willoughby (2002) who found that the FRN is not sensitive to the correctness of the response, but only to the amount of monetary loss.

However, according to Nieuwenhuis and colleagues (2004) this result is possibly due to insufficient salience of the feedback valence information and to the fact that the FRN results were confounded with the P300 (Yeung & Sanfey, 2004; but see Bellebaum, Polezzi, &

Daum, 2010). Indeed, when Nieuwenhuis et al. used color to emphasize a feedback dimension (utility or performance), the FRN became sensitive to the valence information in the

emphasized dimension only. Importantly, this also implies that some attention is necessary for the feedback valence effect on the FRN to occur.

Recent studies show that the FRN is not only sensitive to feedback in the strict sense (i.e., information about the consequences of one’s actions), but also to outcomes that are unrelated to one’s actions, such as offers in an ultimatum game (Boksem & De Cremer, 2010;

Hewig et al., 2011; Polezzi et al., 2008), feedback for the action of other people (Marco-Pallarés, Kramer, Strehl, Schröder, & Münte, 2010) or when no overt choice was made (Yeung, Holroyd, & Cohen, 2005). Mirroring the results reported for normal feedback, outcomes that are worse than expected (e.g., unfair offers) elicit a stronger FRN than favorable outcomes.

I.4.2.4.2 The FRN’s sensitivity to quantitative differences in reward.

In a seminal study investigating reinforcement learning in a simple gambling game, Yeung and Sanfey (2004) found that while the FRN encodes feedback valence, it is insensitive to reward magnitude which is, however, correlated with the P300 amplitude. This proves to be challenging for the RL theory as the following assumption can be derived from this theory: If the FRN represents a dopaminergic reinforcement signal that indicates the deviation of an outcome from prior expectations, then the FRN should be especially pronounced after a monetary loss that is unexpectedly high. The finding that the FRN is insensitive to the degree of monetary loss (Yeung & Sanfey, 2004; see also Hajcak, McDonald, & Simons, 2003a;

Hajcak, Moser, Holroyd, & Simons, 2006) appears to stand in contrast to this hypothesis. In

other words, the FRN could just “code outcomes in a binary fashion” (Bellebaum et al., 2010, p. 3347; see also Hajcak, Holroyd, et al., 2005; Hajcak et al., 2006). However, other ERP studies, in which the presentation of valence and reward magnitude was not divided as in the Yeung and Sanfey’s study, did find an effect of reward magnitude on FRN amplitude (Banis

& Lorist, 2012; Bellebaum et al., 2010; Marco-Pallarés et al., 2010; Wu & Zhou, 2009). In addition, several fMRI findings hint that the ACC is sensitive to the magnitude of rewards (e.g., Knutson, Taylor, Kaufman, Peterson, & Glover, 2005; Rolls, McCabe, & Redoute, 2008). A possible reason for these conflicting results might have been the design of the Yeung and Sanfey’s study. Because they intended to discern the effect of the FRN from the P300, the authors utilized a rather complicated design. Bellebaum et al. (2010) chose a simpler design and effectively varied only the cued magnitude of the reward in a probabilistic learning paradigm. They found a substantial reward magnitude effect on the FRN with a larger

amplitude difference between win and loss for trials with a high reward magnitude. Recently, Banis and Lorist (2012) remarked that the effect of reward magnitude on the FRN is prevalent in studies where this information is salient, i.e., presented along with the feedback valence information, but not in those studies where reward magnitude was presented in an abstract fashion (Hajcak et al., 2006) or at the beginning of the trial (Holroyd et al., 2006). This further supports the assumption that the design of the study – more precisely, the manner in which magnitude information is presented – is central for the effect of reward magnitude on the FRN. It appears that only when reward magnitude information can be encoded without deeper processing, it is available to the reinforcement system underlying the FRN. In contrast,

because the P300 is linked to higher processing, an effect of reward magnitude information on this component can also be observed where this information is less salient (see Section

I.4.3.3).

I.4.2.4.3 The FRN’s sensitivity to feedback expectancy.

Concerning Yeung and Sanfey’s (2004) experiment, Nieuwenhuis, Holroyd and colleagues (2004) noted that the prior knowledge the participants had about the amount of money they gambled on obviously reduced the unexpectedness of the size of the monetary

outcome¹¹. This relates to a more general question, namely whether the FRN is sensitive to the unexpectedness of an outcome – a central hypothesis of the RL theory directly derived from work on the reward prediction error.

Several studies have shown through different manipulations that the FRN is modulated by an interaction between feedback expectancy and feedback valence. Expectancies were manipulated by prior experience (Nieuwenhuis et al., 2002), recent outcomes (Holroyd &

Coles, 2002), the number of alternatives (Holroyd, Nieuwenhuis, Yeung, & Cohen, 2003), and probabilistic cues (Liao, Gramann, Feng, Deák, & Li, 2011) and always resulted in larger FRNs after unexpected negative feedback. Further, when an outcome is completely expected, the reward prediction error should be zero and no amplitude difference should occur. This is similar to the effect of the so-called blocking effect (Kamin, 1969) and accordingly, Baker and Holroyd (2009) found that a modulation of the FRN by feedback valence only occurred when a preceding cue was known to be not predictive. However, – and as expected by the theory – when the cue was predictive for the actual feedback, it was followed by an FRN (see also Dunning & Hajcak, 2007; Liao et al., 2011) . These results augment earlier findings showing that in a learning task the FRN is reduced or even absent when the correct contingencies had been learned (Holroyd & Coles, 2002). Under these conditions, the response itself is predictive for the outcome (and thus evokes an ERN) and the feedback valence expected.

However, in two experiments Hajcak and colleagues (2005) failed to find an effect of prior reward-likelihood information on the FRN. The results of a follow-up study implicated that additional factors contributed to these null results, as Hajcak et al. (Hajcak, Moser, Holroyd, & Simons, 2007) found an association of the participants’ reward predictions with the FRN amplitude, but only when these predictions were made after the choice. This suggests that the salience of the reward likelihood at the moment of feedback presentation is important (see also Holroyd, Krigolson, Baker, Lee, & Gibson, 2009; Moser & Simons, 2009). Furthermore, because two studies showed that for the feedback expectancy effect on the FRN, it is important that the optimal response is in fact learnable (Holroyd et al., 2009) or that participants had gained insight into a hidden rule (Bellebaum & Daum, 2008), it might be that feedback expectancies have be to clear and not diffuse. Alternatively, these results might

11 Some degree of unexpectedness was still given, because the reward magnitude cue only communicated a small reward or loss, but not a specific value.

fall in line with two other studies that showed that the FRN is modulated by the controllability of the situation (P. Li, Han, Lei, Holroyd, & Li, 2011; P. Li et al., 2010).

In sum, results favor the assumption that the FRN is sensitive to feedback expectancy and thus support the RL theory and the proposed association with the reward prediction error.

Still, it remains noteworthy, especially for this dissertation, that salience, attention

(Butterfield & Mangels, 2003) and some explicit understanding might affect the FRN and possibly the reinforcement system.

I.4.2.4.4 Effect of medication on the FRN.

In contrast to other FRN theories, the RL theory specifically assumes a connection between the dopaminergic reward system and the FRN. A direct way to investigate this crucial point is the evaluation of the FRN after the administration of dopaminergic drugs.

Pramipexole, a D2/D3 receptor agonist that reduces firing rate and burst intensity of mescencephalic dopamine neurons, was found to decrease the FRN effect and impaired learning performance in a probabilistic learning task (Santesso et al., 2009). Specifically, there was a larger FRN after positive feedback in the pramipexole group compared to the control group, while the ERP after negative feedback was unaffected. This directly supports the assumption that the FRN reflects differences in the mesencephalic dopamine bursts. In addition, pharmaceutical studies with ERN amplitude as dependent measure (e.g., de Bruijn, Hulstijn, Verkes, Ruigt, & Sabbe, 2004; Ridderinkhof et al., 2002; Tieges, Richard

Ridderinkhof, Snel, & Kok, 2004; Zirnheld et al., 2004) are also in line with the RL theory.

I.4.2.4.5 Individual differences, psychopathology, and the FRN.

The ACC and the basal ganglia are believed to play an important role for individual

differences and psychopathology. Accordingly, the FRN should be associated with both. First, according to the RL theory and the BG-DA model, differences in the mescencephalic

differences and psychopathology. Accordingly, the FRN should be associated with both. First, according to the RL theory and the BG-DA model, differences in the mescencephalic

Im Dokument The role of errors and feedback for learning and decision making (Seite 37-62)