Cross Modal Fragment Priming and its Behavioural and Electrophysiological Correlates.17

If these assumptions are true and [] is underlyingly specified as [LOW] in Turkish and

lexicon differs in activation pattern from a German mental lexicon. Such a case can be created in cross modal fragment priming.

In cross modal fragment priming, a word fragment is presented acoustically to a subject. Immediately at its offset, a word is presented visually on a screen. This word can either contain the fragment that has just been heard, it can differ slightly from the preceding fragment or it can be completely unrelated to it. The same proportion of words and pseudowords is used as visual target words. Participants are asked to do a lexical decision on the targets, indicating by pressing a button whether they had just read a word or a pseudoword.

It is generally assumed that the lexical entry for a word is a modality-independent entity. Auditory and visual information is first processed via two modality-dependent access routes that work independently of each other but feed both into modality-independent lexical entries (Marslen-Wilson, 1990). Cross modal priming provides us with the possibility to investigate lexical access, ruling out the danger that the data are simply due to superficial form priming effects. Expectations on the effects of cross modal fragment priming are based on the assumption, that incoming speech input activates a cohort of possible word candidates.

As more and more information about the input is available, this cohort is narrowed until one item remains. In most cases a fragment, namely the onset of a word, presented acoustically, does not provide the mental lexicon with sufficient information to restrict the cohort on a single item. More than one possible target will stay activated. It is hypothesised, that the activation level of a word in the mental lexicon is mirrored by a person’s reaction time. That is, if the word, that appears on the screen after the fragment has been perceived, has been activated by that fragment and still remains in the active cohort, then the subject will respond to it faster. For instance, Marslen-Wilson (1990) has shown that the acoustic presentation of the fragment fee-, gained from the English word feel, speeded responses to the visual targets feel as well as feed. Subjects were faster to tell that feel and feed are words, compared to some completely unrelated word like name.

Cross modal fragment priming has also been shown to work in semantic priming. In Dutch, the acoustic fragment kapi led to faster lexical decisions in response to the visual targets geld (‘money’) and schip (‘ship’) as compared to semantically unrelated targets. Geld is semantically related to kapitaal (‘capital’), schip is related to kapitain (‘captain’), both beginning with kapi- (Zwitserlood, 1989).

In the examples reported so far, the acoustic input was entirely identical to the probes that were primed by it. Before, it was postulated, that the mental lexicon is underspecified for

certain features. Therefore, it is expected to find cross modal fragment priming effects also in cases where the probe deviates from the acoustic prime, as long as this deviation does not lead to a mismatch condition. These assumptions have been confirmed in several studies. For instance, Friedrich, Lahiri and Eulitz (submitted) conducted the experiment described in section 1.4.4 once more with a cross-modal fragment priming design. The identical coronal fragment (e.g. drach-) as well as the non-coronal pseudoword-onset (*brach-) were assumed to prime the target (in this case Drachen) as compared to unrelated primes. On the contrary, non-coronal targets should have only been primed by their identical, non-coronal primes (e.g.

gren-Grenze), but not by their coronal pseudoword primes (e.g. dren-Grenze). Not only reaction times were measured but also the EEG signal was recorded. The reaction time results did not confirm the FUL model. In both conditions only the identical fragments primed.

However, the design of the study was not a proper behavioural design but served the needs of an EEG-experiment. Event related potentials confirmed the assumptions of the FUL model.

In cross modal fragment priming studies, a component called P350 has been consistently observed. This P350 ranges between 200 and 400 ms and differentiates matching from mismatching or unrelated prime-target pairs (Friedrich, Kotz, Friederici & Alter, 2004;

Friedrich, Kotz, Friederici & Gunter, 2004). Peaking at about 350 ms after the onset of the visual stimulus, the ERP amplitude is consistently more negative for matching prime-target pairs as compared to mismatching or unrelated conditions. The P350 effect has been defined as a difference wave, resulting from the subtraction of the somehow matching or mismatching conditions from the unrelated condition. Subtraction of matching words from unrelated control words reveals more positive amplitudes than subtraction of mismatching words from unrelated controls. This effect is observed predominantly over the left hemisphere.

For the above mentioned experiment the predicted asymmetric results were indeed obtained in the EEG data. The P350 difference wave for target words with coronal onsets was equally positive for both conditions, the one with the identical prime and the one with the prime that was altered in Place of Articulation. However, the P350 effect was reduced when non-coronal words were preceded by fragments with coronal Place of Articulation as compared to matching Place of Articulation.

word in the context of a preceding sentence or priming situation (Friedrich, 2005). The more expected this word, the less effort is needed, the smaller the N400 amplitude. However, in a fragment priming design, the N400 so far has not shown any asymmetric effects. N400 amplitudes are reliably enhanced in the control conditions, where the priming stimulus is completely unrelated from the target, as compared to conditions with identical priming.

Reactions to prime stimuli that slightly deviate from the identical prime can not be differentiated from reactions to situations of identical priming on grounds of the N400 amplitude, no matter whether they cause a nomismatch or a mismatch condition according to the FUL model (Friedrich, 2005).

To sum up, in a cross modal fragment priming experiment, the following responses are expected: In a matching or at least not mismatching situation, the subjects should respond significantly faster to the target words and their P350 difference wave amplitudes should be more positive than in a mismatching or unrelated condition. Furthermore, the N400 amplitude should be more negative in the unrelated control condition than in the experimental conditions.

1.6.3 Experimental Design and Hypotheses for the Present EEG Study 1.6.3.1 In Search for a Phenomenon that Allows Insight

As reported before, the aim of the present study was to test the mental representations of the vowel [] in Turkish and German. The assumption was, that [] is specified as [LOW] in Turkish, while it remains unspecified in German. In order to base this assumption on some experimental evidence, a cross modal fragment priming study was designed. Naturally, an [] would prime itself in any language that has this sound. The crucial question in this experiment was, whether an [] would be able to activate the lexical representation of an [] in the mental lexicon. An [] is definitely produced and perceived as a high, coronal vowel in both languages. In German, it is also assumed to be stored as [HIGH], while for Turkish there is controversy whether it is stored as [HIGH] or as unspecified for Tongue Height, as illustrated in section 1.6.1.2. Table 1.2 lists the place features that are extracted from the speech signal and those that are assumed to be stored in the mental lexicon for the vowels [] and [].

Table 1.2: Surface forms and underlying representations of the vowels [] and [] in Turkish and German. “Signal” refers to the place features extracted from the acoustic signal,

“Representation” refers to the place features stored in the mental lexicon.

Turkish:

Signal Representation [] [COR][HIGH] [HIGH] or []

[] [COR] [LOW] German:

Signal Representation [] [COR][HIGH] [HIGH]

[] [COR] []

It is obvious from Table 1.2 that the place feature [CORONAL] appears only in the signal, but is not represented in the mental lexicon. In case of the [], no Tongue Height feature is extracted from the signal, since it is not produced as a low vowel, but rather as a mid one. In fact, [] does not differ in production between the two languages, but only in its representation. Table 1.3 shows the activation pattern that is expected, as these vowels are matched onto their own mental representation. Since there are features in the signal that are not specified in the lexicon, and also features in the mental lexicon that are not extracted from the signal, the result is a nomismatch in all cases. In fact, a matching pattern can be given for every single feature of a segment. In order to keep matters less complex, in the table only one overall matching pattern is given. That is, as soon as one nomismatch occurs, the overall matching condition will be assigned a nomismatch-label. The same holds true for a mismatch.

Table 1.3: Matching pattern that is expected as the vowels [] and [] are mapped onto their own mental representation. “Signal” refers to the place features extracted from the acoustic signal, “Representation” refers to the place features stored in the mental lexicon.

Prime: [] - Target: []

Signal of [] Representation of [] Matching

Turkish [COR] [LOW] nomismatch

German [COR] [] nomismatch

Prime: [] - Target: []

Signal of [] Representation of [] Matching Turkish [COR][HIGH] [HIGH] or [] nomismatch German [COR][HIGH] [HIGH] nomismatch

Crucial for the following experiment was the difference in expected matching patterns between Turkish and German, as an [] in the signal is mapped onto the mental representation of an [] and vice versa. This scenario is illustrated in Table 1.4.

Table 1.4: Matching pattern that is expected as the vowels [] and [] are mapped onto each other’s mental representation. “Signal” refers to the place features extracted from the acoustic signal, “Representation” refers to the place features stored in the mental lexicon.

Prime: [] - Target: []

Signal of [] Representation of [] Matching Turkish [COR] [HIGH] or [] nomismatch German [COR] [HIGH] nomismatch Prime: [] - Target: []

Signal of [] Representation of [] Matching Turkish [COR][HIGH] [LOW] mismatch

German [COR][HIGH] [] nomismatch

An [] in the signal that is mapped onto an underlying [] also results in a nomismatch situation in both languages because no Tongue Height Feature is extracted from the signal.

Note that in this case it doesn’t make a difference whether Turkish [] is represented as

[HIGH] or underspecified. However, as an [] is presented in the signal, the extracted Tongue Height feature [HIGH] will mismatch with the underlying representation of [] as [LOW] in Turkish. For German once more a nomismatch is expected due to the underspecified Tongue Height feature.

1.6.3.2 Converting Abstract Assumptions into Concrete Predictions

In order to test these assumptions, a cross modal fragment priming design was set up. For this purpose, Turkish and German words with an [] or an [] as the nucleus of their first syllable were collected, thirty of each group. Since in Turkish tense [e] and tense [i] do not exist, German stimuli were also restricted on their lax counterparts [] and []. This was done to assure that the German and Turkish items did not differ in vowel quality. The design included three conditions: In the identity condition, the first syllable of the target word was used as the auditory prime. The related condition differed from the identity condition in that the vowel of the prime syllable was altered. That is, had it contained an [], this was replaced by an [] and vice versa. The syllable that was newly created in this way was not allowed to form the first syllable of any existing word in the language in question. The aim of this related condition was to see whether it would still prime the target word. In order to be able to unequivocally interpret the results, this prime item was not wanted to activate a cohort on its own. In the unrelated condition, the target word was preceded by an entirely different priming stimulus that shared no segments with its target.

Table 1.5: German and Turkish items as an illustration of the conditions realised in the experiment.

Conditions Identity Related Unrelated

bech-Becher (‘mug’) bich-Becher zan-Becher German

skiz-Skizze (‘sketch’) skez-Skizze bag-Skizze teh-tehdit (‘threat’) tih-tehdit fab-tehdit Turkish

bib-biblo (‘library’) beb-biblo nok-biblo

Hypotheses for German

Predictions for German state that in the identity as well as in the related conditions the auditory prime activates a cohort that includes the target word.

Reaction Time: German subjects are expected to be faster in telling that the target is an existing German word if it is preceded by an identical or related prime fragment and slower if the target follows an unrelated fragment.

P350: Both, identity and related condition are expected to differ significantly from the unrelated condition. More precisely, approximately 350 ms after stimulus onset, the amplitudes of the identical and related conditions should be significantly more negative than the amplitude of the unrelated condition. No difference in amplitude is predicted between the identical and the related conditions. Such a result would indicate that in both conditions the target word has been automatically activated in the mental lexicon by the preceding prime fragment.

N400: Also the N400 amplitude should be reduced for the identity as well as for the related condition as compared to the fully unrelated condition due to higher expectedness of the target in the former two conditions.

Symmetry: For the German subjects these predictions hold true irrespective of the order of presentation. This means, an [] in the related prime should activate its corresponding [ ]-word as well as an []-prime is expected to activate an []-word.

Hypotheses for Turkish

For the Turkish subjects, similar outcomes are expected for the identity conditions and for the related condition in which an []-fragment primes its corresponding []-word. In the related condition with reverse order, however, the []-fragment is predicted to mismatch with the representation of the corresponding []-word (see Table 1.4).

Reaction Time: Long reaction times, comparable to those of the unrelated condition, are expected for the related condition where an []-fragment precedes an []-word. In the related condition with reversed prime-fragment order and in the identical condition reactions will be faster.

P350: ERP amplitudes should reveal less negative values for the mismatching related condition than for the identical condition and the related condition resulting in a nomismatch.

This is because an []-fragment is not assumed to trigger lexical activation of []-words. The unrelated condition is assumed to have the least negative amplitude.

N400: The N400 will not differ between the two related conditions and the identical condition. It is simply expected to be enhanced for the unrelated control condition.

Asymmetry: Due to different matching patterns asymmetric priming results are expected in the two related conditions for Turkish subjects.