Analyses

!

Figure 5: Experimental conditions for experiments 3, 5 and 6. In conditions a) and b) participants were presented with a (congruent) positive emotional prime face in the form of a dynamic natural facial expression of a woman. By contrast, the primes in conditions c) and d) were negatively valenced and thus incongruent with the target characters’ facial expression and the sentence valence. In conditions a) and c) the presented potential agents performed a depicted action towards the patient. In conditions b) and c), the actions were absent. The related positively valenced emotional OVS sentence (‘who is doing what to whom’) was identical in all conditions.

!

80 ms were automatically merged with the nearest fixation (threshold distance 0.5 degrees) by the software.

5.2.2 Word Regions

We divided the experimental OVS sentences into word regions using trial-by-trial word onsets (see Appendix A.2). Hence the NP1 region extended from NP1 onset until Verb onset for each trial. Fixations starting before NP1 onset were excluded. The Verb region extended from Verb onset until Adverb onset and the NP2 region extended from NP2 onset until 500 ms after NP2 offset. The long NP2 region permitted us to capture potential late visual context effects, especially regarding possible processing differences in children and older adults. Henceforth, this region will simply be called ‘NP2 region’. Moreover, we analyzed a combined Verb-Adverb region and a region comprising the whole sentence, i.e., a Long region. We chose to analyze the Long region to capture visual cue effects on the processing of the sentence as a whole, irrespective of a specific word region. Furthermore we computed a preNP1 region ranging from the onset of the scene display until NP1 onset to capture potential effects of the emotional prime face that are solely based on prime and target scene effects, independent of any linguistic input. This preNP1 region had a duration of 2000 ms (see Figure 3).

For each word region, we divided the fixation record provided by the Data Viewer software (SR Research, Mississauga, Ontario, Canada) into consecutive 20 ms bins comprising the longest duration of a word region across experimental sentences (see Table 1). Fixation counts were then aggregated for each word region by region of interest (agent vs. distractor) and by prime (positive emotional vs. non- / incongruent emotional) and action (depicted action vs. no action). We aggregated the data separately for subjects and items.

!

NP1 Region Verb Region Adverb Region NP2 Region + 500 ms

Den Marienkäfer kitzelt vergnügt der Kater.

Verb-Adverb Region kitzelt vergnügt Long Region

Den Marienkäfer kitzelt vergnügt der Kater.

English ‘The ladybug tickles happily the cat.’

Mean word onset

0 ms 1228 ms 2380 ms 3720 ms 5060 ms

Longest duration

1620 ms 1540 ms 1740 ms 1180 ms 500 ms

Table 1: Analyzed German word regions, their English translation, mean word onsets and longest duration for each word region. Note that regions were computed for each item individually. We analyzed the preNP1 region (not shown in the table), NP1 region, Verb region, Adverb region, NP2 + 500 ms region (henceforth

‘NP2 region’), Verb-Adverb region and Long region.

5.2.3 Inferential Analyses

Since our main interest lies in what happens in incremental processing of the sentence especially during the Verb and Adverb region (see Section 5.4.1), we performed repeated measure ANOVAs on the fixation data for separate word regions, namely preNP1, NP1, Verb, Adverb, Verb-Adverb, NP2 and Long Region, with special focus on the Verb and Adverb regions. Although all word regions described in Table 1 were analyzed, the critical word regions for the cue integration effects were the verb and adverb of the OVS sentence. The verb was the first region of the sentence in which the depicted action could be referenced to the linguistic input. The adverb was the first region in which the emotional prime could be linked to the positive valence of the adverb and the target agent’s happy facial expression¹⁰.

Prime (prime vs. incongruent prime) and action (depicted action vs. no action) were used as factors in the ANOVAs. Note however, that prime was the only factor used for the preNP1 and NP1 analysis, since during these regions participants cannot

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

10 !Note that because all critical sentences were positively valenced, the verb often referred to a positive but never a negative action (e.g., to kiss, to hug, but also to take a picture, see Appendix A.1). Yet, we did not strictly control for the verb valence and hence take the positively valenced adverb as the first region in which the positive prime can be linked to the positive valence of the sentence and the agent’s happy facial expression. Nevertheless, earlier effects of the positive prime face, i.e., effects in the Verb region could be due to the valence of the verb.!

!

yet link the verb to the depicted action, as they haven’t heard the verb yet. For age comparisons, we included age group as a between-subject factor in the subject analyses and as a within-subject factor in the item analyses.

We performed inferential analyses for each word region by subjects (F1) and items (F2). Data analyses relied on the methods used in Arai et al. (2007). For each word region the aggregated fixation counts (that fell into a word region) by subjects and items were used to calculate the probabilities of fixations towards the agent versus the distractor. Fixations towards the agent in one condition were divided by the fixations towards the distractor in the same condition to calculate the probabilities of looks towards agent and distractor per condition. A constant of 0.5 was added to the fixation counts of the agent and distractor region per condition. The constant was added because the natural log of 0 is undefined; hence the log cannot be computed if fixations to the agent region divided by fixations to the patient region equals 0.

We transformed the probabilities of looks into log-ratios because these probabilities are not linearly independent of one another, i.e., more fixations towards the agent imply fewer looks to the distractor. Log ratios on the other hand express the strength of the visual bias towards the agent relative to the distractor. If both agent and distractor were fixated equally frequently, the score of the log ratios would be 0.

If the values were positive, the agent was fixated more than the distractor. Negative values, by contrast, indicated more looks towards the distractor than the agent. Log ratios are symmetrical around 0, meaning that the absolute score reflects the magnitude of an effect, whereas the direction of the visual bias is expressed by the valence. Moreover, probabilities of looks violate the assumption of homogeneity of variance and are thus not well suited for parametric tests such as repeated measure ANOVAs. Log ratios on the other hand are more suitable because they can take on values between minus and plus infinite in contrast to values between 0 and 1 as it is the case for probabilities (see Arai et al., 2007).

We subjected accuracy scores of the comprehension question for the experimental items to repeated measure ANOVAs for subjects and items with prime and action as within-subject factors¹¹. In experiments 1, 2 and 4 voice (active vs. passive) was used

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

11 !We additionally ran generalized linear mixed model analyses on the accuracy data for all 6 experiments. See Appendix E for details on the mixed model analyses and their accuracy results for all experiments. The results of the mixed model analyses mainly corroborate the results of the ANOVA analyses.!

!

as an additional within-subject factor. For age comparisons, we included age group as a between-subject factor in the subject analyses and as a within-subject factor in the item analyses.

5.2.4 Descriptive Analyses

To descriptively analyze the eye-movement results, we computed time course graphs comprising the whole sentence, i.e., the Long region. The fixation record from NP1 onset¹² to question screen onset was divided into 20 ms bins and aggregated by region of interest, prime and action. A constant of 0.5 was added to each data point before computing log ratios of looks of agent over distractor for each condition. These log ratio of looks are displayed in time course graphs, i.e., line graphs, to visually represent the results of the inferential statistic and to inspect the changes of the effects over time as the sentence unfolds. Values above 0 indicate a preference to look at the agent while values below 0 indicate a preference to look at the distractor (unless otherwise stated). The time course graphs are visually divided into sentence regions based on mean word onsets¹³ (see Table 1). We analyzed the accuracy scores for the memory of the prime face (experiments 3, 5 and 6) descriptively only, as they yielded too few data points per condition per study (160 observations per study, i.e., 1 data point in each condition per subject).