Follow-up study with shorter contexts .1 Background and materials

The stimuli tested in experiment 8 were relatively long and complex as com-pared to those used by Merchant et al. (2013). Instead of a question-answer pair, they consisted of a context story and two turns per character. This might have biased subjects to base their ratings rather on the naturalness of the target ut-terance in discourse than on its grammaticality alone. In order to address this concern, I conducted a follow-up experiment using similar materials, but slightly longer context stories and two-turn dialogues which consisted only of the critical question-answer pair (54).

(54) [Context story] This weekend a famous painting has been stolen from the museum. The newscaster is reporting on the investigation of the robbery.

The investigators are currently discussing how the burglar got into the building.

[Newscaster:] Was glaubt Kommissar Wagner?

[Reporter:] Der Täter ist durch das Fenster eingestiegen(, glaubt er).

3.3.2.6.2 Method

The experiment was presented on the Internet using LimeSurvey. Originally, it was conducted in the same session as the production study on case marking (experiment 2), where subjects were asked to produce utterances referring to

graphical stimuli. The experiment was completed by 38 undergraduate students of Saarland University, who were rewarded with the participation in a lottery of 5

×€ 30.00 among all participants.⁴⁹The task and assignment to lists was identical to experiment 8 and Sententiality was tested between subjects again. Each sub-ject rated 21 items (7 per CCType condition). Materials were presented together with 35 items of experiment 10 and 25 unrelated fillers including four ungrammat-ical controls in individually pseudo-randomized order. Pseudo-randomization en-sured that no two items of the same experiment followed each other. One partic-ipant rated 50% or more of the ungrammatical attention checks as acceptable (6 or 7 points on the scale) and was therefore excluded from further analysis.

3.3.2.6.3 Results

Figure 3.10 shows the aggregated ratings across conditions. The pattern is similar to experiment 8, despite the slightly different ratings in absolute terms, which might be due to the differing materials that were tested together with the items in experiments 8 and the follow-up study.

Figure 3.10: Mean ratings and 95% confidence intervals across condi-tions in the follow-up to experiment 8.

49Due to a technical problem, two out of the six lists (which included 10 subjects) were assigned experimental materials in the incorrect conditions. The corresponding participants were re-placed by subjects recruited on theclickworkercrowdsourcing platform. These subjects rated the correct materials, which were mixed with the stimuli from experiment 10 and the same fillers as in the original lists. Each subject on the replacement lists was paid € 3.00 for partici-pating. Since the distribution issue does not concern the stimuli of experiment 10, in the case of experiment 10 I report the original data.

The statistical analysis followed the same procedure as for experiment 8. First, pairwise analyses compared two of the three levels of CCType at a time. In all three analyses there were significant effects of CCType or significant interac-tions of CCType and Sententiality, therefore I did not pool the data. In all of the analyses, the full model contained main effects for Sententiality, CCType and MatrixVerb as well as all two-way interactions. I also included by-subject random intercepts and slopes for CCType, MatrixVerb and their interaction, as well as by-item random intercepts and slopes for CCType, Sententiality and their interaction. By-item random effects for MatrixVerb were not considered because the matrix verb was not varied between items. The same holds for by-subject Sententiality random effects.

First, I analyzed only the data for the indicative (verb-second and verb-last) complement clauses. The final model is summarized in Table 3.18. A significant effect of Sententiality (𝜒² = 12.64, 𝑝 < 0.001) evidences an overall prefer-ence for fragments over sentprefer-ences with sentprefer-ence-initial CCs. The Sentential-ity:CCType interaction (𝜒² = 7.16, 𝑝 < 0.01) shows that in the case of fragments there is a preference for verb-last CCs with overt complementizers, which how-ever is not observed for sentences, since the main effect of CCType is not signif-icant (𝜒² = 1.67, 𝑝 > 0.1). The MatrixVerb did neither have a significant main effect nor did it interact with any of the other predictors.

Table 3.18: Fixed effects in the final CLMM for the verb-second and verb-last indicative conditions in the follow-up to experiment 8.

Predictor Estimate SE 𝜒² 𝑝

Sententiality 1.315 0.37 12.64 <0.001 ***

CCType -0.347 0.263 1.67 >0.1

Sententiality:CCType -1.289 0.472 7.16 <0.01 **

In a second analysis I compared only the indicative and subjunctive verb-second conditions. In this case, the only significant effect in the final model (see Table 3.19) is an interaction of Sententiality and Condition (𝜒² = 4.81, 𝑝 < 0.05): Subjunctive verb-second CCs are significantly degraded as compared to indicative ones as fragments as compared to the left dislocation conditions.

The effects of Condition (𝜒² = 1.681, 𝑝 > 0.1) and Sententiality (𝜒² = 0.01, 𝑝 > 0.9) are not significant but kept in the model due to the significance of the interaction. Again, there were no effects of MatrixVerb.

Finally, I compared only the data for the subjunctive and the verb-last CCs, which cannot be interpreted as indirect answers. The final model is summarized

Table 3.19: Fixed effects in the final CLMM for the indicative and sub-junctive verb-second conditions in the follow-up to experiment 8.

Predictor Estimate SE 𝜒² 𝑝

Sententiality 0.035 0.533 0.01 >0.9

CCType -0.45 0.343 1.681 >0.1

Sententiality:CCType -1.201 0.535 4.81 <0.05 *

in Table 3.20. The significant effect of CCType (𝜒² = 7.24, 𝑝 < 0.01) shows that subjunctive CCs are overall dispreferred as compared to verb-last CCs. In this analysis, the overall preference for fragments is only marginal (𝜒² = 2.8, 𝑝 > 0.05). Like in the analysis of the indicative data, the Sententiality:CCType interaction (𝜒² = 22.33, 𝑝 < 0.001) shows that verb-last CCs are particularly preferred as fragments. Again, there were no effects of MatrixVerb.

Table 3.20: Fixed effects in the final CLMM for the verb-second indica-tive and the verb-last subjuncindica-tive conditions in the follow-up to exper-iment 8.

Predictor Estimate SE 𝜒² 𝑝

Sententiality 0.579 0.345 2.8 >0.05

CCType -0.971 0.351 7.24 <0.01 **

Sententiality:CCType -2.543 0.53 22.33 <0.001 ***

3.3.2.6.4 Discussion

The follow-up study finds relatively similar results to experiment 8 with shorter contexts. Again, there is evidence for the preference for verb-last CCs with overt complementizers that Merchant et al. (2013) report in fragments, but not in com-plete sentences. Unlike in experiment 8, the subjunctive verb-second fragments were slightly degraded as compared to indicative verb-second fragments, but this effect is also not reflected in left dislocation structures in full sentences.

3.3.3 Experiment 9: CC topicalization in English