Samples, Designs, and Instruments

Samples

For both studies of this dissertation, student groups from secondary school were considered (eight grade for Study I, ninth grade for Study II). Secondary school, and this age group in particular, was chosen for several reasons. First, for this age, student characteristics have already formed in a distinguishable manner, yet is still evolving. This is important, since motivational-affective characteristics are measured

52 Chapter 5. Research Agenda using student questionnaires with self-assessment items (for details see below and methods sections of Study I and Study II in Chapters 6 and 7). Especially motivational-affective characteristics can be measured more reliably and have been formed with regards to specific domains, which is less pronounced for younger students (Marsh, Craven, & Debus, 1998; Krapp, 2000). The latter is crucial in this thesis’ aim to juxtapose different subjects. Furthermore, students’ dispositions in this age group play an especially important role since they also affect higher education and career choices made in the following years (Korhonen et al., 2016; X. Wang, 2013). For possible future interventions, this age group would therefore be especially interesting.

The student sample for both studies came from the higher tracks of German secondary schools. A student group from one or two tracks only (Gymnasium for Study I, Gymnasium and Realschule for Study II) was chose to highlight the within-student characteristics diversity in a group generally considered to be rather homogeneous.

In some scientific discussions, the tracked school system is already considered to be a measure of differentiation according to individual dispositions (Trautwein, L¨udtke, Marsh, K¨oller, & Baumert, 2006). While this might be true to some extent, Study I shows that even in a rather homogeneous group of students, the interaction of different cognitive and motivational-affective characteristics exhibits great diversity. Study II is a consecutive study to Seidel (2006). In this respect, it examines the development of a specific subgroup of students from this study, yet maintains all students for reference purposes. Germany was chosen primarily as the location of the researchers involved in the study. However, its tracked school system (see above) and the national discussion around diagnostic competence in political and local educational research settings make this country an interesting place for this research.

Student disposition data as well as teacher data (for Study I) was collected with regards to different subjects. Study I juxtaposes student dispositions for mathematics and language arts. Study II examines student dispositions with respect to physics.

Hence in total, three important domains of secondary education are included in the studies of this dissertation to offer insights on student learning dispositions for different domains.

5.2. Methodological Approach 53 Designs

The data for both studies was obtained as part of two larger studies: the Inter-action study, supported as ‘Opportunities to Learn’ by a research grant of the German Research Foundation (DFG, SE1397/7-1), for Study I and the German subsample of the IPN Video Study, supported as ‘Teaching and Learning Processes in Physics Instruction’ by a research grant of the German Research Foundation (PR 473/2–2), for Study II. Both studies were designed to connect students dispositions to learning processes. The Interaction study additionally incorporated teachers’ perspectives on student dispositions. Learning processes were measured by student questionnaires (see methods section below) as well as video-taped classroom observations. To focus on the student dispositions (the student and teacher perspectives on within-student characteristics diversity for Study I and the development and its connection to student perceptions of learning processes for Study II), the video observations were disregarded for the studies of this dissertation.

Study I only regards students’ dispositions at the beginning of the school year.

Due to the complexity resulting from the comparison of two perspectives (student and teacher) in two subject domains (mathematics and language arts), the examination of development was left to further research. Study II consideres student disposition at the beginning and the end of the school year and focuses on the development of a specific subgroup of students (the self-underestimating students). Student perceptions of learning processes were measures at mid-school year following the video-taped lesson.

Instruments

The two studies employed similar instruments to measure student dispositions.

Both employ a subscale of the Kognitiver F¨ahigkeitstest (KFT, cognitive ability test), a test frequently used in Germany to measure general cognitive ability (Heller &

Perleth, 2000). The subscale consisted of 25 figure analogy items, which were coded dichotomously (0 = incorrect, 1 = correct). Study I reports the total number, Study II the percentage of correctly answered items due to differences in the subsequent scaling for analyses.

As a further cognitive characteristic, Study I consideres the students’ prior

54 Chapter 5. Research Agenda achievement in terms of their grades from the previous school year. Despite the complexity behind the creation of school grades and discussions around their use in educational research, they are widely used as valid achievement measures (Cliffordson, 2008) and as such were assigned to the cognitive terrain in Study I. After all, prior achievement is a prominent predictor of learning success (Hattie, 2009; Marsh & Martin, 2011). Grades in Germany range from 1 (excellent) to 6 (insufficient), but were recoded so that a higher value indicates a higher prior achievement. In Study II, a knowledge test regarding physics was administered to measure the students’ prior knowledge. Items from various sources were adapted and details on this test can be found in the methods section of Chapter 7.

Both studies conducted student questionnaires surveying students on their interest and self-concept regarding the respective subject domain. Items for Study I were taken from the 2012 and 2009 questionnaires of the Program for International Student Assessment (PISA) going back to (Baumert, Gruehn, Heyn, K¨oller, & Schnabel, 1997) for interest and (Marsh, 1990b) for self-concept. Items for Study II were developed for the IPN Video Study (Hoffmann, H¨außler, & Peters-Haft, 1997; Hoffmann, H¨außler, &

Lehrke, 1998). The item developments for both studies originate in the same theoretical models of interest and self-concept regarding subject domains specified in the theoretical background in Chapter 2 (Adams & Wu, 2002; Seidel, Rimmele, & Dalehefte, 2003).

Teacher questionnaires on their perceptions of student dispositions and student questionnaires on their perception of internal learning activities were specific to each study and therefore described in the methods section of Chapter 6 and 7 respectively.