4. Does what you get depend on who you are with? Effects of student composition on teaching
4.7 Limitations and Outlook
Three limitations apply to the present study. First, the fact that we found no effects on teacher and observer ratings might result from the particular design of our study in which teacher and observer ratings were related to two specific teaching units for which all teachers used the same materials. We know that the quality of tasks plays a vital role in cognitive activation. Although establishing the same conditions for each teacher in the present study made it easier to attribute differences in teaching quality to student composition, in the case of cognitive activation it might also have led to a restricted variation and thus decreased effects.
Maybe teachers would have chosen different materials for their class based on achievement and motivational composition. Future studies with totally untreated samples will have to show if this interpretation is correct.
Second, by definition, a compositional effect occurs when a level 2 predictor has an effect over and above the effect of the same predictor at level 1. In the case of teacher and observer ratings, however, the outcome variables had no level 1 variance. Thus, it was impossible to estimate an effect at the individual level that could be used to correct classroom level effects. Therefore, the comparison of compositional
108
effects based on different raters’ perspectives should be interpreted with the consideration that the predictive models for teaching quality ratings by teachers and observers were not as robust as those for student ratings.
Further research utilizing teacher and observer data that addresses individual students could offset this shortcoming in the future.
Finally, we can speculate that in the context of educational systems with tracking after primary school, the sociocultural composition of the classroom might become more relevant for teaching quality in the later stages after the tracking, but these (undesired) effects were not present in grade three. Additionally, future research will have to show whether stronger measures of students’ SES background will yield similar findings. In the present study, we could only use student reports to assess SES, which limited the possibilities of taking into account the actual parental education and income appropriately.
4.8 Conclusion
In summary, the findings of our study are important in at least three ways: (1) By thoroughly examining the impact of composition on teaching quality, we were able to provide empirical evidence for the theoretical understanding of the mechanisms behind composition effects. (2) Such knowledge is not only of scientific interest but can also offer starting points for approaches seeking to address undesired effects of student composition: If teachers and policymakers are aware of the potential effects of student composition on teaching quality, they will then be able to develop strategies to diminish these effects. (3) Third, our findings also add to the theoretical understanding of teaching quality as an interactional process between students and teachers. They show that teaching quality is not merely a product of certain teachers’ actions in the classroom (Kennedy, 2010). Instead, teaching quality is co-constructed through interactions between students and teachers (Rimm-Kaufman & Hamre, 2010). Hence, our results suggest that class composition also matters.
109
ReferencesAllen, J., Gregory, A., Mikami, A., Lun, J., Hamre, B., & Pianta, R. C. (2013). Observations of Effective Teacher-Student Interactions in Secondary School Classrooms: Predicting Student Achievement with the Classroom Assessment Scoring System-Secondary. School Psychology Review, 42(1), 76-98.
Baumert, J., & Kunter, M. (2006). The COACTIV Model of Teachers' Professional Competence. In M. K.
e. al. (Ed.), Cognitive Activation in the Mathematics Classroom and Cognitive Competence of Teachers: Results from the COACTIV Project. New York: Springer.
Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., . . . Tsai, Y.-M. (2010). Teachers’
Mathematical Knowledge, Cognitive Activation in the Classroom, and Student Progress.
American Educational Research Journal, 47(1), 133-180.
Baumert, J., Stanat, P., & Watermann, R. (2006). Schulstruktur Und Die Entstehung Differenzieller Lern- Und Entwicklungsmilieus. In P. S. R. W. J. Baumert (Ed.), Herkunftsbedingte Disparitäten Im Bildungswesen: Differenzielle Bildungsprozesse Und Probleme Der Verteilungsgerechtigkeit (pp.
95 – 188). Wiesbaden: VS Verlag für Sozialwissenschaften.
Borman, G. D., & Kimball, S. M. (2005). Teacher Quality and Educational Equality: Do Teachers with Higher Standards-Based Evaluation Ratings Close Student Achievement Gaps? The Elementary School Journal, 106(1), 3-20.
Bos, W., Lankes, E.-M., Prenzel, M., Schwippert, K., Valtin, R., Voss, A., & Walther, G. (2005). Iglu.
Skalenhandbuch Zur Dokumentation Der Erhebungsinstrumente. Münster: Waxmann.
Brault, M.-C., Janosz, M., & Archambault, I. (2014). Effects of School Composition and School Climate on Teacher Expectations of Students: A Multilevel Analysis. Teaching and Teacher Education, 44, 148-159.
Brophy, J. E. (2000). Teaching (Vol. I). Geneva: International Academy of Education / International Bureau of Education (IAE).
Burns, R. B., & Mason, D. A. (2002). Class Composition and Student Achievement in Elementary Schools. American Educational Research Journal, 39(1), 207-233.
Casteel, C. A. (1998). Teacher–Student Interactions and Race in Integrated Classrooms. The Journal of Educational Research, 92(2), 115-120.
Clausen, M. (2002). Unterrichtsqualität: Eine Frage Der Perspektive? . Münster: Waxmann.
Coleman, J., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., F.D., W., & York., R. L.
(1966). Equality of Educational Opportunity. Washington, DC: U.S. Government Printing Office.
Creemers, B. P. M., & Kyriakides, L. (2008). The Dynamics of Educational Effectiveness. New York:
Routledge.
Curby, T. W., Rimm-Kaufman, S. E., & Ponitz, C. C. (2009). Teacher–Child Interactions and Children’s Achievement Trajectories across Kindergarten and First Grade. Journal of Educational
Psychology, 101(4), 912-925.
Decristan, J., Hondrich, A. L., Büttner, G., Hertel, S., Klieme, E., Kunter, M., . . . Hardy, I. (2015). Impact of Additional Guidance in Science Education on Primary Students’ Conceptual Understanding.
The Journal of Educational Research, 108(5), 358-370.
den Brok, P., & Levy, J. (2005). Teacher–Student Relationships in Multicultural Classes: Reviewing the Past, Preparing the Future. International Journal of Educational Research, 43(1), 72-88.
Desimone, L. M., Smith, T. M., & Frisvold, D. E. (2010). Survey Measures of Classroom Instruction:
Comparing Student and Teacher Reports. Educational Policy, 24(2), 267-329.
Doyle, W. (2006). Ecological Approaches to Classroom Management. In C. Evertson & C. Weinstein (Eds.), Handbook of Classroom Management: Research, Practice and Contemporary Issue. US:
Lawrence Erlbaum Association.
Dreeben, R., & Barr, R. (1988). Classroom Composition and the Design of Instruction. Sociology of Education, 61(3), 129-142.
110
Driessen, G. (2002). School Composition and Achievement in Primary Education: A Large-Scale Multilevel Approach. Studies in Educational Evaluation, 28(4), 347-368.
Duru-Bellat, M., & Mingat, A. (1998). Importance of Ability Grouping in French “Collèges” and Its Impact upon Pupils’ Academic Achievement. Educational Research and Evaluation, 4(4), 348-368.
Enders, C. K. (2010). Applied Missing Data Analysis. New York, NY: Guilford Press.
Evans, M. D. R., Kelley, J., Sikora, J., & Treiman, D. J. (2010). Family Scholarly Culture and Educational Success: Books and Schooling in 27 Nations. Research in Social Stratification and Mobility, 28(2), 171-197.
Fauth, B., Decristan, J., Rieser, S., Klieme, E., & Büttner, G. (2014a). Grundschulunterricht Aus Schüler-, Lehrer- Und Beobachterperspektive: Zusammenhänge Und Vorhersage Von Lernerfolg.
Zeitschrift für Pädagogische Psychologie, 28(3), 127-137.
Fauth, B., Decristan, J., Rieser, S., Klieme, E., & Büttner, G. (2014b). Student Ratings of Teaching Quality in Primary School: Dimensions and Prediction of Student Outcomes. Learning and Instruction, 29, 1-9.
Fricke, K., Ackeren, I. V., Kauertz, A., & Fischer, H. E. (2012). Students’ Perceptions of Their Teachers’
Classroom Management in Elementary and Secondary Science Lessons and the Impact on Student Achievement. In T. Wubbels, P. d. Brok, J. v. Tartwijk, & J. Levy (Eds.), Interpersonal
Relationships in Education: An Overview of Contemporary Research (pp. 167-185). Rotterdam:
Sense Publishers.
Gamoran, A. (1993). Alternative Uses of Ability Grouping in Secondary Schools: Can We Bring High-Quality Instruction to Low-Ability Classes? American Journal of Education, 102(1), 1–22.
Gilbert, M. C., Musu-Gillette, L. E., Woolley, M. E., Karabenick, S. A., Strutchens, M. E., & Martin, W.
G. (2014). Student Perceptions of the Classroom Environment: Relations to Motivation and Achievement in Mathematics. Learning Environments Research, 17(2), 287-304.
Gitomer, D. H., & Bell, C. A. (2013). Evaluating Teaching and Teachers APA Handbook of Testing and Assessment in Psychology, Vol. 3: Testing and Assessment in School Psychology and Education.
(pp. 415-444). Washington, DC, US: American Psychological Association.
Goldhaber, D., & Brewer, D. J. (2000). Does Teacher Certification Matter? High School Teacher
Certification Status and Student Achievement. Educational Evaluation and Policy Analysis, 22(2), 129-145.
Goldsmith, P. R. (2011). Coleman Revisited: School Segregation, Peers, and Frog Ponds. American Educational Research Journal, 48(3), 508-535.
Goodenow, C. (1993). Classroom Belonging among Early Adolescent Students: Relationships to Motivation and Achievement. The Journal of Early Adolescence, 13(1), 21-43.
Hamre, B., Pianta, R. C., Mashburn, A. J., & Downer, J. T. (2007). Building a Science of Classrooms:
Application of the Class Framework in over 4,000 U.S. Early Childhood and Elementary Classrooms.
Hanushek, E. A., Kain, J. F., Markman, J. M., & Rivkin, S. G. (2003). Does Peer Ability Affect Student Achievement? Journal of Applied Econometrics, 18(5), 527-544.
Harker, R., & Tymms, P. (2004). The Effects of Student Composition on School Outcomes. School Effectiveness and School Improvement, 15(2), 177-199.
Harris, D. N. (2010). How Do School Peers Influence Student Educational Outcomes? Theory and Evidence from Economics and Other Social Sciences. Teachers College Record, 112(4), 1163 – 1197.
Harris, D. N., & Sass, T. R. (2011). Teacher Training, Teacher Quality and Student Achievement. Journal of Public Economics, 95(7–8), 798-812.
Hattie, J. A. (2002). Classroom Composition and Peer Effects. International Journal of Educational Research, 37(5), 449 – 481.
Helmke, A. (Ed.) (2010). Unterrichtsqualität Und Lehrerprofessionalität. Diagnose, Evaluation Und Verbesserung Des Unterrichts. [Instructional Quality and Teacher Professionalism. Diagnosis, Evaluation and Improvement of Instruction]. Seelze: Klett-Kallmeyer.
111
Hutchison, D. (2003). The Effect of Group-Level Influences on Pupils' Progress in Reading. British Educational Research Journal, 29(1), 25-40.
Kennedy, M. M. (2010). Attribution Error and the Quest for Teacher Quality. Educational Researcher, 39(8), 591-598.
Klieme, E., Pauli, C., & Reusser, K. (2009). The Pythagoras Study: Investigating Effects of Teaching and Learning in Swiss and German Mathematics Classroom. . In T. Janik & T. Seidel (Eds.), The Power of Video Studies in Investigating Teaching and Learning in the Classroom (pp. 137-160).
Münster: Waxmann.
Klieme, E., Schümer, G., & Knoll, S. (2001). Mathematikunterricht in Der Sekundarstufe I:
"Aufgabenkultur" Und Unterrichtsgestaltung. In E. Klieme & J. Baumert (Eds.), Timss - Impulse Für Schule Und Unterricht (pp. 43-57). Bonn: Bundesministerium für Bildung und Forschung.
Kunter, M., & Baumert, J. (2006). Who Is the Expert? Construct and Criteria Validity of Student and Teacher Ratings of Instruction. Learning Environments Research, 9, 231-251.
Kunter, M., Baumert, J., & Köller, O. (2007). Effective Classroom Management and the Development of Subject-Related Interest. Learning and Instruction, 17(5), 494-509.
Lenhard, W., & Schneider, W. (2006). Elfe 1-6. Ein Leseverständnistest Für Erst- Bis Sechstklässler.
Göttingen: Hogrefe.
Lipowsky, F., Rakoczy, K., Pauli, C., Drollinger-Vetter, B., Klieme, E., & Reusser, K. (2009). Quality of Geometry Instruction and Its Short-Term Impact on Students' Understanding of the Pythagorean Theorem. Learning and Instruction, 19(6), 527-537.
Lüdtke, O., Robitzsch, A., Trautwein, U., & Kunter, M. (2009). Assessing the Impact of Learning Environments: How to Use Student Ratings of Classroom or School Characteristics in Multilevel Modeling. Contemporary Educational Psychology, 34(2), 120-131.
Lüdtke, O., Trautwein, U., Kunter, M., & Baumert, J. (2006). Reliability and Agreement of Student Ratings of the Classroom Environment: A Reanalysis of TIMMS Data.
Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus (Version 7) [Computer Software]. Los Angeles, CA:
Muthén & Muthén.
National Council of Teacher Quality. (2018). 2017 State Teacher Policy Yearbook. National Summary.
Retrieved from https://www.nctq.org/publications/2017-State-Teacher-Policy-Yearbook Nikolova, R. (2011). Grundschulen Als Differenzielle Entwicklungsmilieus. Objektive Kontextmerkmale
Der Schülerzusammensetzung Und Deren Auswirkungen Auf Die Mathematik- Und Leseleistungen. Münster: Waxmann.
Oakes, J., Gamoran, A., & Page, R. (1992). Curriculum Differentiation, Opportunities, Outcomes, and Meanings. In P. Jackson (Ed.), Handbook of Research on Curriculum (pp. 570–608). New York:
McMillan.
OECD. (2016). Pisa 2015 Technical Report. Paris: OECD Publishing.
Opdenakker, M.-C., & Damme, J. (2007). Do School Context, Student Composition and School Leadership Affect School Practice and Outcomes in Secondary Education? British Educational Research Journal, 33(2), 179-206.
Opdenakker, M.-C., & Van Damme, J. (2001). Relationship between School Composition and Characteristics of School Process and Their Effect on Mathematics Achievement. British Educational Research Journal, 27(4), 407-432.
Opdenakker, M.-C., & Van Damme, J. (2006). Differences between Secondary Schools: A Study about School Context, Group Composition, School Practice, and School Effects with Special Attention to Public and Catholic Schools and Types of Schools. School Effectiveness and School
Improvement, 17(1), 87-117.
Opdenakker, M.-C., Van Damme, J., De Fraine, D. F., Van Landeghem, G., & Onghena, P. (2002). The Effect of Schools and Classes on Mathematics Achievement. School Effectiveness and School Improvement, 13(4), 399-427.
Page, R. N. (1991). Lower Track Classrooms: A Curricular and Cultural Perspective. New York:
Teachers College Press.
112
Peetsma, T., van der Veen, I., Koopman, P., & van Schooten, E. (2006). Class Composition Influences on Pupils' Cognitive Development. School Effectiveness and School Improvement, 17(3), 275-302.
Pianta, R. C., & Stuhlman, M. W. (2004). Teacher–Child Relationships and Children's Success in the First Years of School. School Psychology Review, 33(3), 444−458.
Portes, A., & Hao, L. (2004). The Schooling of Children of Immigrants: Contextual Effects on the Educational Attainment of the Second Generation. Proceedings of the National Academy of Sciences of the United States of America, 101(33), 11920-11927.
Praetorius, A.-K., Klieme, E., Herbert, B., & Pinger, P. (2018). Generic Dimensions of Teaching Quality:
The German Framework of Three Basic Dimensions. ZDM - International Journal on Mathematics Education, 50(3).
Praetorius, A.-K., Pauli, C., Reusser, K., Rakoczy, K., & Klieme, E. (2014). One Lesson Is All You Need?
Stability of Instructional Quality across Lessons. Learning and Instruction, 31, 2-12.
Ready, D. D., & Wright, D. L. (2011). Accuracy and Inaccuracy in Teachers’ Perceptions of Young Children’s Cognitive Abilities: The Role of Child Background and Classroom Context. American Educational Research Journal, 48(2), 335-360.
Reinboth, M., Duda, J. L., & Ntoumanis, N. (2004). Dimensions of Coaching Behavior, Need Satisfaction, and the Psychological and Physical Welfare of Young Athletes. Motivation and Emotion, 28(3), 297-313.
Rice, J. K. (2003). Teacher Quality: Understanding the Effectiveness of Teacher Attributes. Washington, DC: Economic Policy Institute.
Rimm-Kaufman, S. E., & Hamre, B. K. (2010). The Role of Psychological and Developmental Science in Efforts to Improve Teacher Quality. Teachers College Record, 112(12), 2988-3023.
Rjosk, C., Richter, D., Hochweber, J., Lüdtke, O., Klieme, E., & Stanat, P. (2014). Socioeconomic and Language Minority Classroom Composition and Individual Reading Achievement: The Mediating Role of Instructional Quality. Learning and Instruction, 32, 63-72.
Rjosk, C., Richter, D., Hochweber, J., Lüdtke, O., & Stanat, P. (2015). Classroom Composition and Language Minority Students' Motivation in Language Lessons (Vol. 107).
Ross, L. (1977). The Intuitive Psychologist and His Shortcomings: Distortions in the Attribution Process.
In L. Berkowit (Ed.), Advances in Experimental Social Psychology, Vol. 10. New York: Academic Press.
Rubie-Davies, C. M. (2007). Classroom Interactions: Exploring the Practices of High- and Low-Expectation Teachers. Br J Educ Psychol, 77(Pt 2), 289-306.
Rumberger, R. W., & Palardy, G. J. (2005). Does Segregation Still Matter? The Impact of Student
Composition on Academic Achievement in High School. Teachers College Record, 107(9), 1999-2045.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass Correlations: Uses in Assessing Rater Reliability.
Psychological Bulletin, 86(2), 420–428.
Skinner, E. A., & Belmont, M. J. (1993). Motivation in the Classroom: Reciprocal Effects of Teacher Behavior and Student Engagement across the School Year. Journal of Educational Psychology, 85, 571-581.
Tenenbaum, H. R., & Ruck, M. D. (2007). Are Teachers' Expectations Different for Racial Minority Than for European American Students? A Meta-Analysis. Journal of Educational Psychology, 99(2), 253-273.
Thrupp, M. (1995). The School Mix Effect: The History of an Enduring Problem in Educational Research, Policy and Practice. British Journal of Sociology of Education, 16(2), 183-203.
Tomlinson, C., Brighton, C., Hertberg, H., Callahan, C., Moon, T., Brimijoin, K., . . . Reynolds, T. (2003 ). Differentiating Instruction in Response to Student Readiness, Interest, and Learning Profile in Academically Diverse Classrooms: A Review of Literature. Journal for the Education of the Gifted, 27(2/3), 119–145.
Urdan, T., & Schoenfelder, E. (2006). Classroom Effects on Student Motivation: Goal Structures, Social Relationships, and Competence Beliefs. Journal of School Psychology, 44, 331–349.
113
Van Ewijk, R., & Sleegers, P. (2010). The Effect of Peer Socioeconomic Status on Student Achievement:
A Meta-Analysis. Educational Research Review, 5(2), 134-150.
Walshaw, M., & Anthony, G. (2008). The Teacher’s Role in Classroom Discourse: A Review of Recent Research into Mathematics Classrooms. Review of Educational Research, 78(3), 516-551.
Wang, M. C., Haertel, G. D., & Walberg, H. J. (1993). Toward a Knowledge Base for School Learning.
Review of Educational Research, 63(3), 249-294.
Weiß, R. H., & Osterland, J. (2012). CFT 1-R: Grundintelligenztest Skala 1 Göttingen: Hogrefe.
Wentzel, K. R. (1997). Student Motivation in Middle School: The Role of Perceived Pedagogical Caring.
Journal of Educational Psychology, 89, 411–419.
Wilkinson, I. A. G., Hattie, J. A., Parr, J. M., & Townsend, M. A. R. (2000). Influence of Peer Effects on Learning Outcomes: A Review of the Literature. Wellington, NZ: Ministry of Education.
Wubbels, T., Brekelmans, M., & Hooymayers, H. P. (1992). Do Teacher Ideals Distort the Self-Reports of Their Interpersonal Behavior? Teaching and Teacher Education, 8(1), 47-58.
Zimmer, R. W., & Toma, E. F. (2000). Peer Effects in Private and Public Schools across Countries.
Journal of Policy Analysis and Management, 19(1), 75-92.
Zurbriggen, C. (2016). Schulklasseneffekte: Schülerinnen Und Schüler Zwischen Komparativen Und Normativen Einflüssen. Wiesbaden: Springer VS
114
4.9 Supplementary MaterialAppendix A Measures and variables
N Mean SD Min Max % α
Student ratings of TQ
Classroom Management 989 0 1 -3.36 1.14 0.92
Supportive Climate 989 0 1 -1.54 1.82 0.91
Cognitive Activation 981 0 1 -3.28 0.97 0.81
Student characteristics Number of books
More than 25 books 797 80.18
25 books or less 197 19.82
Migration Background
No migration background 510 55.98
One / both parents born outside
Germany 401 44.02
Cognitive abilities (CFT) 991 0 1 -2.77 3.24
Reading competence (ELFE) 981 0 1 -2.67 3.7
Interest 992 0 1 -2.35 0.98
Volition 993 0 1 -5.35 0.95
Classroom composition
Number of books 54 0.56 0.18 0.15 0.93
Migration Background 54 0.44 0.21 0.00 0.83
Cognitive abilities (CFT) 54 0.04 0.95 -2.79 2.97 Reading competence (ELFE) 54 0.04 0.94 -2.47 2.02
Interest 54 0.00 0.98 -2.40 1.34
Volition 54 0.02 0.97 -3.38 1.86
Teacher ratings of TQ
Classroom Management 50 -0.07 0.71 -0.17 6.93 0.84
Supportive Climate 41 -0.01 0.99 -2.70 1.20 0.86
Cognitive Activation 48 -0.00 0.98 -2.18 1.20 0.76
Observer ratings od TQ
Classroom Management
53
0.03 0.96 -2.06 0.99Supportive Climate
53
0.03 0.99 -2.81 1.27Cognitive Activation
53
0.01 0.96 -2.4 0.81Note. All continuous variables are standardized. α: Cronbach's α, TQ: Teaching quality