scheerens

The Use of International Comparative Assessment Studies to Answer Questions about Educational Productivity and Effectiveness: 

The Use of International Comparative Assessment Studies to Answer Questions about Educational Productivity and Effectiveness Conference of the Centre for Livelong Learning (CRELL) of the EU on Methodological Tools for Accountability Systems in Education. Ispra, Italy 6-8 February, 2006 Jaap Scheerens University of Twente The Netherlands j.scheerens@gw.utwente.nl

Slide3: 

Average achievement in reading literacy (Source: PISA 2000 and PISA plus). Countries are ranked according to their average reading literacy score.

Slide4: 

Grade level equivalents relative to the United States (source: John H. Bishop, 1997)

Slide5: 

Ranking of the “top ten” countries in 8 international assessments; the last column shows the proportion of being in the top ten out of all studies in which the country participated (Sources: OECD, 2001 and 2004, IEA, 2004, a and b).

Slide6: 

Ranking of the “top ten” countries in 8 international assessments (continued)

Slide7: 

Estimates of the Variance Explained by Schools and Classes (Source: Scheerens, Vermeulen & Pelgrum, 1989) - based on Second International Mathematics and Science Study; Mathematics scores

Slide11: 

Partitioning of the between-school variance in reading, mathematical and scientific literacy by student background characteristics, school context variables, all school variables, and each of three groups of malleable school characteristics (Source: Scheerens & Visscher, 2004).

Slide12: 

Percentages of between school variance in reading literacy explained by student background variables, school context variables and malleable school variables

Predictor variables with significant positive (+) or negative (-) associations (5% level) with mathematics achievement, when the variance component model is analyzed by means of the VARCL-Programme – Scheerens et al., 1989 - SIMS: 

Predictor variables with significant positive (+) or negative (-) associations (5% level) with mathematics achievement, when the variance component model is analyzed by means of the VARCL-Programme – Scheerens et al., 1989 - SIMS

Predictor variables with significant positive (+) or negative (-) associations (5% level) with mathematics achievement, when the variance component model is analyzed by means of the VARCL-Programme - continued: 

Predictor variables with significant positive (+) or negative (-) associations (5% level) with mathematics achievement, when the variance component model is analyzed by means of the VARCL-Programme - continued

Slide16: 

Teacher and school indicators discriminating effective and ineffective schools (top 15) (Source: Postlethwaite & Ross, 1992)

Slide17: 

School variables significantly related to reading literacy performance, after adjustment for student background characteristics (Source: Scheerens & Visscher, 2004). *) significant for OECD–countries only

Most effective schools according to Willms and Somers (2001):: 

Most effective schools according to Willms and Somers (2001): high levels of school resources, including a low pupil-teacher ratio, more instructional materials, a large library, and well-trained teachers; classrooms which are not multigrade, and where students are not grouped by ability; classrooms where students are tested frequently” classrooms and schools with a high level of parental involvement; and classrooms that have a positive classroom climate, especially with respect to classroom discipline

Interesting areas for further reflection: 

Interesting areas for further reflection Consideration of selection, admission and grouping policies as a way to stimulate productivity Study of gradients (achievement / SES) to diagnose in equity patterns

Methodological advances: 

Methodological advances Alternative interpretations of school effects Use of multi-level IRT More elaborate modeling (multi-path) More elaborate items (scales) in background questionnaires Outlier designs Longitudinal designs

Intermediary causal structure of leadership at school : 

Intermediary causal structure of leadership at school

Regression-discontinuity(Cutting-point design): 

Regression-discontinuity (Cutting-point design) Assignment of students to grades determined by date of birth (e.g. cut-off point = 1 Sept.) Effect of one year schooling = difference in achievement between upper and lower grade minus effect of date of birth (age) Y = β0 + β1AGE + β2GRADE

Integration of multilevel analysis and regression-discontinuity: 

Integration of multilevel analysis and regression-discontinuity Assessment of absolute effect (overall) through regression-discontinuity approach Assessment of variation of the effect between schools through multilevel analysis

PISA 2000; relationship between age and student performance (Luyten, 2006): 

PISA 2000; relationship between age and student performance (Luyten, 2006)