ELI_Rowan.Final

Working With Schools to Promote Instructional Improvement: Lessons from Research on CSR Programs : 

Working With Schools to Promote Instructional Improvement: Lessons from Research on CSR Programs Brian Rowan School of Education Institute for Social Research The University of Michigan

America’s Academic Achievement Problem : 

America’s Academic Achievement Problem The U.S.A. has among the highest rates of childhood poverty of all developed nations in the world. Not surprisingly, it also has among the highest socioeconomic disparities in educational achievement in the world. In the U.S.A, poor children enter school well behind non-poor children in academic achievement, and gaps in achievement widen as students progress. Much of the blame for this pattern is given to schools (not to poverty). Schools—and especially high poverty schools—have therefore become a target of intense social action aimed at instructional improvement

Helping “Failing” Schools Improve : 

Helping “Failing” Schools Improve Americans have worked on school improvement at two system levels: States (working with ED) have: upgraded academic standards tied sanctions to test performance These initiatives mostly tell schools what to achieve, not how to achieve it. And in this environment, many schools are still “failing” to meet standards. This leads to a second “level” of improvement work, where states and LEAs work directly with schools to help them improve. Examples include: State systems of support District-led school improvement initiatives

Helping “Failing” Schools Improve : 

Helping “Failing” Schools Improve In general, efforts to work directly with schools have been derived from two very different theories about schools as organizations and about how to fix them Theory One: Schools are seen as “organic” in form—require adaptive approaches to improvement: Great deal of variability in school circumstances High degree of “asset” specificity Therefore: use local professional community to “discover” effective designs Theory Two: Schools are seen as “mechanistic” in form—require programmed approaches to improvement: Great deal of uniformity in school conditions High degree of “asset” generality Therefore: develop scaleable models or designs for school improvement

American’s Often Favor “Adaptive” Intervention Programs : 

American’s Often Favor “Adaptive” Intervention Programs Educational researchers and professionals, especially, assume that schools operate “soft” technologies that cannot (and should not) be routinized. These same groups also prize “organic” management, wanting to move away from bureaucratic models of control. For this reason, much writing on planned educational change stresses the need for “adaptive” interventions. However, my research has questioned these assumptions about schooling: Teachers view teaching as a routine task In non-experimental studies, schools with organic management: Show greater teacher satisfaction But do not outperform other schools in terms of student achievement

What is the Evidence from Interventions Research? : 

What is the Evidence from Interventions Research? Adaptive models of school improvement occasionally produce big improvements in student outcomes, but mostly don’t: Reason is lack of “coherence” – “Christmas Tree” approach Not enough fidelity to “evidence-based” practice Programmed models are more coherent and have a better chance of improving student outcomes But: programmed models succeed only when certain conditions are met: Program is built around an effective instructional design Program has a sound plan for getting that design implemented. Such a plan: Works to improve a specific curriculum area across multiple grade levels Has clear protocol(s) for teachers/administrator action Protocols are spelled out in written materials Protocol compliance developed through extensive professional develop Program personnel coach/teach/monitor for implementation fidelity

This Evidence Leads to Comprehensive School Reform Models : 

This Evidence Leads to Comprehensive School Reform Models Move away from isolated “best practices” to “coherent designs” New American Schools (public/private venture) Federal CSR demonstration act US has developed a handful of demonstrably effective models US has established “What Works” and “Best Evidence” Clearinghouses Synthesize evaluation evidence Report to public on specific programs Idea is to create a consumer protection agency and promote evidence-based reform

Case Studies of CSR Programs : 

Case Studies of CSR Programs

The CPRE Research : 

The CPRE Research To investigate how school improvement interventions work, CPRE decided to study three well-known and widely-disseminated CSR programs: the Accelerated Schools Program America’s Choice Success for All This study was designed as a quasi-experiment: Two-thirds of schools in the study were in high poverty neighborhoods ≈ 30 schools were selected per intervention program, plus ≈ 30 comparison schools Samples of schools were matched through sampling and post hoc propensity score matching

Preview of Main Findings : 

Preview of Main Findings At the time of the research (2000-2004), the three CSR programs under study were using distinctive strategies to promote instructional change in schools. These strategies: Were built around distinctive instructional designs; Relied on different organizational processes to stimulate instructional improvement. We found that the programs had varying degrees of success in promoting instructional change. We found that the programs produced varying effects on student achievement in reading.

The Results Presented Here are Based on Extensive Data Collected in Schools : 

The Results Presented Here are Based on Extensive Data Collected in Schools We collected annual data on schools: Demographic data School structure and programs inventory School Leader Survey Teacher Questionnaire We collected extensive data on instruction and student learning: Two cohorts of students (≈ 24 per cohort per school) Teacher logs trace instructional experiences across time Fall-Spring assessment of students (Terra Nova) We collected survey data on student and family Background: Annual student motivation form Annual student rating form One-time parent survey

Finding 1: The CSR Models Used Different Approaches to Program Implementation : 

Finding 1: The CSR Models Used Different Approaches to Program Implementation

ASP Schools Pursued an “Adaptive” Approach to Improvement : 

ASP Schools Pursued an “Adaptive” Approach to Improvement

AC and SFA Schools Were More Programmatic in Emphasis : 

AC and SFA Schools Were More Programmatic in Emphasis

Only the Two “Programmatic” Approaches (AC and SFA) Produced Distinctive “Instructional Regimes” in Schools : 

Only the Two “Programmatic” Approaches (AC and SFA) Produced Distinctive “Instructional Regimes” in Schools

Instruction Was More Consistent and Distinctive in AC and SFA than ASP and Control Schools : 

Instruction Was More Consistent and Distinctive in AC and SFA than ASP and Control Schools AC SFA ASP Control 21 17 5

Accumulation of Instructional Coverage Across Years:SFA Comprehension Instruction : 

Accumulation of Instructional Coverage Across Years:SFA Comprehension Instruction + 138 days of skill-based instruction

Accumulation of Instructional Coverage Across Years:AC Writing Instruction : 

Accumulation of Instructional Coverage Across Years:AC Writing Instruction +117 days of literature-based reading instruction

Finding: Differences in Implemented Instructional Designs Affected Student Achievement : 

Finding: Differences in Implemented Instructional Designs Affected Student Achievement In ASP schools: Teaching practices were similar to those in the control schools. Patterns of achievement growth in reading comprehension were NO DIFFERENT from the control group. In SFA schools: Teachers used a “skill-based” teaching regime Achievement growth in reading comprehension: was HIGHER in the lower grades (+2 months additional growth/per year vs. control group) was NO DIFFERENT in the upper grades In AC Schools: Teachers used a “literature-based” teaching regime Achievement growth in reading comprehension: was NO DIFFERENT in the lower grades was HIGHER in the upper grades ( +3 to 4 months additional growth per year vs. control group)

Estimated Model Gains Translated into Percentile Rank Movement for Students in Lower Grades (RSS) : 

Estimated Model Gains Translated into Percentile Rank Movement for Students in Lower Grades (RSS)

Estimated Model Gains Translated into Percentile Rank Movement for Students in Upper Grades (RSS) : 

Estimated Model Gains Translated into Percentile Rank Movement for Students in Upper Grades (RSS)

What Do the Case Studies Show? : 

What Do the Case Studies Show? Instructional change is possible in American schools. It is not as hard as many researchers assume—even in high poverty schools Instructional change is made possible by school improvement efforts that: Focus on changing a targeted curricular area across multiple grade levels Are based on pre-existing designs that spell out clearly needed changes in curriculum and teaching Involve close monitoring for implementation fidelity These findings might require a change in our preferred approach to school change: Professional autonomy might not be a good model of change Local professional community might be less important than external partner in promoting to instructional improvement Instruction might require more proceduralization/standardization and external control

Applications Beyond the Programs : 

Applications Beyond the Programs Is it possible to implement the CSR design principles identified here without external assistance? That is, can local schools: Identify a curricular area in need of improvement? Develop a well-specified (and evidence-based) design for curriculum and instruction? Have leaders and support staff teach the design and monitor for fidelity? All of this might require a change in current management procedures: rather than needing more autonomy, perhaps local personnel need more central direction rather than needing locally-initiated designs for change, locals might need externally-developed and research-based designs for change rather than assuming good teachers are born, we should assume they are made (through design-based education, training, and supervision)

Problems Moving Forward : 

Problems Moving Forward Effective designs for school improvement are few in number: They are more prevalent in elementary reading and math, emerging for high schools, but largely absent at the middle grades Effective designs for school improvement are narrow in scope: They tend to focus on changing instruction to improve academic learning They pay less attention to socio-emotional learning They often work inside of schools, but ignore larger community-based resources In this light, we must become much better at “learning from practice.” We need more local experiments and quasi-experiments We need to experiment with blending academic and socio-emotional learning We need to experiment to find better uses of community resources for learning