Action research & evaluation research

Category: Education

Presentation Description

Action research & evaluation research


Presentation Transcript

Action research & evaluation research:

Action research & evaluation research Lisa Ann Rodriguez, PhD © 2012


Content Action Research Rigor How to increase rigor What Action Research is and what it’s not Evaluation Research How Evaluation is like and how it’s different from Action Research Six Aspects of Education Formative vs. Summative evaluation Six Categories of Evaluation Models Methods of Evaluation Organization of Evaluation Research

Action research:

Action research

Rigor = Level of Quality:

Rigor = Level of Quality Quantitative Validity Reliability Qualitative Accuracy Credibility Dependability

How to Increase Rigor::

How to Increase Rigor: Repetition of cycle – cyclical nature of action research. Not just one experiment. Do it, modify it, observe, make records, modify, repeat Prolonged engagement, persistent observation. Deliberate, careful observations, surveys, interviews, etc. Experience with the process – depending on experience of the researcher. Previous studies? Novice—maybe bring in experienced facilitator TRIANGULATION of data. Multiple sources of data (i.e. test scores, teacher’s grades, students’ opinions) MEMBER CHECKING – ASK participants to look at the data and verify that the researcher got it right—interpreted interview or survey responses as was intended by the participant. Also they can then further explain their point of view. Participant Debriefing: Participants can provide more insight. Focus on emotions and feelings. Can talk about things that may have affected them – nervous, … or inhibit memory (lack of sleep?)

Action Research:

Action Research Done by educators for educators (not by professional researchers) Purpose – Reflection and positive change Focus on a new product, plan, or activity Good teacher buy-in because it’s practical and relevant to them. Teachers can access the findings and interpret them easily. It makes sense to them. Develops collaborative relationships among teachers. Can be more difficult than typical research. Not just the usual teaching practice. It’s planned, systematic observation and analysis. Not just problem solving. It has to refer to something NEW and reflection on that new thing (strategy, technology, curriculum, etc.) Not done to other people, like an experiment. It’s done on our own work for our own purposes. NOT generalizable. Its focus is at a LOCAL level. What it is…. What it’s not…

Evaluation research:

Evaluation research


EVALUATION RESEARCH Appraises quality of something that already exists (doesn’t develop something new) Larger in scope (whole district or school) Intended for broader application AKA Program Evaluation May be tied to funding Administrators need the data in order to decide to keep or dump a program? Determine the cost benefit How it’s similar to Action Research: How it’s different from Action Research: Practical Usually by school districts to determine effectiveness of products, procedures, curriculum, programs, etc. Done by educators (not professional researchers) Local level of focus Resolves problems or answers relevant questions NOT to be generalized to a larger population

Six Aspects of Education Evaluation Research Often Examines::

Six Aspects of Education Evaluation Research Often Examines: Organizations Educators Students Can be touchy—a sensitive subject. The evaluation is focusing on people rather than programs. Caution must be taken. Instructional methods Curriculum materials Programs Often done by graduate students. Educators like to know more about the materials and programs they use.

Formative vs. Summative Evaluation:

Formative vs. Summative Evaluation Done during development and implementation of a new program, product, strategy, or technology Identifies weaknesses Helps educators improve the program while it’s still being implemented Formative Summative Evaluates overall quality or effectiveness of a program Compares teachers and/or students’ performance against predetermined criteria.

PowerPoint Presentation:

6 Categories of Evaluation Research Models Focus on evaluating how well predetermined goals or objectives have been met. Focus on giving managers info with which to make informed decisions. Focus on evaluating products to inform decisions to buy or not. Focus on evaluating educational program quality. Focus on evaluating opposing points of view. All stakeholders involved in determining criteria and evaluating product or strategy. Click each hexagon for a definition. ( Mertler & James, 2011, Chapter 16, p. 7-8)

Methods of Evaluation (Mertler & James, 2011, Chapter 16, p. 9):

Methods of Evaluation ( Mertler & James, 2011, Chapter 16, p. 9) Comparative Content Analysis: components of method or program identifyied and analyzed. How well correspond to goal. Criteria used to judge value. Usually focuses on evaluation of textbooks, texts, programs Analysis of theoretical, philosophical or moral tenets…Examples: Culture & gender equity in textbooks and curriculum. Focus on evaluating if predetermined goals or objectives have been met. Teacher Acceptance – adoption by teachers…if they like and use a product. Measures reactions. Changes produced in teachers….Does in-service training carry over into classroom practice? Data from interviews, surveys, teacher observations. Student acceptance and involvement – Do students accept and participate willingly? Resultant student achievement or behavior – scores? Increased comprehension?

Organization of Evaluation Research:

Organization of Evaluation Research Problem statement Questions/ Hypothesis Design to obtain data Data Data Analysis Findings Conclusion


References Crotty , Y. (2010). Action Research Made Simple. Retrieved from http:// . Mertler , C. & Charles, C. (2011 ). Introduction to educational research (7 th ed.). San Francisco: Allyn and Bacon. ISBN: 9780137013449