brodeur lrng assessmt

Learning Assessment: 

Learning Assessment Doris R. Brodeur, Ph.D. Massachusetts Institute of Technology dbrodeur@mit.edu November 2003

Today’s Learning Objectives : 

Today’s Learning Objectives This presentation is designed so that you will be able to Recognize that assessment practices are based on assumptions and principles of teaching and learning Explain the keys to sound classroom assessment Select appropriate assessment methods matched to learning objectives Compare different assessment methods in terms of advantages, limitations, and appropriateness Use feedback to support student learning and improve instruction.

Review: A Learning Design (Wiggins & McTighe, 1998): 

Review: A Learning Design (Wiggins & McTighe, 1998)

Learning Assessment(Huba & Freed, 2000): 

Learning Assessment (Huba & Freed, 2000) The process of gathering and discussing information from multiple and diverse sources in order to develop a deep understanding of what students know, understand, and can do with their knowledge as a result of their educational experiences The process culminates when assessment results are used to improve subsequent learning

Teacher-Centered vs. Learner-Centered(Huba & Freed, 2000): 

Teacher-Centered vs. Learner-Centered (Huba & Freed, 2000) Teaching and assessing are separate Assessment is used to monitor learning Emphasis is on right answers Desired learning is assessed indirectly through the use of objectively scored tests Culture is competitive and individualistic Only students are viewed as learners Teaching and assessing are intertwined Assessment is used to promote and diagnose learning Emphasis is on generating better questions and learning from errors Desired learning is assessed directly through papers, projects, performances, portfolios, etc. Culture is cooperative, collaborative, and supportive Professor and students learn together

Reflection : 

Reflection Which characteristics best describe your beliefs and practices as a teacher? In what ways does your practice seem to fall within the teacher-centered approach? In what ways does your practice seem to fall within the learner-centered approach?

Basic Assessment Principles: 

Basic Assessment Principles Assessment requires attention to outcomes and equally to experiences that lead to those outcomes. Different types of learning objectives require different methods of assessment. Measures of learning are external indicators of internal states. Any assessment is only a sample. Assessment works best when it is ongoing, not episodic. There are trade-offs between authenticity and efficiency, i.e., the closer the tasks are to real-world experiences, the more time and resources they require.

Discussion: 

Discussion Divide into six groups with each group taking one assessment principle. In your own words, explain the meaning of the assessment principle as it relates to your courses. Give examples, if possible. Choose someone to report to the whole group.

Keys to Sound Assessment(Stiggins, 1997): 

Keys to Sound Assessment (Stiggins, 1997) Clear objectives Clear purpose Appropriate sampling Appropriate methods Control of bias

Review: Clear Objectives(What to assess?): 

Review: Clear Objectives (What to assess?) What are the characteristics of clear and effective learning objectives? What are the categories of learning objectives into which your own course objectives fall?

Clear Purpose(Why assess?): 

Clear Purpose (Why assess?) To define valued outcomes for students To capture students’ time and attention To generate appropriate student learning activities To help students internalize the discipline’s standards To identify opportunities for improvement To provide a basis for instructor and program evaluation

Appropriate Methods(How to assess?): 

Appropriate Methods (How to assess?) “Appropriate assessment engages students in exactly the kind of learning activity you want to take place.” (Gibbs, 1999) Appropriate for Learning objectives Purpose of assessment Audience Valid and reliable methods Practical, feasible, efficient

Control of Bias(How to assess more validly?): 

Control of Bias (How to assess more validly?) Bias: A situation in which an assessment produces results which give one group an advantage or disadvantage over other groups because of factors such as native language, gender, or race resulting in the assessment representing different outcomes for different groups. Guidelines: Select appropriate assessment methods for specified learning objectives Use written records rather than relying on memory Create rubrics with specific criteria and scoring schemes Assess student work or performance closely following completion of the task

Appropriate Sampling(How much to assess?): 

Appropriate Sampling (How much to assess?) All assessment is only a sample of what students know or can do. Guidelines: Be sure that assessment questions and tasks are matched to specified learning objectives Use a variety of assessment methods to collect information about students’ abilities, achievements, and performance Assess students and provide feedback at several regular time intervals in the course

Categories of Learning Objectives : 

Categories of Learning Objectives Conceptual understanding Problem solving Skill/process Creation or synthesis Attitude mastery of content; “knowing what” or “knowing about”; deep learning use of knowledge to solve problems “knowing how” to do something synthesis of knowledge and skills to produce something new Disposition, opinion; affective domain

Categories of Assessment: 

Categories of Assessment Written exams Oral exams Performance Products Reflection and self-report

Matching Assessment With Learning Objectives: 

Matching Assessment With Learning Objectives

Exercise: 

Exercise Divide into the same groups as before. Using the matrix provided, match the categories of learning objectives with appropriate assessment methods. Place an X in the box where matches are appropriate. Give reasons for your choices.

Written Exams and Quizzes: 

Written Exams and Quizzes Concept questions and multiple-choice questions Open-ended essay questions Case study or critical incident CAT’s from Angelo and Cross (1993): Memory Matrix (p.42) Analytic Memo (p. 177) Concept Maps (p. 197) Directed Paraphrasing (p.232) Student-Generated Test Questions (p. 240)

Oral Exams and Interviews: 

Oral Exams and Interviews Spontaneous questions put to students by experts. Students must think on their feet, draw upon relevant facts, theories, and/or perspectives, and speak in a coherent, organized fashion. (See sample rubric for documenting and assessing oral exams.)

Performance Assessment: 

Performance Assessment Students prepare and present a performance of a valued activity Examples: Oral presentations and technical briefings Problem-solving Group work Teamwork skills (See samples of rubrics for assessing technical briefings and team collaboration.)

Product Assessment: 

Product Assessment A project whose focus is on the development of a tangible product. The product itself, the process, and quality of reasoning are all assessed. Use rubrics that address specific criteria relevant to the product, process and quality of reasoning and scales with 3 to 5 levels of mastery.

Reflection and Self-Report: 

Reflection and Self-Report Examples: Reflective portfolios Journals and lab books Process analysis (Angelo & Cross, p.307) Course-related self-confidence survey (Angelo & Cross, p. 275) Benefits to students: Develop a sense of themselves as learners Understand more deeply what they have learned and not learned See relationships among learning experiences Become more invested in their own learning

Feedback From Students: 

Feedback From Students “When we observe students while they are learning and collect frequent feedback from them, we can learn much about how they learn and, more specifically, about how they respond to particular teaching practices.” (Angelo & Cross, 1993) Examples: The Minute Paper (p. 148) Muddiest Point (p. 154) One-Sentence Summary (p. 183)

Reflection : 

Reflection Which of the assessment formats listed in the previous slides do you use in your courses? What have you found to be their advantages and disadvantages? Which assessment formats could you integrate into your courses?

Feedback To Students: 

Feedback To Students “Providing feedback on students’ work is one of the most expensive components of their education, but it is often not an effective investment simply because it happens too slowly.” (Gibbs, 1999) Guidelines: Provide frequent feedback throughout the term Be prompt in returning assignments and exams Make the feedback as specific as possible so that students know how to improve Provide progress reports one or two times during the term Correct errors, but avoid sarcasm and condescending comments

Guidelines for Grading: 

Guidelines for Grading Communicate the grading criteria to students when homework and exams are assigned so they know what is expected Apply the grading criteria consistently across students, across assignments, and across multiple graders Use rating scales to increase objectivity in grading, e.g., when evaluating presentations, journals, and written reports Clarify with students how the overall subject grade is determined Be sure that weighting systems reflect the proportion of time spent on each requirement

Review : 

Review Estimate your progress toward achieving today’s objectives: Recognize that assessment practices are based on assumptions and principles of teaching and learning Explain the keys to sound classroom assessment Select appropriate assessment methods matched to learning objectives Compare different assessment methods in terms of advantages, limitations, and appropriateness Use feedback to support student learning and improve instruction.

References: 

References Angelo, T. A., & Cross, P. K. (1993). Classroom assessment techniques: A handbook for college teachers, 2d ed. San Francisco, CA: Jossey-Bass. Field-tested learning assessment guide. Available at http://www.wcer.wisc.edu/nise/cl1/flag Gibbs, G. (1999). Using assessment strategically to change the way students learn. In Brown, S. & Glasner, A. (Ed.). Assessment matters in higher education. Buckingham, England: The Society for Research into Higher Education and Open University Press. Huba, M. E., & Freed, J. E. (2000). Learner-centered assessment on college campuses. Boston, MA: Allyn and Bacon. Stiggins, R. J. (1997). Student-centered classroom assessment, 2nd ed. Upper Saddle River, NJ: Merrill.

For Further Reading: 

For Further Reading Anderson, R. S., & Speck, B. W. (Eds.). (1998). Changing the way we grade student performance: Classroom assessment and the new learning paradigm. New Directions in Teaching and Learning, 74, Summer. San Francisco, CA: Jossey-Bass. Biggs, J. (1999). Teaching for quality learning at university. Buckingham, England: The Society for Research into Higher Education and Open University Press. Trevisan, M. S., Davis, D. C., Calkins, D. E., & Gentili, K. L. (1999). Designing sound scoring criteria for assessing student performance. Journal of Engineering Education, 88 (1), 79-85. Walvoord, B. E., & Anderson, v. J. (1998). Effective grading: A tool for learning and assessment. San Francisco, CA: Jossey-Bass.

Summary Reflection: 

Summary Reflection 3 important ideas I learned about assessment 2 concerns I still have about assessment 1 step that I will take related to assessment