2006 Univ TA Lab assessment

Slide1: 

There are Handouts by the door, please make sure you grab them

Designing Effective Lab Assessments:Begin at the End: 

Designing Effective Lab Assessments: Begin at the End Briana Timmerman Director, Biology Undergraduate Program 2006 University TA Training

What’s special about assessment in Labs?: 

What’s special about assessment in Labs? Labs are about DOING SCIENCE So Assessments should focus on students’ ability to perform science - not review for lecture or regurgitation of facts Can use “performance assessments” - students complete a relevant and realistic task similar to those used by practicing scientists

Spectrum of interaction of assessment and learning: 

Spectrum of interaction of assessment and learning Assessment plays no role in learning: students have no idea if they are learning the material until it’s too late (exam grade already earned). Grade reflects ability to answer exam questions. Assessment is central to learning - students are provided with as much feedback as they desire prior to the earning the grade. Grade reflects mastery of skills and learning goals. Which type of assessment is used in the “real world?” How will your learning in graduate school be assessed?

NRC on Effective Assessment: 

NRC on Effective Assessment Greatest difference between experts and novices is not how much information they know, but how that information is organized; experts organize knowledge into schemas that support pattern recognition and application of knowledge. Assessments should focus on making students’ thinking visible so that misconceptions can be identified and altered during the learning process and so that the students’ abilities to manipulate and apply the knowledge is the basis of the grade. from National Research Council “Knowing what students know: the science and design of educational assessment”

Two types of assessment: 

Two types of assessment Summative: How you are going to assess what the student has learned and assign a grade. Examples: lab reports, lab practicals, exams, quizzes Formative: frequent, timely, informative feedback necessary for effective learning. How the student knows whether or not they are learning the material prior to earning a grade. Examples: outcome-oriented tasks (produce a result) “If you understood “A” you should be able to complete “B”), low-penalty quizzes, peer questioning, today’s focus

Introductions and Examples: 

Introductions and Examples

Beginning at the End: 

Beginning at the End For assessments to be effective, they must be planned prior to teaching Assessment must be integrated into the lesson are students provided with formative feedback so they can identify their areas of weakness before the summative assessment? does the lesson support the goals? does the summative assessment measure achievement of the goals?

Effective assessment requires knowing what you value : 

Effective assessment requires knowing what you value Meaningful learning goals What new thing do you want your students to be able to do by the end of the lab/project What skills/abilities are valued in your field? What lab-based abilities do scientists you admire possess? Those valued abilities should be the learning goals for labs

Slide10: 

Do these assessments match? Should they?

One example: USC Biology: 

One example: USC Biology Abilities Valued: to be a practicing scientist whose work is externally funded and well-published to design and perform innovative, elegant and rigorous experiments to communicate the results of those experiments in persuasive manuscripts which are published in well respected journals.

USC Biology Curriculum Goals: 

USC Biology Curriculum Goals Be able to “think like a scientist” Identify assumptions Create and evaluate hypotheses Create abstract models of data Design experiments relevant to questions Analyze qualitative and quantitative data Assess validity of work, identify gaps in knowledge Evaluate the results of analyses and experiments and decide on next step Identify unintended results as opportunities for discovery Integrate new concepts with current knowledge

Slide13: 

What “data source” contains information about all these abilities in students?

Universal Rubric for Lab Reports: 

Universal Rubric for Lab Reports lab reports ~ manuscripts 15 criteria which explicitly define how a student would demonstrate achievement of the curriculum goal Rubric - more than a list of criteria, explicitly describes student achievement at each level

Examples of student responses Hypotheses: Testable (p 2.): 

Examples of student responses Hypotheses: Testable (p 2.) “The hypothesis to explain the crisis event of 1977 that caused many finches to die and some to survive is a series of events.” What level of performance is this?

2nd example of student response Hypotheses: Testable (p 2.): 

2nd example of student response Hypotheses: Testable (p 2.) “The purpose of the experiment is to determine if sex-linked traits produce different ratios of phenotypes than non-sex-linked traits. We hypothesize that the sex-linked trait will produce a 50/50 mix of red and white-eyed offspring. The null hypothesis is that there will be no difference between the ratio of the sex linked cross and the autosomal mono-hybrid cross.” What level of performance is this?

3rd example Hypotheses: Testable: 

3rd example Hypotheses: Testable “We observe that dandelions in regularly mowed fields are shorter than those in un-mowed fields and determined that the height difference has a genetic basis. We hypothesize that this evolutionary shift is due to the selective pressure generated by mowing. Other explanations include genetic drift or mutation. If genetic drift is responsible for the height differences, than populations released from mowing will remain stunted on average due to reduced genetic diversity. If a mutation is responsible, than mowed populations will all contain unique genetic markers not found in un-mowed populations. If natural selection is the cause than populations released from mowing should show a diversity of heights and un-mowed populations should respond to early season mowing with a decrease in average height at time of blooming.”

How to use the Universal Rubric : 

How to use the Universal Rubric Identify learning goals and corresponding criteria from rubric. Write the assignment making the criteria and their rationales explicit (assign point values to performance levels for each criterion). Design the lesson so that students practice the skills necessary to achieve each learning goal. Advantage over old way of grading: Performance levels already defined range of variation already defined mental map is now on paper, reduces your cognitive load

Your Assessment: 

Your Assessment Now we’re going to generate some ideas for you to use in your class Starting with the “big picture” What goals and abilities are valued in your field?

What abilities are valued in your field?: 

What abilities are valued in your field? analyze data, see patterns and predict observe, propose hypotheses, design experiments writing and communication teamwork, collaboration Do these values correlate with the types of assessments typically used in classrooms?

Effective assessment: 

Effective assessment E“valu”ate = your means of evaluating the students should be consistent with the learning goals you value. How are these proficiencies measured for practicing scientists? Generate examples - “think, pair, share”

How are these abilities measured in practicing scientists?: 

How are these abilities measured in practicing scientists?

From Values to Goals: 

From Values to Goals What are your learning goals for your students? Write them down Learning goals should focus on what you want your students to be able to do by the end of the lab/project Pick verbs that are measurable: draw, diagram, explain, compare, contrast, etc. “understand” and “know” are not measurable

From goals to assessment: 

From goals to assessment How is this ability measured in a “real scientists?” What is the classroom analog to that real life task? What relevant components can you measure?

Measurement to Grade: 

Measurement to Grade Once you have identified the points of measurement, Delineate the different levels of performance that will achieve each grade and write them down (make a rubric) Or use the universal rubric and just assign point values and relative weights to the performance levels already indicated.

Recap: Beginning at the End: 

Recap: Beginning at the End For assessments to be effective, they must be planned prior to teaching and integrated within the unit Values = Learning Goals= Assessments must align, grades should reflect meaningful assessments of scientifically relevant abilities Lesson must support learning - students must have sufficient practice and formative feedback before being tested on desired abilities

Don’t Forget Formative Assessment: 

Don’t Forget Formative Assessment Formative assessment also = feedback which is critical for learning Provide frequent “low penalty or penalty-free” feedback, so students can know if they are on the right track or not How do you know when your research is going well - what types of feedback do you get as a practicing scientist? peer feedback expected outcomes if controls are functioning

In the future…: 

In the future… National Research Council “Knowing what students know: the science and design of educational assessment” If you use the rubric, I would be delighted to receive your feedback on it’s usefulness or just know in what classes you are trying it.