EkinsALT2007presenta tion

The Use of Interactive online Quizzes in Maths: 

The Use of Interactive online Quizzes in Maths Dr Judy Ekins The Open University UK j.m.ekins@open.ac.uk

The context: 

The context Level 1 mathematics (two 30 CATs points) MU120 Open Mathematics MST121 Using Mathematics 3000 to 4000 students per year on each 60% completion rate Assessed by TMAs and CMAs, each covering several weeks’ work Feedback several weeks after study of earlier topics

Usefulness of Assessment Gibbs and Simpson summary gives 11 criteria: 

Usefulness of Assessment Gibbs and Simpson summary gives 11 criteria MU120 & MST121 satisfy most criteria, but not: Sufficient feedback is provided, both often enough and in enough detail The feedback is timely in that it is received by students while it still matters to them and in time for them to pay attention to further learning or receive further assistance Pilot quizzes are designed to rectify this deficiency.

Short Internet Quizzes: 

Short Internet Quizzes give speedy feedback feedback when students would pay attention to it and use it in their learning formative assessment most appropriate for pilots on MU120 and MST121 Open Mark has appropriate question types, with tailored feedback after each of several attempts

Some Question types: 

Some Question types Enter numerical / algebraic answers Multiple choice Plotting of points and lines on graphs Matching pairs Dragging and dropping words or symbols into appropriate places in mathematical expressions or text (Show Examples)

Examples of question feedback: 

Examples of question feedback pointing out a standard error when made answer is too large or too small hint as to how to proceed indicating correct / incorrect parts of multi-part answer The feedback after the each attempt often gives progressively more detailed hints on how to proceed

End of quiz feedback: 

End of quiz feedback Summary of performance on all questions Total marks Study advice appropriate to mark References to appropriate sections of teaching materials Students can: quickly check on any areas which need more attention repeat quiz questions to get more practice

MU120 quizzes: 

MU120 quizzes based upon eight preparatory topics studied over several weeks / months Test understanding after each topic About 6 questions per quiz of different types 3 attempts each question, but can re-try question type many times–(different each time). early registering students kept involved Late registering students might attempt quizzes just once

Hints and Feedback: 

Hints and Feedback For the pie chart question, The hint is: “There are 20 shaded segments and so 20 students voted out of 24.” If the student gives the answer 20/24 they get “You have the correct number but the fraction can be simplified”. Other answers receive feedback similar to hint. Where possible all feedback is given to the right of the question, so as it is all on one screen,

Feedback on a correct answer : 

Feedback on a correct answer

Feedback on a correct answer : 

Feedback on a correct answer

Feedback includes further work: 

Feedback includes further work

Feedback includes hint: 

Feedback includes hint

Co-ordinate Question and feedback: 

Co-ordinate Question and feedback

End of quiz feedback: 

End of quiz feedback

MU120 User Feedback: 

MU120 User Feedback feedback question at the end of each pilot quiz video of student users “thinking aloud” Enjoyed quizzes, which aided their learning Took about 5 to 20 minutes per quiz Short quizzes better than a long CMA Liked the immediate targeted feedback Critical if feedback less specific to their answer

Administrators’ reports: 

Administrators’ reports Individual students can be tracked on the administrative reports, ( time on-line, all their responses and scores). The reports also highlight problems E.g summary of the question scores, highlights a low mean for a question. Inspection of responses and the “thinking aloud” video, led to improvements in question feedback.

Administrators’ report: 

Administrators’ report

Think aloud video: 

Think aloud video Quizzes stimulate much learning. Students look up relevant references, work on paper and use their calculators. Usually read the feedback carefully even if correct. Said quizzes stimulated their learning more than doing exercises “from a book” (answers are in the back) If stuck or went wrong, generally got useful timely feedback, relevant to their actual answers. Critical if feedback not relevant or helpful Enjoyed the interactive quizzes Useful in checking understanding and stimulating learning.

Conclusion: 

Conclusion Successful “Open Mark” interactive internet assessment piloted with level 1 students Students liked more immediate detailed tailored feedback, useful within their learning. Users find the quizzes fun Initial authoring more time-consuming, but less work subsequently Authors like administrator reports Still more to do analysing feedback & administrator reports to improve quizzes / teaching materials. Hope increased motivation and improved feedback will lead to better student retention Much interest in the Faculty to improve assessment and teaching