Evaluating your course

Evaluating your course : 

Evaluating your course It’s not just the end of course questionnaire…

Why? : 

Why? "Evaluation is the collection of, analysis and interpretation of information about any aspect of a programme of education or training as part of a recognised process of judging its effectiveness, its efficiency and any other outcomes it may have." Thorpe, M.(1988:32) Evaluating open and distance learning, Harlow : Longman

Why? : 

Why? Not just for a pat-on-the-back : formative function Quality control of current course Improvement of future courses When? In-course End-course Post course Who? 360° evaluation involving all stakeholders : teachers, learners, parents, line manager, training officer lecturers, etc

When? : 

When? Summative and perhaps formative evaluation of teachers, teaching methods, or courses usually occurs at the end of courses when there is little opportunity for students to see or reap benefits from their comments. Also at this time, teachers cannot immediately implement recommended changes. Formative evaluations should occur earlier while there is time to change. Stockham, S. L. and Amann, J. F. Facilitated Student Feedback to Improve Teaching and Learning. Seen 22/11/10 http://web.archive.org/web/20011122184513/http://vega.lib.vt.edu/ejournals/JVME/V21-2/stockham.html

What? : 

What? Reaction (what the students feel about the training) Learning (the resulting change in knowledge, skills, attitudes) Behaviour (the extent to which the learning is implemented outside the classroom) Results (the effects resulting from the training) (Kirkpatrick, D. L. (1959) Evaluating Training Programs, 2nd ed., Berrett Koehler, San Francisco.)

Reaction : What? : 

Reaction : What? Did the trainees enjoy the training? Did they consider it relevant to their needs? Were they happy with the methodology used? Did they find the trainer courteous, competent etc. Were they happy with the venue, facilities,timing, etc of the course Etc etc

Reaction : How and When? : 

Reaction : How and When? Evaluation should be on-going – eg smileys at end of first session Feedback questionnaires Lesson redesigns See http://reviewing.co.uk/evaluation/methods1.htm for many other ideas Data relatively quick and easy to obtain On-going evaluation allows problems to be solved before they become critical

But … : 

But … Qualitative data : How valid are the results? Doesn’t tell us anything about the other levels of the model

Reactions of other stakeholders… : 

Reactions of other stakeholders… Do the teachers enjoy teaching the course? Do they find it meets Ss’ communicative and learning needs? Are they happy with the methodology? etc - questionnaires, diary reflection etc.

Learning : 

Learning Were the programme objectives achieved? Did the trainees acquire the knowledge, skills etc that it was intended would be taught? How much progress was made? - Formal testing – eg comparison of pre and post course tests

Behaviour : 

Behaviour Does the trainee apply the learning outside the classroom? When : Post course How : Interviews, questionnaires(?) observation Essential to validate any sort of ESP programme?

Results : 

Results What effect did the course have? Did the company get more contracts because of the participants’ improved ability to negotiate in English? Did the number of complaints fall because of the hotel staff’s improved ability to communicate effectively? Did the learners get better results on their postgrad courses because of the improvement in their abiltiy to read academic texts, understand lectures etc Did the students pass their FCE exam? Did the children get better marks in their English course at school?

Slide 13: 

This presentation is part of an on-line course run by Business Talk Milan If you would like to find out more about our on-line and face-to-face training courses, please contact us via our website : www.business-talk.it