Slide1 : in Kamloops, BC
Slide2 : Why Mega-lectures In the winter of 2003, because of: government concerns for maximizing enrolments institutional pressures for efficiency Departments at TRU were asked to develop strategies both to increase FTE's without hiring additional faculty and to reduce the number of students on wait lists.
Slide3 : Strategy I proposed a strategy of increasing my normal 60 student PHIL 111 — Introduction to Critical Thinking class to 200 students, by using the course management software package WebCT and a teaching assistant. This extra large class increased my semester student load from the normal 120 to 260 and my annual load from the normal 240 to 520.
Slide4 : Material Covered in Presentation I will discuss and demonstrate: how to link to external information from a WebCT quiz. quiz security issues in an Internet environment. the student tracking ability that WebCT provides. Finally, I present the results of a survey evaluating student satisfaction with WebCT in this course. the different types of WebCT quizzes used. managing a WebCT quiz in a computer lab.
Slide5 : PHIL111 — Course Content The primary concepts covered in my PHIL111 course are: argument definition, arrow diagrams, validity and soundness, truth-tables, deductive and inductive inference, valid and invalid argument forms, and categorical logic with syllogisms.
Slide6 : PHIL111 — Structure of Course Delivered in a multi-media lecture theatre. Content is available online before the lecture. Lectures have associated WebCT self-tests. Supervised weekly WebCT quizzes are administered to test students’ comprehension of the previous week’s lecture. Weekly WebCT assignments are given to encourage reading of the lecture material. LECTURES: SELF-TESTS: SEMINARS: ASSIGNMENTS: MIDTERM and FINAL: WebCT exams evaluate total comprehension of the material.
Slide7 : Evaluation Essentially, there are four components to evaluation: Seminar Quizzes — 25% Assignments — 15% Midterm Exam — 25% Final Exam — 35% Each component of evaluation is a WebCT Quiz.
Slide8 : Teaching Assistant The TA was a fourth year student, with a strong background in philosophy, especially logic, and who also had experience with WebCT. The TA's duties and responsibilities primarily focused around the evaluation of the student through monitoring the seminar quizzes, marking assignments and assisting with exams. Evaluation was administered using the WebCT Quiz Tool. Most of the WebCT quizzes were automatically marked by the computer; however, there were a number of paragraph or short answer type WebCT questions that had to be manually marked.
Slide9 : Midterm Exam Example
Slide10 : Midterm Exam Example
Slide11 : Midterm Exam Example
Slide12 : Midterm Exam Example
Slide13 : Final Exam Example
Slide14 : Final Exam Example
Slide15 : Final Exam Example
Slide16 : Lectures The course could easily be completed with only the WebCT lecture notes, though there is a recommended text. The lectures were delivered in a 200-seat lecture theatre equipped with a multi-media work station that included two ceiling-mounted projectors. Each lecture in WebCT had links to the overhead presentations for that lecture, as well as links to self-tests, to a glossary of terms, to a search function and various external links such as a philosophy dictionary. Probably 75 to 100 students attended lectures regularly. The remaining students obtained their material from the online notes.
Slide17 : Seminars — Quizzes For the remaining seminars there were 9 monitored quizzes and 1 monitored midterm exam given in the seminar hour, all completed in WebCT. The first two seminars of the term were used to familiarize the students to the WebCT environment. There were 10 seminars throughout the week and if a student could not make his/her registered seminar, he/she was allowed to attend any other seminar, as long as the lab had spare computers. This significantly reduced make-ups for missed quizzes. There were 9 quizzes given over the term; the student received an average out of 25 of their best 8 quizzes.
Slide18 : Seminars — Quizzes Quizzes were closed book. The difficult questions from the previous week’s quiz were setup in a practise set and then discussed at the beginning of the following week’s seminars. This problem set was available to the end of term. Quizzes varied in duration from 30 minutes to 45 minutes. No browser window or e-mail client that was not part of the quiz could be active in the background. This was monitored using a program called NetSupport. (If caught in violation, first time penalty was failure on the quiz, second time penalty was failure in the course.)
Slide23 : Assignments — 15% of Final Grade There were 9 assignments, delivered as WebCT quizzes. Assignments could be completed anywhere that the student could access the Internet. Assignments were due on Sunday nights at midnight. All assignments were released well in advance of the due dates and answers were released in most cases immediately after Sunday midnight.
Slide24 : Assignments — 15% of Final Grade For assignments that the computer could mark automatically, the student was allowed 3 attempts, and received the average of the 3 attempts. For these three-attempt assignments, a mark was released immediately upon completion, but the student was not told the correct or incorrect answers. If a student wanted to come to my office to review his/her submission before another attempt, I would review with him/her the questions that he/she did not understand.
Slide25 : Exams — Midterm and Final Both the midterm and the final exams were WebCT quizzes. Both the midterm and the final exams were composed of two parts; a multiple choice part and a written analysis part. The midterm was administered in the seminar hour. The final exam was set in an exam period at the end of term. As UCC has no contiguous labs with 200 computer workstations, we broke the group for the final exam into two sittings.
Slide26 : Quiz and Exam Statistics Every WebCT Quiz provides a wealth of detailed statistics and information at both the class and individual level:
Slide27 : Quiz and Exam Statistics
Slide28 : Quiz and Exam Statistics
Slide29 : Quiz and Exam Statistics
Slide30 : Quiz and Exam Statistics
Slide31 : WebCT Quiz Security Issues When administering an online WebCT quiz, whether totally online or in an invigilated lab, you definitely must familiarize yourself with the various quiz settings. Doing so could make the difference between having a fair evaluation of student achievement versus an utter disaster resulting in appeals for re-evaluation.
Slide32 : WebCT Quiz Security Issues Controlled release to specific accounts. One can release quizzes to the whole class or to only a subsection of the class, such as one person. Controlled release to specific IP address. Even though you can control release to one account, more than one person can sign into an account. So, a student could sign into a quiz, and their bright friend in Timbuktu sign in at exactly the same time and complete the quiz for them, while the student sits in front of the workstation appearing to complete the quiz. You can release quizzes to a single IP address or to a range of IP address.
Slide33 : WebCT Quiz Security Issues Controlled release by password. A special password can be set for each quiz, and the password is released only at the time the quiz begins. Labs should be setup to monitor the students’ workstations. To prevent students from accessing a practise set of questions, or even communicating by e-mail during a quiz, you require additional software to monitor such events. Two methods of prevention are available
secure browser
a program that monitors the student workstation, e.g. NetSupport, Silicon Chalk.
Slide35 : Tracking Most things that a student does in WebCT can be tracked a record shows when a student first signed in a record shows when they last signed in a record shows when they went to the course/lecture notes a record shows if they accessed the bulletins a record shows if they posted to the bulletins Placing tools and utilities in content modules enables tracking
Slide36 : Tracking — Students
Slide37 : Tracking — Students
Slide38 : Tracking — Pages
Slide39 : Tracking
Slide40 : Go to Course PHIL 111 — Critical Thinking
Slide41 : Student Survey of Satisfaction with WebCT I created and administered a student survey through WebCT. The student survey was designed to evaluate the student's satisfaction with WebCT in the course. Of 321 who completed the course (out of approximately 400 who started), 158 students answered the survey (almost 50%).
Slide42 : Student Survey of Satisfaction with WebCT I also administered a similar survey at my institution to all students using WebCT in the winter term 2004 (approximately 6800) and of these 486 completed the survey. Two questions on the survey enabled the student to write detailed comments on what they liked and disliked most about WebCT.
Slide43 : Student Survey of Satisfaction with WebCT Please tell us what aspect(s) of WebCT you LIKED the most. (e.g., Getting the lecture notes at any time, doing quizzes, ease of use, interacting with others, etc.) Please tell us what aspect(s) of WebCT you DISLIKED the most. (e.g., Problems logging on, difficulty navigating around the site, amount of information was overwhelming, etc.)
Slide44 : Student Survey of Satisfaction with WebCT The following summaries are from the survey administered to all students using WebCT at my university.
Slide45 : Aspects of WebCT Students Liked Most – written answer The feature that the majority of students mentioned most frequently (19.5%) was the ability to access course notes, lecture notes, handouts, etc., whenever they wanted to do so. Next in frequency mentioned (16.8%) was access to practise quizzes / self-tests / practise exams. This was followed by easy access to the results of assignments, quizzes and exams, and the answer keys (13%).
Slide46 : Viewing marks with statistical comparison to class, especially immediately upon completion of quiz and/or assignment (12.7%). Easy access to information from anywhere at any time especially home, e.g., information about cancelled classes, etc. (8%) WebCT is simple to use. (6.8%) Quick communication with instructor (e-mail, bulletin board). (6.5%) Aspects of WebCT Students Liked Most – written answer
Slide47 : Easy to review course AND see progress. Easy to get clarification about course material, especially when missing a class. WebCT should be used by all faculty. Because notes online, one could listen to the instructor during lecture and not spend time trying to write everything down. Aspects of WebCT Students Liked Most – written answer
Slide48 : Aspects of WebCT Students Disliked Most – written answer No problems at all with WebCT. (34.7%) Either the server or the network crashing. (10%) Some complained about WebCT quizzes. There were a variety of problems noted, everything from the student’s poor typing skills, to fact that the multiple choice/short answer type of question was too inflexible both because of format and because of possible answer (wouldn’t allow for spelling mistakes).
Slide49 : There were a few students who felt that WebCT made them into a 'number' and distanced them from both their course, their classmates and their instructor. Some functions could have been better developed, e.g., Index. Aspects of WebCT Students Disliked Most – written answer
Slide50 : Navigation for some functions caused concern, because it was awkward and not intuitive, with the browser 'BACK' button topping the list of problematic navigation links. The large size of some PowerPoint and Adobe files (any large files in general) caused problems when downloading and printing, especially from home. There were the expected logging on problems. Aspects of WebCT Students Disliked Most – written answer
Slide51 : Multiple Choice — Instruction on use of WebCT
Slide52 : Multiple Choice — Attitude to being forced to use WebCT
Slide53 : Multiple Choice — Time required for WebCT
Slide54 : Multiple Choice — The Quiz Tool was effective.
Slide55 : Multiple Choice — The Self-test Tool was effective.
Slide56 : Multiple Choice — The MyGrades Tool was effective.
Slide57 : Multiple Choice — Took action because of WebCT.
Slide58 : Multiple Choice — Studied more because of WebCT.
Slide59 : Multiple Choice — More involved in course because of WebCT.
Slide60 : Multiple Choice — Improved communication with instructor.
Slide61 : Multiple Choice — Prefer class with WebCT to one without.
Slide62 : Multiple Choice — Overall experience with WebCT.
Slide63 : Multiple Choice — Overall Impression of WebCT
Slide64 : Student Results — Fall 2003 The September class started with slightly over 200 students (approximately 196 by stable enrolment date) and 163 completed the course to final exam.
Slide65 : Student Results — Winter 2004 The January class started with slightly over 200 students (approximately 196 by stable enrolment date) and 158 completed the course to final exam.
Slide66 : The End Dan O’Reilly
Assistant Professor of Philosophy
Department of Philosophy, History and Politics
Thompson Rivers University
E-mail: oreilly@tru.ca
WWW: oreilly.cjb.net