Washback Effect of OPI

Some Personal Observations on the Washback Effect of OPI : 

Some Personal Observations on the Washback Effect of OPI Lan Yu Asian School One CG 3 FDD 2009

Slide 2: 

I. THEORETICAL ISSUES

Slide 3: 

What is Washback? Alderson and Wall (1993) “Does Washback Exist?”: The way that tests are…perceived to influence classroom practices, and syllabus and curriculum planning. (p117)

Slide 4: 

Theoretical models Alderson and Wall (1993) Does Washback Exist?: test influences teaching, learning, what to teach, how to teach, what to learn, how to learn, rate and sequence of teaching and learning, degree and depth of teaching and learning, attitudes to content, method of T/L, etc. Hughes (1994): washback on three constituents: the participants, the processes and the products of the educational system (Tsagari 2007)

Slide 5: 

Participants--- students, teachers, materials, writers and curriculum designers, researchers Process--- the way teaching is executed Products--- learning, teaching, new materials and new curricula, research results

Slide 6: 

Empirical Studies, sample of Liying Chen (1995) How Does Washback Influence Teaching? Implications for Hong Kong the washback effect of the revised Hong Kong Certificate of Education Examination in English in Hong Kong secondary schools. Findings indicate that the washback effect worked quickly and efficiently to bring about changes in teaching materials, but somewhat slowly, reluctantly, and with difficulty in the methodology that teachers employ. It is suggested that teaching content has so far received the most intensive washback effects, although washback effects have also been observed in teachers' attitudes and behaviors and in the English curriculum.

Slide 7: 

Wall, Dianne and J.C. Alderson (1992): Examining Washback: The Sri Lankan Impact Study The authors examined the washback effect of an English test on the way teachers use textbooks and teach students. Their conclusion is that there is no obvious washback effect when teachers teach textbooks and there is obvious washback effect when teachers do not use textbooks but use their own materials to prepare students for the test. As to the former case, they states that there is no washback effect because teachers are not clear about the goals of the test nor the goals of the textbook.

Slide 8: 

Why study washback? Washback leads to change and the change can be good or bad. Studying washback is to decide if it is positive or negative for learning.

Slide 9: 

POSITIVE OR NEGATIVE WASHBACK, HOW TO DECIDE? Liying Cheng (2006): It is not conclusive if the washback is positive or negative Negative washback: when tests does not match the content of the program, the tests have a negative impact in discouraging the implementation of program (Chen 2006) This makes sense only when the tests are modifiable To suite the contents of the program. What about external tests that are not modifiable?

Slide 10: 

Washback? modification --Positive modification --Negative modification Criteria: test goals. When tests are not modifiable, program and teaching method should be modified to meet test goals.

Slide 11: 

Test grades vs test goals Test goal is always good intentional. Test designers sincerely believe their tests promote genuine learning. Good grades can be achieved without learner really meeting the goals. High grades but low ability

Slide 12: 

TEST GOALS OF OPI a standardized method of measuring actual proficiency in language skills required to function in given life/job situations, as well as a testing tool with low risk of compromise. http://www.dlielc.org/testing/opi_test.html So, if OPI should have positive washback effects on the teaching of speaking, it should promote genuine development of communicative competence.

Slide 13: 

My idea: Positive washback: modifications help achieving the genuine goals of the external tests Negative washback: modifications intended primarily at achieving high grades, sometimes sacrificing test Goals=Teaching to test

Slide 14: 

Methodology of Studying Washback Tsagari (2007) listed 29 empirical researches and their methods The major research methods include: questionnaire, field observations, interviews and analysis of documents. Other researches also use test scores and test analysis

Slide 15: 

Method of Current Research A. Personal experience---subjective B. Questionnaire Preferred methods of training speaking The most/least effective way of teaching speaking How OPI knowledge influences ways to teach speaking How desirable OPI knowledge is for non-testers How would one teach speaking if there had been no OPI? The good way to improve discourse quality of speaking How current ways of speaking training contribute to learners’ appropriate communication with native speakers Teachers’ emotional reaction to speaking training C Document analysis

Slide 16: 

The Washback Effects of OPI on Teaching The method of teaching speaking The content of speaking practice Organizational and material development The attitude towards speaking methods

Slide 17: 

Methods of Speaking Practice Currently Used 1. in class pair or group speaking activities prescribed in the textbooks 2. split session speaking hours arranged by the team, catering to specific lessons. ---a. retelling of texts and open topic conversation ( PEP Speaking) ---b. role play situations 3. split session speaking hours practicing unit speaking test tasks (team-made)

Slide 18: 

4. Total immersion policy ---a. In class Chinese-only policy ---b. TI day at Fort Ord 5. 1-on-2, 1-on-1 speaking practices 6. personalized speaking outlineJia Jie.doc 7. Speaking club 8. DA on speaking 9 Mock OPI 10. Tester support of graduating class 11. Presentations in Chinese on Chinese culture/history/society etc 12. Writing samples of OPI tasks Method: 1 2 3 4 5 scale, x number of selection

Slide 19: 

Preferred Teaching Method as Shown in Questionnaire

Slide 20: 

Preferred way to improve the sentence and discourse quality of student speech 1 to 5 scale more solid grammar learning the revision and critique of student writings c. groups discussion, pair speaking, role play situations d. 1-on-1 correction and ad hoc tutoring e. reciting presentations and nice sample paragraphs Abced Bcdea ebacd Ecabd Ecabd d abecd Ebdac Beacd F—modeling—raise consciousness in the context, corrections/guidance that can melt to the content

Slide 21: 

Teacher Perception and Attitude How does OPI knowledge influence ways to teach speaking? 3=30%; 4=40%; 5=20% 35/50=70% In which way? Format, topic, levels, organization of the test 2. How desirable is OPI knowledge? 4=10%; 5=40% 3. To what extent do teachers enjoy current ways of teaching speaking? 1st semester: 32 2nd semester: 23 3rd semester: 18 (OPI specific practice)

Slide 22: 

Perception of communicative competence 1. How current ways of speaking training contributes to learners’ appropriate communication with NS? 1=10; 3=50%;4=40% (OPI=authentic communication) 2. How would one teach speaking if there had been no OPI? more speaking; real communication; more naturally and authentically*; 1-on-1;more debate and presentations; more emphasis on speaking *the respondent confirmed that he regarded current way of teaching speaking is not real communication

Slide 23: 

Ways to improve speaking Effective: talk to NS; 1-on-1; 1-on-1 early start; personalized conversation; TL from day 1; role play w/correction and summarization Ineffective: recite written articles; writing sample after SII a waste of time; memorize OPI tasks; recite presentations; OPI unpredictable

Slide 24: 

OPI Materials Outside Regular Textbooks 1. Role Play Situations -- role play scenarios, vocab, used as speaking activity checklist 2. CBC Oral Training Course Fang and Zhu 2005 -- OPI task sample, vocab, used as speaking session reference, no in-depth study with exercises 3. Level II Speaking Tasks 2004 Lin 2004 -- OPI task samples with audio files, vocab, written assignment, used in department B consistently under Lin and in other departments inconsistently 4. Sporadic task samples written by instructors, used frequently 5. Speaking Outline, quite popular

Slide 25: 

Workshops I came to DLI in 1999. OPI was already institutionalized at that time but I do not remember intensive effort at preparing students for OPI other than pre-test briefing and speaking practice. Later, more emphasis was given to OPI achievement. 1-on-1 speaking appeared earlier. Chinese OPI Familiarization Workshop Dec. 29-30, 2005. with live OPI demo and analysis.

Slide 26: 

Discussion: Questionnaire indicates strong belief in: 1. Speaking practices directly related to OPI, especially 1-on-1. Emphasis is on the ability to produce speech and the ability to accomplish OPI tasks. 2. Spoken mode as opposed to written mode as means to improve discourse quality 3. OPI-oriented speaking = authentic communication (private talks with some testers however, negate this) And lesser belief in 3. Grammar learning in non-speaking context Still lesser belief in writing as a way to develop discourse competence . And…….?

Slide 27: 

Content of Speaking Practice Content of speaking is predominately determined by the tasks of OPI and the required graduation level (previously 1+ and now 2) Effort of pushing students to more advanced level of speaking is negligible. In questionnaire on the actual use of speaking method, the presentations in Chinese on advanced topic has a value of only 23, the lowest. Term III speaking not matching reading and listening as far as IRL level is concerned. S=2, R L=2+ or 3

Slide 28: 

The Use of Personnel and Materials 1. Textbook interactive activities is not regarded as optimal way of practicing speaking, mainly because they are not so OPI oriented and they do not produce obvious and immediate effect. 2. The two OPI sample textbooks are used only as a reference, without solid learning of vocab, cohesive devices and communication strategies. They are not used by all the departments. Not integrated into program. 3. Personal speaking outline is used extensively because they predict topics of OPI. 4. Tester support of graduating class is not uncommon.

Implications: To what extent is OPI a proficiency test? : 

Implications: To what extent is OPI a proficiency test? OPI can still be used authentically for interviewing job applicants who have no idea about OPI and have never been trained to deal with it. In OPI training context, however, learners already know about OPI procedures and have been trained to deal with them. Case of student COE: as long as they use SVO structure, I can understand and I can answer their qs. Most qs expected and well prepared. Student feedback: it is a game (too much but some truth) Similar case: TOEFL and GRE in China

Slide 30: 

Two stories A class was taking its graduation tests, beginning with OPI. All the teachers and students were excited, waiting to see the results of their more than 60 weeks toil. Students coming back from OPI either were happy or not so happy. When one happy student appeared in the classroom building, a teacher-tester asked him:” how many probes did you get?” “Two!” the student said. “ Good, good”, the teacher said, “No problem, then!” At speaking club a student came in. After I learned he had much travel experience, I began to get him into a description task without stating it explicitly. The student knowingly smiled and said: “ You teachers are really good at getting us to practice OPI. I have had it the whole day and I just want to talk something else randomly (suibian tantan). Thought provoking, isn’t it? What does the stories tell us?

Two Problems with OPI : 

Two Problems with OPI It does not measure what it claims to measure. It’s goal is questionable. It is not a real conversation as it is claimed to be, but a structured interview that lacks many features of real conversational communication. (Marysia Johnson (2001) :The Art of Nonconversation) Back to the stories, they indicate that OPI has a predictable procedure that is being used by teachers to prepare students, Well trained students are highly aware of test procedures and tester expectations of them. As a result, the test is more like semi-achievement test than proficiency test.

Example of Fake Proficiency : 

Example of Fake Proficiency One description fills all My Mother My Sister My Wife

Slide 33: 

Problems with OPI-oriented Speaking Practice Low efficiency of teaching. Many basic grammar issues are taught in 1-on-1 sessions because they are not well learned in the classroom.It is not uncommon to see students parroting. Artificiality in speaking practice. Speaking becomes a topic-by-topic rehearsal. Speaking does not help advanced listening and reading because of the limited range of OPI tasks and the grammar patterns used.

Slide 34: 

Speaking clichés not authentic at all Hobbies=? instruction: how to prepare (I wonder if it is the same case with other languages)? Past narration: the most unforgettable thing/people Real life communication failure: Students coming to teacher for speaking practice do not know what to talk about and give the teacher sole power to lead the conversation which turns out to be an interview or even interrogation.

The positive washback of OPI : 

The positive washback of OPI OPI leads to speaking practice in the program, be it artificial. Teacher-student speaking sessions are useful in training students to speak. (http://www.nclrc.org/essentials/speaking/spindex.htm)(Also see Yoffe 1997 “An overview of the ACTFL profiency interview: a test of speaking ability?”) A lot of these factors are not teachable unless in a actually speaking situation, better with a native speaker. Student feedback: I gained a lot from speaking practice. Now I am more natural in using connectives etc. Development of materials that might be the precursor of a proficiency-oriented speaking course that might push students to higher levels of speaking.

How to prepare speaking under current framework? : 

How to prepare speaking under current framework? Lay the foundation well. Basic grammar should not be taught in 1-on-1. Add unpredictability. Add advanced writing as a component of the program

Foundation : 

Foundation the foundations of producing acceptable sentences lies in a solid mastery of vocabulary and grammar, which can be taught collectively. Ignoring basic training in this aspect is unwise. In intensive programs, vocab memorization and grammar drills are important because they reduce the time for internalizing the building blocks of speech.

Unpredictablity Real life communication involves unpredictability and negotiations.Example: description of photos never seen before by learners is a better way to test their ability in using descriptive language. : 

Unpredictablity Real life communication involves unpredictability and negotiations.Example: description of photos never seen before by learners is a better way to test their ability in using descriptive language.

Slide 39: 

Reading maketh a full man; conference a ready man; and writing an exact man. And therefore, if a man write little, he had need have a great memory; if he confer little, he had need have a present wit: and if he read little, he had need have much cunning, to seem to know that he doth not. Francis Bacon The Use of Writing

How writing benefits speaking : 

How writing benefits speaking Exactness of wording and sentence formation Organization of thought Level promotion: advanced speaking discourse bears many features of written discourse. Note: Writing is not for producing OPI samples to memorize, even in a natural way.

How to prepare for OPI : 

How to prepare for OPI Prepare for OPI as if preparing for SOPI The SOPI is a tape-recorded test consisting of six parts. It begins with simple personal background questions. Part one is the "warm-up". Parts two, three, and four employ pictures in a test booklet to check for the examinee's ability to perform the various functions that characterize the Intermediate and Advanced levels of the ACTFL proficiency guidelines, or levels one and two of the ILR skill level descriptions. Parts five and six of the SOPI require the examinee to tailor his or her discourse strategies to selected topics and real-life situations. These parts assess the examinee's ability to handle the functions and content that characterize the Advanced and Superior levels of the ACTFL guidelines, or levels two through four of the ILR skill level descriptions. The SOPI ends with a "wind-down.“ Pictures for description and narration, maps for direction,

Slide 42: 

Stansfield, Charles W. An Evaluation of Simulated Oral Proficiency Interviews as Measures of Spoken Language Proficiency. 1990-00-00 Abstract:A review of research on different SOPI tests in different contexts reveals a high correlation with the OPI and some practical and psychometric advantages over it. In SOPI, testees cannot lead the tester (a machine) to well practiced topics by highlighting certain parts of his self-introduction.

Slide 43: 

Finally: Speaking training should go beyond OPI ?? 2008:“????????????????:For cross-cultural communication students should develop not only linguistic competence, but also social competence, discourse competence, strategy competence, sensitivity, tolerance and flexibility. Current OPI-oriented speaking class is insufficient in developing cross-cultural communicative competence. A teacher in answering to my questionnaire wrote something thought-provoking:

Slide 44: 

The irony here is we need to think outside the box before we can improve our teaching. The testing, curriculum, class arrangements, etc. have framed us with this huge box and bobwires that teachers would not even touch. Only when we think about what learning and teaching a language means to us and the students in the long term can we raise the bar for language learning/teaching. ========================= Questions?