Assessing speaking

Views:
 
Category: Education
     
 

Presentation Description

A look at different ways of assessing speaking, and a summary of the key considerations taken from Brown (2003) Language Assessment Principles.

Comments

By: goodaymate (53 month(s) ago)

thanx alot.

Presentation Transcript

Language Assessment : 

Language Assessment Assessing Speaking

5 Basic types of speaking : 

5 Basic types of speaking

IMITATIVE : 

IMITATIVE It is simply the ability to parrot back a word or phrase or a sentence.

INTENSIVE : 

INTENSIVE It is the production of short stretches of oral language. Examples include directed response tasks, reading aloud, sentence and dialogue completion, limited picture-cued tasks.

repsonSIVE : 

repsonSIVE The tasks include interaction and test comprehension but at the limited level of short conversations, standard greetings, small talk, requests, and comments.

INTERACTIVE : 

INTERACTIVE The length and complexity of the interaction are more in interactive tasks than in responsive ones. The task sometimes includes multiple exchanges and/or multiple participants.

EXTENSIVE : 

EXTENSIVE The tasks include speeches, oral presentations, and story-telling. Oral interaction from listeners is either highly limited or ruled out altogether.

Assessment Tasks: Imitative Speaking : 

Assessment Tasks: Imitative Speaking What tasks to assess imitative speaking? Typical – listen and repeat. What kinds of prompts would you use? Examples: Test-takers hear: beat/bit bat/vat I bought a boat yesterday. The glow of the candle is growing. Test-takers repeat the stimulus. What tasks to assess imitative speaking? Typical – listen and repeat. What kinds of prompts would you use? Examples: Test-takers hear: beat/bit bat/vat I bought a boat yesterday. The glow of the candle is growing. Test-takers repeat the stimulus.

Scoring for IMITATIVE tasks : 

Scoring for IMITATIVE tasks How would you score these tasks? Example scoring scale: 2 acceptable pronunciation. 1 comprehensible, partially correct. 0 silence, seriously incorrect. How would you score these tasks? Example scoring scale: 2 acceptable pronunciation. 1 comprehensible, partially correct. 0 silence, seriously incorrect.

Phonepass Test : 

Phonepass Test Could you design a test that is done over the phone? Where the test-taker talks to a computer? What tasks could you include to test all the five basic types of speaking? Let’s look and listen at a test called PHONEPASS (Versant English Test – now offered by Pearson Education, Inc.) http://www.ordinate.com/samples/english.jsp

Phonepass Test : 

Phonepass Test What do you think about this test? Use the five basic criteria.

Phonepass Test : 

Phonepass Test What speaking skills can be assessed/inferred from the PHONEPASS test? Examples?

Assessment Tasks: Intensive Speaking : 

Assessment Tasks: Intensive Speaking What range of tasks can be used to assess intensive speaking? List some with your partner…

Assessment Tasks: Intensive Speaking : 

Assessment Tasks: Intensive Speaking Typical: Directed Response Tasks What kinds of prompts would you use? Examples: Test-takers are directed to respond: Tell me he went home. Tell me that you like rock music. Tell me that you aren’t interested in tennis. Tell him to come to my office at noon. Remind him what time it is.

Assessment Tasks: Intensive Speaking : 

Assessment Tasks: Intensive Speaking Typical: Reading aloud What kinds of prompts did PHONEPASS use? What other tasks could you design? List two other tasks with your partner…

Variations on Read-Aloud tasks : 

Variations on Read-Aloud tasks Reading a scripted dialogue. Reading sentences containing minimal pairs. Examples: The man beat his dog. The man bit his dog. Reading information from a table or chart. Reading a scripted dialogue. Reading sentences containing minimal pairs. Examples: The man beat his dog. The man bit his dog. Reading information from a table or chart.

Assessment Tasks: INTENSIVE : 

Assessment Tasks: INTENSIVE What elements of speech would you assess? List two of the major ones. Typical: Pronunciation Fluency

Assessment Tasks: INTENSIVE : 

Assessment Tasks: INTENSIVE Pronunciation: what scale would you design for scoring? Example: 0.0—0.4 frequent errors and unintelligible. 0.5—1.4 occasionally unintelligible. 1.5—2.4 some errors but intelligible. 2.5—3.0 occasional errors but always intelligible. Pronunciation: what scale would you design for scoring? Example: 0.0—0.4 frequent errors and unintelligible. 0.5—1.4 occasionally unintelligible. 1.5—2.4 some errors but intelligible. 2.5—3.0 occasional errors but always intelligible.

Assessment Tasks: INTENSIVE : 

Assessment Tasks: INTENSIVE Fluency: what scale would you design for scoring? Example: 0.0– 0.4 slow, hesitant, and unintelligible. 0.5– 1.4 non-native pauses and flow that interferes with intelligibility. 1.5- 2.4 non-native pauses but the flow is intelligible. 2.5- 3.0 smooth and effortless. Fluency: what scale would you design for scoring? Example: 0.0– 0.4 slow, hesitant, and unintelligible. 0.5– 1.4 non-native pauses and flow that interferes with intelligibility. 1.5- 2.4 non-native pauses but the flow is intelligible. 2.5- 3.0 smooth and effortless.

PHONEPASS SCORE REPORT : 

PHONEPASS SCORE REPORT Look at the PHONEPASS sample score report and answer these questions. What is the range of possible scores? What are the different bands? What did the ‘sample’ candidate score? What does the score tell the candidate? How does PHONEPASS measure up against other speaking tests?

Advantages of Intensive speaking task? : 

Advantages of Intensive speaking task? What are the advantages? Comparisons between students are quite simple. Tests are easy to prepare and to administer. Predictable output, practicality, and reliability in scoring. What are the advantages? Comparisons between students are quite simple. Tests are easy to prepare and to administer. Predictable output, practicality, and reliability in scoring.

DisAdvantages of Intensive speaking task? : 

DisAdvantages of Intensive speaking task? What are the disadvantages? It is not ‘really’ authentic is it? Exceptions A parent reading to a child, sharing a story (news) with someone giving a scripted oral presentation. It is not communicative in real contexts.

TOEFL iBT Speaking Test : 

TOEFL iBT Speaking Test Let’s take a look at some speaking task samples from the TOEFL iBT TEST. As you listen to the sample, make a note of the similarities and differences to the PHONEPASS test what tasks are used to assess the different types of speaking See research paper for background of test design. You can download the full sample iBT test.

Sentence/Dialogue Completion Tasks in iBT : 

Sentence/Dialogue Completion Tasks in iBT First, test-takers are given time to read through the dialogue to get its gist, then the tape/teacher produces one part orally and the test-taker responds. Advantages? More time to anticipate an answer No potential ambiguity created by aural misunderstanding (oral interview).

Assessment of Tasks in iBT : 

Assessment of Tasks in iBT TOEFL iBT identifies three broad areas to assess in their scoring standards. What are they? D E L I V E R Y L A N G U A G E U S E T O P I C D E V E L O P M E N T TOEFL iBT identifies three broad areas to assess in their scoring standards. What are they? D E L I V E R Y L A N G U A G E U S E T O P I C D E V E L O P M E N T

Interactive Speaking : 

Interactive Speaking Oral Interview: a test administrator and a test-taker direct face-to-face exchange proceed through a protocol of questions and directives. It can vary in length from 5 to 45 minutes, depending on purpose and context. Placement interviews may need only 5 minutes Proficiency Interviews may require an hour. Oral Interview:? a test administrator and a test-taker direct face-to-face exchange proceed through a protocol of questions and directives. It can vary in length from 5 to 45 minutes, depending on purpose and context. Placement interviews may need only 5 minutes Proficiency Interviews may require an hour.

IELTS Speaking Test : 

IELTS Speaking Test Let’s take a look at the IELTS TEST, part 1 and IELTS TEST, part2. As you listen to the sample, make a note of the the differences between IELTS and PHONEPASS the role of the ‘examiner’ what different tasks are used, and why

Assessment of Tasks in IELTS : 

Assessment of Tasks in IELTS IELTS has five broad areas in scoring: C O M P R E H E N S I O N G R A M M A R V O C A B U L A R Y P R O N U N C I A T I O N C O M M U N I C A T I V E C O M P E T E N C E IELTS has five broad areas in scoring: C O M P R E H E N S I O N G R A M M A R V O C A B U L A R Y P R O N U N C I A T I O N C O M M U N I C A T I V E C O M P E T E N C E

IELTS Speaking Test : 

IELTS Speaking Test Let’s take a look at the IELTS TEST, part 2. As you listen to the sample, try to assess the speaker using the scoring grid provided. Note your score for each aspect Compare with your partner at the end

What makes an Oral Interview a good assessment tool? : 

What makes an Oral Interview a good assessment tool? How do these relate to the 5 criteria? Clear administrative PRACTICALprocedures Focusing the questions and VALIDprobes on the purpose ofthe assessment Biased for best AUTHENTIC & WASHBACKperformance Creating a consistent, RELIABLEworkable scoring system How do these relate to the 5 criteria? Clear administrative PRACTICALprocedures Focusing the questions and VALIDprobes on the purpose ofthe assessment Biased for best AUTHENTIC & WASHBACKperformance Creating a consistent, RELIABLEworkable scoring system

LINGUISTIC PROFILING : 

LINGUISTIC PROFILING Let’s take a look at another totally different approach to assessing speaking. Linguistic profiling suggests that language development (first or second) follows a standard schedule. Speech samples collected from learners Analysed to locate the patterns Patterns matched from the sample to the regularities of the standard development schedule. See RAPID PROFILE as an example: http://groups.uni-paderborn.de/rapidprofile/

Training - TEXT : 

Training - TEXT

TRAINING - AUDIO : 

TRAINING - AUDIO

TRAINING REPORT : 

TRAINING REPORT

Overview of various speaking tasks : 

Overview of various speaking tasks For your reference, what follows is a summary of various tasks that can be used to assess different aspects of speaking. These have been summarized from Brown (2003) Language Assessment Principles.

Responsive Speaking : 

Responsive Speaking Question and Answer Examples: What is this called in English? ( to elicit a predetermined correct response) What are the steps governments should take, if any, to stem the rate of de-forestation in tropical countries? ( given more opportunity to produce meaningful language in response)

Questions Eliciting Open-Ended Responses : 

Questions Eliciting Open-Ended Responses What do you think about the weather today? Why did you choose your academic major? Personal questions: Have you ever been to the U. S. before? What other countries have you visited? Why did you go there? What did you like best about it?

Giving Instructions & Directions : 

Giving Instructions & Directions Examples: how to operate an appliance how to put a bookshelf together, or how to create a dish. Scoring: based on Comprehensibility Specified grammatical/discourse categories.

Giving Instructions & Directions : 

Giving Instructions & Directions Test-takers hear: Describe how to make a typical dish What’s a good recipe for making _____? How do you access email on a PC computer? How do I get from ___ to ____ in your city? Test-takers respond.

Eliciting Instructions or Directions : 

Eliciting Instructions or Directions The task should require the test-taker to produce at least 5 or 6 sentences. Use familiar topics and test linguistic competence. Paraphrasing, e.g. paraphrasing a story or a phone message

Considerations of Paraphrasing : 

Considerations of Paraphrasing elicit short stretches of output the criterion being assessed: Is it a listening task more than production? Does it test short-term memory rather than linguistic ability? How does the teacher determine scoring of responses?

A Framework for Oral Proficiency Testing : 

A Framework for Oral Proficiency Testing Four stages: Warm-up, Level check, Probe, and Wind-down. Warm-up the interviewer directs mutual introductions, helps the test-taker become comfortable with the situation, apprises the format, and reduces anxieties. Level check Through preplanned Qs, the test-takers respond using expected forms and functions. Linguistic target criteria are scored.

A Framework for Oral Proficiency Testing : 

A Framework for Oral Proficiency Testing Probe: In this phase, test-takers go to the heights of their ability and extend beyond the limits of the interviewer’s expectation. Through probe questions, the interviewer discovers the test-taker’s proficiency. At the lower levels of proficiency, probe items may demand a higher range of vocabulary and grammar than predicted. At the higher levels, probe items will ask the t-t to give an opinion, to recount a narrative or to respond to questions.

A Framework for Oral Proficiency Testing : 

A Framework for Oral Proficiency Testing Wind-down the interviewer encourages the test-taker to relax with some easy questions, sets the t-t’s mind at ease, and provides information about when and where to obtain the results of the interview. This part is not scored.

Sample Questions of an Oral Interview : 

Sample Questions of an Oral Interview 1. Warm-up: How are you?/What’s your name?/What country are you from?/Let me tell your about this interview. 2. Level check: How long have you been in this city?/tell me about your family./What is your major?/How long have you been working at your degree?/What are your hobbies or interests?/Why do you like your hobby? What is your favorite food?/Tell me about your exciting experience you’ve had.

Sample Questions of an Oral Interview : 

Sample Questions of an Oral Interview 3. Probe: What are your goals for learning English in this program?/Describe your academic field to me. What do you like or dislike about it?/Describe someone you greatly respect, and tell me why you respect that person./If you were [president, prime minister] of your country, what would you like to change about your country?

Sample Questions of an Oral Interview : 

Sample Questions of an Oral Interview 4. Wind-down: Did you feel okay about this interview?/You’ll get your results from this interview next week./Do you have any question to ask?/It was interesting to talk with you. Best wishes.

Picture-Cued Tasks : 

Picture-Cued Tasks A picture-cued stimulus requires a description from the test-taker. It may elicit a word, a phrase, a story, or incident. Scoring scale for intensive tasks: 2 comprehensible; acceptable target form 1 comprehensible; partially correct 0 silence; or seriously incorrect

Translation : 

Translation Translation is a communicative device in contexts where English is not a native lang. English can be called on to be interpreted as a second language. Conditions may vary from an instant translation of a native word, phrase, or sentence to a translation of longer texts. Advantages: the control of the output & easily specified scoring.

Role Play : 

Role Play It is a popular pedagogical activity in communicative language-teaching classes. The test administrator must determine the assessment objectives of the role play, then devise a scoring technique that pinpoints those objectives. Examples: “Pretend that you’re a tourist asking me for directions”, “You are buying a necklace from me in a flea market, and want a lower price”.

Discussions & Conversations : 

Discussions & Conversations As informal techniques to assess learners, D & C offer a level of authenticity and spontaneity that other assessment techniques may not provide: clarifying, questioning, paraphrasing, intonation patterns, body language, eye contact, and other sociolinguistic factors

Designing Assessments: Extensive Speaking : 

Designing Assessments: Extensive Speaking Extensive speaking tasks are frequently variations on monologues, usually with minimal verbal interaction. Oral Presentations: Examples: presenting a report, a paper, a marketing plan, a sales idea, a design of a new product, or a method.

Designing Assessments: Extensive Speaking : 

Designing Assessments: Extensive Speaking Rules for effective assessment: (a) specify the criterion, (b) set appropriate tasks, Elicit optimal output, and (d) establish practical, reliable scoring procedures. Oral presentation checklist 3 excellent 2 good 1 fair 0 poor Content: The purpose or objective of the presentation was accomplished. The introduction was lively and got my attention. The main idea or point was clearly stated toward the beginning. The supporting points were clearly expressed and supported well by facts and argument. The conclusion restated the main idea or purpose.

Designing Assessments: Extensive Speaking : 

Designing Assessments: Extensive Speaking Delivery The speaker used gestures and body language well. The speaker maintained eye contact with the audience. The speaker’s language was natural and fluent. The volume of speech was appropriate. The rate of speech was appropriate. The pronunciation was clear and comprehensible. The grammar was correct and didn’t prevent understanding. Used visual aids, handouts, etc., effectively. Showed enthusiasm and interest. Responded to audience questions well.

Picture-Cued Story-Telling : 

Picture-Cued Story-Telling At this level, a picture/a series of pictures is used as a stimulus for a longer story or description. The objective of eliciting narrative discourse needs to be clear. (p. 181) (Tell & use the P. tense) For example, are you testing for oral vocabulary, (girl, telephone, wet) for time relatives (before, after, when), for sentence connectors (then, so), for past tense of irregular verbs (woke, drank, rang), or for fluency in general? Criteria for scoring need to be clear.

Retelling a Story, News Event : 

Retelling a Story, News Event Test-takers hear /read a story or news event that they are asked to retell. It differs from the paraphrasing task discussed above in that it is a longer stretch of discourse and a different genre.

Translation (of Extended prose) : 

Translation (of Extended prose) Longer texts are presented for the test-taker to read in the native language and then translate into English. Texts vary in forms: dialogue, directions, play, movie, etc. Advantages: the control of the content, vocabulary, the grammatical and discourse features. Disadvantages: a highly specialized skill is needed.