logging in or signing up Standardized testing steveneufeld Download Post to : URL : Related Presentations : Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Uploaded from authorPOINT lite Insert YouTube videos in PowerPont slides with aS Desktop Copy embed code: (To copy code, click on the text box) Embed: URL: Thumbnail: WordPress Embed Customize Embed The presentation is successfully added In Your Favorites. Views: 1664 Category: Education License: Some Rights Reserved Like it (0) Dislike it (0) Added: October 23, 2009 This Presentation is Public Favorites: 1 Presentation Description No description available. Comments Posting comment... By: hokkiboy (17 month(s) ago) Looks useful - but a lot of the text only slides appeared as blurred... Saving..... Post Reply Close Saving..... Edit Comment Close By: anitarobins (30 month(s) ago) Thank you. I found it very informative. Anita Saving..... Post Reply Close Saving..... Edit Comment Close Premium member Presentation Transcript STANDARDS BASEDTESTING : STANDARDS BASEDTESTING A Critical Survey An ode to tests… : An ode to tests… Please God may I not fail Please God may I get over sixty per cent Please God may I get a high place Please God may all those likely to beat me get killed in road accidents and may they die roaring. Irish novelist McGahern Please God may I not fail Please God may I get over sixty per cent Please God may I get a high place Please God may all those likely to beat me get killed in road accidents and may they die roaring. Irish novelist McGahern Overview : Overview Testing Questions Types of standards-based teaching Examples of standards-based testing Why standardized tests are used Interpreting standardized test results Evaluating standardized tests Testing Questions : Testing Questions What is actually being tested by the test we are using? What is the “best” test to use? What relevant information does the test provide? How is testing affecting teaching and learning behaviour? Is language testing “fair”? What is actually being tested by the test we are using? What is the “best” test to use? What relevant information does the test provide? How is testing affecting teaching and learning behaviour? Is language testing “fair”? Types of Language Tests : Types of Language Tests Achievement test associated with process of instruction assesses where progress has been made should support the teaching to which it relates Alternative Assessment need for assessment to be integrated with the goals of the curriculum learners are engaged in self-assessment Achievement test associated with process of instruction assesses where progress has been made should support the teaching to which it relates Alternative Assessment need for assessment to be integrated with the goals of the curriculum learners are engaged in self-assessment Types of Language Tests : Types of Language Tests Proficiency test aims to establish a test taker’s readiness for a particular communicative role general measure of “language ability” measures a relatively stable trait high-stakes test Proficiency test aims to establish a test taker’s readiness for a particular communicative role general measure of “language ability” measures a relatively stable trait high-stakes test Types of Standards : Types of Standards Common European Framework of Reference for Languages (CEFR) Association of Language Testers in Europe http://www.alte.org/ (ALTE) World-Class Instructional Design and Assessment (WIDA) http://www.wida.us/ American Council on the Teaching of Foreign Languages (ACTFL) http://www.actfl.org/ Common European Framework of Reference for Languages (CEFR) Association of Language Testers in Europe http://www.alte.org/ (ALTE) World-Class Instructional Design and Assessment (WIDA) http://www.wida.us/ American Council on the Teaching of Foreign Languages (ACTFL) http://www.actfl.org/ CEFR : CEFR CEFR : CEFR CEFR : CEFR ACTFL : ACTFL Equivalency : Equivalency Equivalency : Equivalency Some ways of describing tests : Some ways of describing tests Objective Subjective Indirect Direct Discrete-point Integrative Aptitude/Achievement Proficiency/Performance External Internal Norm -Referenced Criterion-Referenced Objective Subjective Indirect Direct Discrete-point Integrative Aptitude/Achievement Proficiency/Performance External Internal Norm -Referenced Criterion-Referenced Evaluating the usefulness of a language test : Evaluating the usefulness of a language test Usefulness=? Practicality+ Reliability+ Validity+ Authenticity+ Washback (Brown, 2003) Usefulness=? Practicality+ Reliability+ Validity+ Authenticity+ Washback (Brown, 2003) Evaluating the usefulness of a language test : Evaluating the usefulness of a language test Usefulness= TEST USEFULNESS RELIABILITY VALIDITY Washback Authenticity Practicality Evaluating the usefulness of a language test : Evaluating the usefulness of a language test Essential measurement qualities reliability construct validity Evaluation: test taker - test task - Target Language Use (TLU) TLU Test Task Test Taker TWITTER discussion #efl413 : TWITTER discussion #efl413 Are standardized tests better than teacher-made tests? When are standardized tests useful? Some people claim standardized tests are discriminatory. Do you agree? Why? Will your experience of standardized tests affect how you teach/test? How? Overview of common language proficiency tests : Overview of common language proficiency tests TOEFL IELTS ETS, US AU, EU Purpose of TOEFL? : Purpose of TOEFL? Purpose of TOEFL? : Purpose of TOEFL? iBT Test construct? : iBT Test construct? Do the item types and tasks appropriately represent the conceptualization of language proficiency (i.e. language ability) What do you predict the iBT test construct is in terms of skills, language focus, scoring, reliability and validity? Test construct… : Test construct… Test Specifications : Test Specifications Outline – what considerations? Time/sections/equipment/seating/etc Skills/language points – which ones? Grammar, vocabulary, reading, listening, etc. Test outline : Test outline Time up to four hours to complete all four sections - same day. Reading ability to understand academic reading material Listening ability to understand spoken English as used in universities. Speaking six tasks that measure the ability to speak English in an academic setting. Writing two tasks that measure the ability to write in a way that is appropriate for college and university Time up to four hours to complete all four sections - same day. Reading ability to understand academic reading material Listening ability to understand spoken English as used in universities. Speaking six tasks that measure the ability to speak English in an academic setting. Writing two tasks that measure the ability to write in a way that is appropriate for college and university Test Specifications : Test Specifications Outline – what considerations? Time/sections/equipment/seating/etc Skills/language points – which ones? Grammar, vocabulary, reading, listening, etc. Test items and tasks – two processes? Input and output Validity : Validity Integrative test types Some questions in the Speaking and Writing require the test takers to combine, or integrate, information from more than one source. For example, read a passage, listen to a short lecture about a topic, and then speak or write in response. Why? These integrated tasks are designed to simulate the academic experience. Construct validity? Face Validity? Test specification - Reading : Test specification - Reading Test design – TOEFL Reading : Test design – TOEFL Reading Test design - objectives : Test design - objectives Main idea (#11) Stated details (#17, #19) Unstated details (#12, #15, #18) Implied details (#14, #20) Vocabulary in context (#13, #16) Main idea (#11) Stated details (#17, #19) Unstated details (#12, #15, #18) Implied details (#14, #20) Vocabulary in context (#13, #16) Norms : Norms Gives meaning to individual scores Age-equivalent Grade-equivalent Percentile scores Caution Are norms appropriate? Same age, skill, etc. Use for individual not group assessment Norm or Criteron referenced? : Norm or Criteron referenced? Percentile scores : Percentile scores Indicates the percentage of people an individual did better than in a norming or reference group. A person scoring at the 86th percentile did better than 86% of those in the norming or reference group. A percentile is calculated for each obtained score. Indicates the percentage of people an individual did better than in a norming or reference group. A person scoring at the 86th percentile did better than 86% of those in the norming or reference group. A percentile is calculated for each obtained score. Calculating Percentile scores : Calculating Percentile scores Calculate the percentage of scores which fall below a score. Calculate the percentage of times that score occurs and divide by two. Add the percentages in steps 1 and 2 and round up or down. 10 scores: 15 12 10 9 8 7 7 6 4 2 : 10 scores: 15 12 10 9 8 7 7 6 4 2 Step 1: Calculate the percentage of scores which fall below a score. There are 3 scores below 7. These are 6, 4, and 2. There are 10 scores total. 3 divided by 10 times 100 equals 30. 10 scores: 15 12 10 9 8 7 7 6 4 2 : 10 scores: 15 12 10 9 8 7 7 6 4 2 Step 2: Calculate the percentage of times that score occurs and divide by two. There are two 7s. 2 divided by 10 times 100 equals 20. Dividing 20 in half equals 10. 10 scores: 15 12 10 9 8 7 7 6 4 2 : 10 scores: 15 12 10 9 8 7 7 6 4 2 Step 3: Add the percentages in steps 1 and 2 and round up or down. 30 (step 1) plus 10 (step 2) equals 40. Thus, a person scoring a 7 would be at the 40th percentile. Discussion : Discussion What do percentile scores tell us about students’ language abilities? How is this information different to scores on classroom tests? Test of English as a Foreign Language : Test of English as a Foreign Language Objective Subjective Discrete-point Integrative Proficiency Achievement discord between test and understanding of language and communication (validity type?) passive recognition of language general proficiency academic proficiency International English Language Testing System : International English Language Testing System Academic/General Results reported in band scores 1-9 Listening G.Reading A.Reading G.Writing A.Writing Speaking International English Language Testing System : International English Language Testing System Objective Subjective Discrete-point Integrative Proficiency Achievement test tasks reflective of academic tasks score reporting is diagnostic need for reliability research Washback: The Impact of Tests on Teaching and Learning : Washback: The Impact of Tests on Teaching and Learning “The power of tests has a strong influence on curriculum and learning outcomes” (Shohamy, 1993) good test positive washback form of test impact depends on educational context and condition process of testing consequences of evaluation (Wall, 2000) “The power of tests has a strong influence on curriculum and learning outcomes” (Shohamy, 1993) good test positive washback form of test impact depends on educational context and condition process of testing consequences of evaluation (Wall, 2000) Critical Language Testing : Critical Language Testing Tests have become tools which, in the name of objectivity, have created and perpetuated new subjective powers, and defined and dictated society’s knowledge by building on the fear and trust of the public who are affected by their results. (Shohamy, 2001, p.159) Tests have become tools which, in the name of objectivity, have created and perpetuated new subjective powers, and defined and dictated society’s knowledge by building on the fear and trust of the public who are affected by their results. (Shohamy, 2001, p.159) Critical Language Testing : Critical Language Testing Tests have become tools where are used to perpetuate power and control, to screen and keep out those who are not part of the mainstream knowledge. (Shohamy, 2001, p.159) Tests have become tools where are used to perpetuate power and control, to screen and keep out those who are not part of the mainstream knowledge. (Shohamy, 2001, p.159) Critical Language Testing : Critical Language Testing The tests, originally developed for democratizing purposes have become authoritative and centralized tools which are being manipulated in the hands of ‘a few’. This is perpetuated by the symbolic values of tests in most modern societies. (Shohamy, 2001, p.159) The tests, originally developed for democratizing purposes have become authoritative and centralized tools which are being manipulated in the hands of ‘a few’. This is perpetuated by the symbolic values of tests in most modern societies. (Shohamy, 2001, p.159) Critical Language Testing : Critical Language Testing Test have become symbols of quality, standards, achievements and higher level order. (Shohamy, 2001, p.159) Test have become symbols of quality, standards, achievements and higher level order. (Shohamy, 2001, p.159) Critical Language Testing : Critical Language Testing Focus on consequence and ethics of test use Tests are embedded in cultural, educational, and political arenas whose agenda? Questions traditional testing knowledge English proficiency= academic success? English: got it or get it! Responsible test use Cultural bias : Cultural bias Look at the follow reading from a proficiency test. Work with a partner…how would you critique this test item in terms of critical language testing? Discussion point : Discussion point Compare the differences in conceptualization of language proficiency represented by IELTS and TOEFL. Do they promote our current understanding of communicative language ability? What are the strengths and weaknesses of each? Plenary : Plenary Types of standards-based teaching Examples of standards-based testing Why standardized tests are used Interpreting standardized test results Evaluating standardized tests Testing Questions Acknowledgements : Acknowledgements Parts of this presentation were based on a presentation by Ruth Hungerland, at Memorial University of Newfoundland, TESL Newfoundland and Labrador You do not have the permission to view this presentation. In order to view it, please contact the author of the presentation.
Standardized testing steveneufeld Download Post to : URL : Related Presentations : Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Uploaded from authorPOINT lite Insert YouTube videos in PowerPont slides with aS Desktop Copy embed code: (To copy code, click on the text box) Embed: URL: Thumbnail: WordPress Embed Customize Embed The presentation is successfully added In Your Favorites. Views: 1664 Category: Education License: Some Rights Reserved Like it (0) Dislike it (0) Added: October 23, 2009 This Presentation is Public Favorites: 1 Presentation Description No description available. Comments Posting comment... By: hokkiboy (17 month(s) ago) Looks useful - but a lot of the text only slides appeared as blurred... Saving..... Post Reply Close Saving..... Edit Comment Close By: anitarobins (30 month(s) ago) Thank you. I found it very informative. Anita Saving..... Post Reply Close Saving..... Edit Comment Close Premium member Presentation Transcript STANDARDS BASEDTESTING : STANDARDS BASEDTESTING A Critical Survey An ode to tests… : An ode to tests… Please God may I not fail Please God may I get over sixty per cent Please God may I get a high place Please God may all those likely to beat me get killed in road accidents and may they die roaring. Irish novelist McGahern Please God may I not fail Please God may I get over sixty per cent Please God may I get a high place Please God may all those likely to beat me get killed in road accidents and may they die roaring. Irish novelist McGahern Overview : Overview Testing Questions Types of standards-based teaching Examples of standards-based testing Why standardized tests are used Interpreting standardized test results Evaluating standardized tests Testing Questions : Testing Questions What is actually being tested by the test we are using? What is the “best” test to use? What relevant information does the test provide? How is testing affecting teaching and learning behaviour? Is language testing “fair”? What is actually being tested by the test we are using? What is the “best” test to use? What relevant information does the test provide? How is testing affecting teaching and learning behaviour? Is language testing “fair”? Types of Language Tests : Types of Language Tests Achievement test associated with process of instruction assesses where progress has been made should support the teaching to which it relates Alternative Assessment need for assessment to be integrated with the goals of the curriculum learners are engaged in self-assessment Achievement test associated with process of instruction assesses where progress has been made should support the teaching to which it relates Alternative Assessment need for assessment to be integrated with the goals of the curriculum learners are engaged in self-assessment Types of Language Tests : Types of Language Tests Proficiency test aims to establish a test taker’s readiness for a particular communicative role general measure of “language ability” measures a relatively stable trait high-stakes test Proficiency test aims to establish a test taker’s readiness for a particular communicative role general measure of “language ability” measures a relatively stable trait high-stakes test Types of Standards : Types of Standards Common European Framework of Reference for Languages (CEFR) Association of Language Testers in Europe http://www.alte.org/ (ALTE) World-Class Instructional Design and Assessment (WIDA) http://www.wida.us/ American Council on the Teaching of Foreign Languages (ACTFL) http://www.actfl.org/ Common European Framework of Reference for Languages (CEFR) Association of Language Testers in Europe http://www.alte.org/ (ALTE) World-Class Instructional Design and Assessment (WIDA) http://www.wida.us/ American Council on the Teaching of Foreign Languages (ACTFL) http://www.actfl.org/ CEFR : CEFR CEFR : CEFR CEFR : CEFR ACTFL : ACTFL Equivalency : Equivalency Equivalency : Equivalency Some ways of describing tests : Some ways of describing tests Objective Subjective Indirect Direct Discrete-point Integrative Aptitude/Achievement Proficiency/Performance External Internal Norm -Referenced Criterion-Referenced Objective Subjective Indirect Direct Discrete-point Integrative Aptitude/Achievement Proficiency/Performance External Internal Norm -Referenced Criterion-Referenced Evaluating the usefulness of a language test : Evaluating the usefulness of a language test Usefulness=? Practicality+ Reliability+ Validity+ Authenticity+ Washback (Brown, 2003) Usefulness=? Practicality+ Reliability+ Validity+ Authenticity+ Washback (Brown, 2003) Evaluating the usefulness of a language test : Evaluating the usefulness of a language test Usefulness= TEST USEFULNESS RELIABILITY VALIDITY Washback Authenticity Practicality Evaluating the usefulness of a language test : Evaluating the usefulness of a language test Essential measurement qualities reliability construct validity Evaluation: test taker - test task - Target Language Use (TLU) TLU Test Task Test Taker TWITTER discussion #efl413 : TWITTER discussion #efl413 Are standardized tests better than teacher-made tests? When are standardized tests useful? Some people claim standardized tests are discriminatory. Do you agree? Why? Will your experience of standardized tests affect how you teach/test? How? Overview of common language proficiency tests : Overview of common language proficiency tests TOEFL IELTS ETS, US AU, EU Purpose of TOEFL? : Purpose of TOEFL? Purpose of TOEFL? : Purpose of TOEFL? iBT Test construct? : iBT Test construct? Do the item types and tasks appropriately represent the conceptualization of language proficiency (i.e. language ability) What do you predict the iBT test construct is in terms of skills, language focus, scoring, reliability and validity? Test construct… : Test construct… Test Specifications : Test Specifications Outline – what considerations? Time/sections/equipment/seating/etc Skills/language points – which ones? Grammar, vocabulary, reading, listening, etc. Test outline : Test outline Time up to four hours to complete all four sections - same day. Reading ability to understand academic reading material Listening ability to understand spoken English as used in universities. Speaking six tasks that measure the ability to speak English in an academic setting. Writing two tasks that measure the ability to write in a way that is appropriate for college and university Time up to four hours to complete all four sections - same day. Reading ability to understand academic reading material Listening ability to understand spoken English as used in universities. Speaking six tasks that measure the ability to speak English in an academic setting. Writing two tasks that measure the ability to write in a way that is appropriate for college and university Test Specifications : Test Specifications Outline – what considerations? Time/sections/equipment/seating/etc Skills/language points – which ones? Grammar, vocabulary, reading, listening, etc. Test items and tasks – two processes? Input and output Validity : Validity Integrative test types Some questions in the Speaking and Writing require the test takers to combine, or integrate, information from more than one source. For example, read a passage, listen to a short lecture about a topic, and then speak or write in response. Why? These integrated tasks are designed to simulate the academic experience. Construct validity? Face Validity? Test specification - Reading : Test specification - Reading Test design – TOEFL Reading : Test design – TOEFL Reading Test design - objectives : Test design - objectives Main idea (#11) Stated details (#17, #19) Unstated details (#12, #15, #18) Implied details (#14, #20) Vocabulary in context (#13, #16) Main idea (#11) Stated details (#17, #19) Unstated details (#12, #15, #18) Implied details (#14, #20) Vocabulary in context (#13, #16) Norms : Norms Gives meaning to individual scores Age-equivalent Grade-equivalent Percentile scores Caution Are norms appropriate? Same age, skill, etc. Use for individual not group assessment Norm or Criteron referenced? : Norm or Criteron referenced? Percentile scores : Percentile scores Indicates the percentage of people an individual did better than in a norming or reference group. A person scoring at the 86th percentile did better than 86% of those in the norming or reference group. A percentile is calculated for each obtained score. Indicates the percentage of people an individual did better than in a norming or reference group. A person scoring at the 86th percentile did better than 86% of those in the norming or reference group. A percentile is calculated for each obtained score. Calculating Percentile scores : Calculating Percentile scores Calculate the percentage of scores which fall below a score. Calculate the percentage of times that score occurs and divide by two. Add the percentages in steps 1 and 2 and round up or down. 10 scores: 15 12 10 9 8 7 7 6 4 2 : 10 scores: 15 12 10 9 8 7 7 6 4 2 Step 1: Calculate the percentage of scores which fall below a score. There are 3 scores below 7. These are 6, 4, and 2. There are 10 scores total. 3 divided by 10 times 100 equals 30. 10 scores: 15 12 10 9 8 7 7 6 4 2 : 10 scores: 15 12 10 9 8 7 7 6 4 2 Step 2: Calculate the percentage of times that score occurs and divide by two. There are two 7s. 2 divided by 10 times 100 equals 20. Dividing 20 in half equals 10. 10 scores: 15 12 10 9 8 7 7 6 4 2 : 10 scores: 15 12 10 9 8 7 7 6 4 2 Step 3: Add the percentages in steps 1 and 2 and round up or down. 30 (step 1) plus 10 (step 2) equals 40. Thus, a person scoring a 7 would be at the 40th percentile. Discussion : Discussion What do percentile scores tell us about students’ language abilities? How is this information different to scores on classroom tests? Test of English as a Foreign Language : Test of English as a Foreign Language Objective Subjective Discrete-point Integrative Proficiency Achievement discord between test and understanding of language and communication (validity type?) passive recognition of language general proficiency academic proficiency International English Language Testing System : International English Language Testing System Academic/General Results reported in band scores 1-9 Listening G.Reading A.Reading G.Writing A.Writing Speaking International English Language Testing System : International English Language Testing System Objective Subjective Discrete-point Integrative Proficiency Achievement test tasks reflective of academic tasks score reporting is diagnostic need for reliability research Washback: The Impact of Tests on Teaching and Learning : Washback: The Impact of Tests on Teaching and Learning “The power of tests has a strong influence on curriculum and learning outcomes” (Shohamy, 1993) good test positive washback form of test impact depends on educational context and condition process of testing consequences of evaluation (Wall, 2000) “The power of tests has a strong influence on curriculum and learning outcomes” (Shohamy, 1993) good test positive washback form of test impact depends on educational context and condition process of testing consequences of evaluation (Wall, 2000) Critical Language Testing : Critical Language Testing Tests have become tools which, in the name of objectivity, have created and perpetuated new subjective powers, and defined and dictated society’s knowledge by building on the fear and trust of the public who are affected by their results. (Shohamy, 2001, p.159) Tests have become tools which, in the name of objectivity, have created and perpetuated new subjective powers, and defined and dictated society’s knowledge by building on the fear and trust of the public who are affected by their results. (Shohamy, 2001, p.159) Critical Language Testing : Critical Language Testing Tests have become tools where are used to perpetuate power and control, to screen and keep out those who are not part of the mainstream knowledge. (Shohamy, 2001, p.159) Tests have become tools where are used to perpetuate power and control, to screen and keep out those who are not part of the mainstream knowledge. (Shohamy, 2001, p.159) Critical Language Testing : Critical Language Testing The tests, originally developed for democratizing purposes have become authoritative and centralized tools which are being manipulated in the hands of ‘a few’. This is perpetuated by the symbolic values of tests in most modern societies. (Shohamy, 2001, p.159) The tests, originally developed for democratizing purposes have become authoritative and centralized tools which are being manipulated in the hands of ‘a few’. This is perpetuated by the symbolic values of tests in most modern societies. (Shohamy, 2001, p.159) Critical Language Testing : Critical Language Testing Test have become symbols of quality, standards, achievements and higher level order. (Shohamy, 2001, p.159) Test have become symbols of quality, standards, achievements and higher level order. (Shohamy, 2001, p.159) Critical Language Testing : Critical Language Testing Focus on consequence and ethics of test use Tests are embedded in cultural, educational, and political arenas whose agenda? Questions traditional testing knowledge English proficiency= academic success? English: got it or get it! Responsible test use Cultural bias : Cultural bias Look at the follow reading from a proficiency test. Work with a partner…how would you critique this test item in terms of critical language testing? Discussion point : Discussion point Compare the differences in conceptualization of language proficiency represented by IELTS and TOEFL. Do they promote our current understanding of communicative language ability? What are the strengths and weaknesses of each? Plenary : Plenary Types of standards-based teaching Examples of standards-based testing Why standardized tests are used Interpreting standardized test results Evaluating standardized tests Testing Questions Acknowledgements : Acknowledgements Parts of this presentation were based on a presentation by Ruth Hungerland, at Memorial University of Newfoundland, TESL Newfoundland and Labrador