Language_assessment_principles

Views:
 
Category: Entertainment
     
 

Presentation Description

No description available.

Comments

Presentation Transcript

Language Assessment Principles : 

Language Assessment Principles EFL413

In theory…but in practice? : 

In theory…but in practice? Tests Assessment Teaching

In theory…but in practice? : 

In theory…but in practice? Choral drill Practicing pronunciation in pairs Reading aloud Information gap Singing songs Writing in a learner journal What percentage of these teaching techniques is related to assessment?

Basic principles : 

Basic principles Regular (formal and informal) assessment Monitor progress and motivate students Get learners to set goals, see development Assessment tools appropriate to product and process Reinforce, remember and apply Self-assessment and guided autonomy Diagnostic Strengths and weaknesses (teaching and learning) Regular (formal and informal) assessment Monitor progress and motivate students Get learners to set goals, see development Assessment tools appropriate to product and process Reinforce, remember and apply Self-assessment and guided autonomy Diagnostic Strengths and weaknesses (teaching and learning)

Current Issues : 

Current Issues M u l t i p l e I n t e l l i g e n c e s A l t e r n a t i v e A s s e s s m e n t C o m p u t e r T e s t i n g M u l t i p l e I n t e l l i g e n c e s A l t e r n a t i v e A s s e s s m e n t C o m p u t e r T e s t i n g

Intelligence : 

Intelligence Reasoning? Memory? Problem solving? Reasoning? Memory? Problem solving?

Can you measure (test) IQ? : 

Can you measure (test) IQ? Traditional IQ test Norm-referenced Discrete-point Timed Type of ‘intelligence’ Linguistic Logical-Mathematical Alternative views Gardner (1983) Spatial, Musical, Bodily-kinesthetic, Interpersonal, Intrapersonal, Naturalistic Sternberg (1988) Creative thinking – not timed Manipulating strategies - innovative Goleman (1995) EQ – manage emotions Anger, grief, self-doubt impair peak performance

The jury’s out…how to balance : 

The jury’s out…how to balance Objectivity Discrete-point Analytical Impersonal Subjectivity and intuition Creative Communicative Interactive

Traditional versus Alternative : 

Traditional versus Alternative

Alternative assessment : 

Alternative assessment Traditional One-off, standardized Timed, multiple-choice Decontextualized Score = feedback Norm-referenced One ‘right’ answer Summative Measure the ‘product’ Non-interactive Extrinsic motivation Alternative Continuous, long-term Unlimited, free response Context, communicative Individualized, washback Criterion-referenced Open-ended, creative Formative Measure the ‘process’ Interactive Intrinsic motivation

Computers and testing : 

Computers and testing Computer based Computer adaptive

Computer testing and you : 

Computer testing and you Have you had any experiences with computer-based and or computer-adaptive testing? From your experience, what were the advantages and disadvantages?

Features : 

Features Computer based ? Computer adaptive ?

Features : 

Features Computer based Hot potatoes – gaps, M/C Text or multi-media Mainly closed-ended Can combine open-ended (assessed by human) New trend – artificial intelligence (see Pearson Test of English – computer grades writing and speaking) Computer adaptive Test-taker must answer questions in sequence More efficient – test-taker only sees questions appropriate to ability Generally shorter Differentiated to individual learner Difficult to design

Computers and testing : 

Computers and testing Pros ? Cons ?

Computers and testing : 

Computers and testing Pros Classroom-based and self-assessment Practice for high-stakes exams Large-scale – easy to administer Cons Security Internet – dubious testing standards Multiple choice preferred Miss human/interactive element

How do you know if a test is effective? : 

How do you know if a test is effective? What are the key questions?

How do you know if a test is effective? : 

How do you know if a test is effective? Can it be given within the time and space allowed? Are the results dependable? Does it measure what we want it to? Is the measure accurate? Does it have a positive effect on teaching and learning? Can it be given within the time and space allowed? Are the results dependable? Does it measure what we want it to? Is the measure accurate? Does it have a positive effect on teaching and learning?

Five key criteria : 

Five key criteria

PRACTICAL : 

PRACTICAL What makes a test practical? Easy to administer Cost (not expensive) Time (within constraints) Resources (people, rooms, equipment) Easy to mark Scoring procedure is clear Criteria are specific Process is time-efficient What makes a test practical? Easy to administer Cost (not expensive) Time (within constraints) Resources (people, rooms, equipment) Easy to mark Scoring procedure is clear Criteria are specific Process is time-efficient

RELIABLE : 

RELIABLE What makes a test reliable? Consistent and reliable… Same test, same/similar students, same results What can negatively affect reliability? Learner variables… Illness, psychology, etc. What makes a test reliable? Consistent and reliable… Same test, same/similar students, same results What can negatively affect reliability? Learner variables… Illness, psychology, etc.

RELIABLE : 

RELIABLE What can negatively affect reliability? Learner variables… Illness, psychology, etc. Scoring variables… Inter-scoring… two scorers, different results Need clear standards and standardization Intra-scoring… time, carelessness, prejudice Read all, review, blind, analytical/holistic grading What can negatively affect reliability? Learner variables… Illness, psychology, etc. Scoring variables… Inter-scoring… two scorers, different results Need clear standards and standardization Intra-scoring… time, carelessness, prejudice Read all, review, blind, analytical/holistic grading

RELIABLE : 

RELIABLE What can negatively affect reliability? Test conditions… Noise, chairs, temperature, equipment, etc. The test… Ambiguous, or culturally biased Favours one type of student – same ‘intelligence’ as test creator What can negatively affect reliability? Test conditions… Noise, chairs, temperature, equipment, The test… Ambiguous, or culturally biased Favours one type of student – same ‘intelligence’ as test creator

VALID : 

VALID “…the extent to which inferences made from assessment results are appropriate, meaningful, and useful in terms of the purpose of the assessment.” Brown, 2004 Can you think of examples of a testing situation that isn’t ‘valid’? Think about a driving test…

VALID : 

VALID What makes a test valid? Content validity… if it reflects the content you are trying to measure.

VALID : 

VALID What makes a test valid? What is the content validity of this quiz in a listening/speaking class of Elementary group after a class discussion about animals in zoos?

VALID? : 

VALID?

VALID : 

VALID Direct Test-taker performs the task E.g. saying words with the correct stress. Indirect Test-taker completes a task that is somehow related E.g. marking the stress patterns above words on paper.

VALID : 

VALID What makes a test valid? Criterion validity… it the test results showing the achievement of objectives are reflected in a different test of the same objectives. E.g. If a student gets 60% on a METU Proficiency and gets 6.0 on an IELTS (concurrent or predictive) What makes a test valid? Criterion validity… it the test results showing the achievement of objectives are reflected in a different test of the same objectives. E.g. If a student gets 60% on a METU Proficiency and gets 6.0 on an IELTS (concurrent or predictive)

VALID : 

VALID What makes a test valid? Construct validity… if the test results can be shown to reflect an ability or mental trait which cannot necessarily be directly observed or measured. Language tests attempt to measure the different constructs which underlie language ability, e.g. listening is the ability, the construct is meaning inferred by intonation. In addition to language ability itself, motivation, attitude and acculturation are all relevant constructs. ALTE What makes a test valid? Construct validity… if the test results can be shown to reflect an ability or mental trait which cannot necessarily be directly observed or measured. Language tests attempt to measure the different constructs which underlie language ability, e.g. listening is the ability, the construct is meaning inferred by intonation. In addition to language ability itself, motivation, attitude and acculturation are all relevant constructs. ALTE

VALID : 

VALID What makes a test valid? Construct validity would be in question in an oral speaking test for proficiency if … the test only measured pronunciation and grammar, because... the theoretical construct of proficiency would also embrace fluency, vocabulary use, appropriate socio-linguistic responses. ALTE

VALID : 

VALID What makes a test valid? Face validity… it the test “looks” right and “appears” to measure what it intends to measure. Based on subjective judgment of the test-takers, administrators and teachers What makes a test valid? Face validity… it the test “looks” right and “appears” to measure what it intends to measure. Based on subjective judgment of the test-takers, administrators and teachers

VALID : 

VALID What makes a test valid? Face validity is high if… Format is familiar Tasks are doable within the time limits No deliberate tricks to confuse Instructions are clear Tasks are related (i.e. content validity) i+1 – challenging but not impossible to do well What makes a test valid? Face validity is high if… Format is familiar Tasks are doable within the time limits No deliberate tricks to confuse Instructions are clear Tasks are related (i.e. content validity) i+1 – challenging but not impossible to do well

AUTHENTICITY : 

AUTHENTICITY What makes a test authentic? A test is authentic if… the test tasks reflect language use in a non-test situation.

AUTHENTICITY : 

AUTHENTICITY What makes a test authentic? An authentic test… uses natural language as much as possible presents items in context includes topics that are meaningful to students has a theme carried through the items includes ‘real-world’ tasks What makes a test authentic? An authentic test… uses natural language as much as possible presents items in context includes topics that are meaningful to students has a theme carried through the items includes ‘real-world’ tasks

WASHBACK : 

WASHBACK Washback is the effect a test has on teaching and learning Washback can be negative if … the test encourages students to ‘cram’ the test encourages teachers to ‘teach for the test’ the result is only given as a grade or percentage Washback is the effect a test has on teaching and learning Washback can be negative if … the test encourages students to ‘cram’ the test encourages teachers to ‘teach for the test’ the result is only given as a grade or percentage

WASHBACK : 

WASHBACK Washback is the effect a test has on teaching and learning Washback can be positive if … The tests reinforce the teaching techniques and formative assessment Feedback is meaningful and promotes basic language learning principles… Intrinsic motivation, self-confidence, language ego, strategic investment, etc. Washback is the effect a test has on teaching and learning Washback can be positive if … The tests reinforce the teaching techniques and formative assessment Feedback is meaningful and promotes basic language learning principles… Intrinsic motivation, self-confidence, language ego, strategic investment, etc.

WASHBACK : 

WASHBACK Washback is the effect a test has on teaching and learning Washback can be applied best in formative assessment, but it can be applied in summative assessment if… the summative assessment is given before the end of the course, and feedback provided before the end often done through the use of ‘mock’ tests. Washback is the effect a test has on teaching and learning Washback can be applied best in formative assessment, but it can be applied in summative assessment if… the summative assessment is given before the end of the course, and feedback provided before the end often done through the use of ‘mock’ tests.

MULTIPLE INTELLIGENCES : 

MULTIPLE INTELLIGENCES Make a list of Gardner's intelligences. Choose one or two intelligences, and brainstorm some teaching activities that foster that type of intelligence. Now, come up with some assessment tasks that may presume the same intelligence in order to perform well. Share your ideas. Consider if they are practical, reliable, valid, authentic and the washback

Reading : 

Reading Brown course book (download link from FACEBOOK group) Chapters 1 and 2. MOODLE Form groups for testing tasks Follow up task for this session to follow in MOODLE. Start your e-portfolio and DIALANG Also, course expections, theory of language

authorStream Live Help