The Test and the Learner

The Test … and the Learner: 

The Test … and the Learner

Slide2: 

“A test is not just a question paper or a set of specifications: far more importantly, it is what happens when real candidates interact with these. So an important aspect of the description of a test is the description of its intended candidature. All and any details are relevant: age, gender, linguistic/ cultural/ educational background, interests, previous experience of language tests, what they want to do with the language in future.” Wharton, Characterising Your Language Test seen at : http://www.philseflsupport.com/testing1.htm

Learner A: 

Learner A A 23 year old woman from Kuwait who is at upper intermediate level (CEFR B2). She is following a six month course in Britain, prior to taking the IELTS exam with a view to studying medicine at a British university next year. She has been in your class for the last month and knows that there is a progress test every two weeks. You have recently been reviewing various verb forms and, in particular, the passive.

Why use a progress test anyway?: 

Why use a progress test anyway? Motivation to study – positive backwash : knowing there are regular tests may encourage A to revise and consolidate what she’s done in class. That, and the test itself, provide useful recycling of the language taught Results provide feedback, both for the learner herself and the teacher, which show what needs to be done in future

Learner A and Test A: 

Learner A and Test A Learner A will be living in the UK. Test focuses on social conversation, therefore may appear relevant to A (face validity). But is really “simulated” conversation – unrealistic in terms of the real features of the spoken language. If conversational ability is only tested like this, could have negative backwash. If the class has been focusing on the passive, the test lacks content validity as a progress test. And while there is a variety of active forms, it is too easy. Will not discriminate reliably. Is an indirect test, so no test of language use – which will be important for A. Could have negative backwash. Focuses on the accurate production of discrete items – positive backwash as accuracy will be important for A in her studies.

Learner A and Test B: 

Learner A and Test B Very little content validity. Very little face validity

Learner A and Test C: 

Learner A and Test C Direct test of language in use – and a topic A will undoubtedly have to talk about. So good face validity. But – no guarantee that anything but simple past and present forms will be used. Opportunities for others – but they’re “avoidable”.

Learner A and Test D: 

Learner A and Test D High face validity – lexis and phrases used relevant to medical English – liable to be motivating. Positive backwash in emphasis on accuracy and type of lexis included – would encourage the T to include meet A’s needs But – not real “medical English” but a newspaper report. Possible negative backwash on understanding of language use if this was the type of text always given to A Content validity – high in that it “tests what its supposed to test” – ie a range of verb forms, asking the learners to distinguish between the need for active and passive. Will show A’s understanding. But – only tests affirmative forms – won’t show if she can also control neg/int. Level – vocabulary problems could affect the results. And if the vocabulary level is too high, then could be demotivating for A Again – some items have more than one possible answer. Could confuse A or cause marking problems and lead to her having a higher/lower view of her own progress than is true. (Reliability)

Learner A and Test E: 

Learner A and Test E Face validity – lexis used to carry the grammar relevant to medical English. So useful, and motivating - A can see the relevance to her subject. Also positive backwash – the course will need to include similar lexis. Content validity – tests a range of forms and includes aff, neg, int. Indirect, discrete item test – tells us only about A’s ability to recognise the forms, but emphasis on accuracy will have positive backwash for her.

Learner A and Test F: 

Learner A and Test F Connected to A’s specialist subject – face validity – but involves non-linguistic knowledge which could invalidate the test Tests the ability to take a long turn – will be necessary for A during her studies (tutorials etc) – Face validity and hopefully positive backwash Ample opportunity for a range of verb forms including passives to be used – content validity

Learner B: 

Learner B An adult Italian male of about 30. He has previously taken a placement test and tested out at around pre-intermediate level (CEFR A2), though with above-average fluency for his level. He is a training manager in a pharmaceutical company which has just been taken over by an American multinational, and needs to improve his English for his work. His needs analysis has shown that his primary need for English will be to liaise with American colleagues by Email, on the phone and when they visit the Italian company for meetings. You, his teacher, want to find out exactly what areas you should work on during the course.

Learner B – Diagnostic test: 

Learner B – Diagnostic test Of the items that he would definitely have covered by this level, what has and hasn’t he learnt/acquired ? Of the items that he might have covered by this level, what has and hasn’t he learnt/acquired?

Learner B and Test A: 

Learner B and Test A Level –about right. Will show any gaps in areas he should know, and give evidence on areas which he might or might not know. Will therefore be useful for course planning. No specific link with work requirements – lowers face validity - though items have transversal application As an indirect test it tests competence rather than performance – the fact that B can produce the verbs forms in the test, doesn’t necessarily mean he would produce them accurately in real communication

Learner B and Test B – positive points: 

Learner B and Test B – positive points Clear relevance to his work. Simulates an a situation which is authentic to his needs. Thus has face validity and is liable to be motivating. Level appropriate – the task is “do-able” for an A2 learner but not easy. It should therefore be possible to distinguish which areas cause most difficulty. Is a direct test of communicative ability, during which B will have to integrate the use of various sub-skills. Is therefore a more valid test of performance than discrete item tests. (construct validity) The “content” of the conversation is provided by the rubric – thus B is unable to “avoid” specific functions that he may be unsure of and must provide the evidence required for the diagnosis.

Learner B and Test B- negative points: 

Learner B and Test B - negative points General problems with direct testing : If B uses a form inaccurately, is this a performance or a competence problem? - If B uses a particular exponent for a function, does he know any others?

Learner B and Test C: 

Learner B and Test C If B has to socialise with his overseas colleagues, this is a topic that may well come up – content and face validity. Level – the task is “do-able” within the range of language a learner would know at this level. Should therefore produce a reliable result The group format of the task would show his interactive skills, ability to ask and respond to questions etc. Relevant to his work.

Learner B and Test D: 

Learner B and Test D Topic relevant – has face validity Level –more suitable than it might seem as: - he is familiar with the field - a large proportion of the words are Italian cognates. - the test focuses mainly on present and past verb forms. The active verbs will have been taught, the passives don’t cause Italians particular problems. Could therefore be motivating for him to realise he could cope with an authentic text But – would it diagnose anything?

Learner B and Test E: 

Learner B and Test E Three out of five sentences focus on perfect forms – too difficult at this stage, potentially confusing and demotivating, and of no diagnostic value.

Learner B and Test F: 

Learner B and Test F Tests ability to take a long turn – will be relevant if he has to attend meetings Might be too difficult – could be demotivating – though as he is particularly fluent, might cope.

The best choice?Probably …: 

The best choice? Probably … Learner A : Task E and Task F Learner B : Task A and Task B