Language Testing Services

Assessing Reading Literacy in the Language of Instruction : 

Assessing Reading Literacy in the Language of Instruction John H.A.L. de Jong Intergovernmental Conference: Languages of Schooling towards a Framework for Europe Strasbourg, 16-18 October 2006

Overview : 

Overview Background of the PISA studies Design of the PISA studies PISA approach to Reading Literacy Some central findings of PISA 2000 Extensions for PISA 2009 Relationships between PISA & CEF Applying CEF in Upper Secondary Vocational

Background of PISA study : 

Background of PISA study OECD: Organisation for Economic Co-operation and Development 30 member countries committed to democracy and market economy Provide comparative data and analyses For countries to compare policy experience; seek answers to common problems; identify good practice; co-ordinate policies. To support economic growth, boost employment, raise living standards.

History OECD : 

History OECD 1948 Organisation for European Economic Co-operation (OEEC), set up with support from the United States and Canada to co-ordinate the Marshall Plan for the reconstruction of Europe after World War II (www.oecd.org/history). 1961 Created as an economic counterpart to NATO, the OECD took over from the OEEC Early recognition of importance of education of its Citizens 1985  Gather indicators on investment in education Education at a glance yearly publication 2000 first PISA

PISA Goals : 

PISA Goals After studying resources & investments in education for a number of consecutive years: Wish to know the yield of education: at the end of compulsory education how well are students prepared to continue further education or to start working life?

PISA assessment schedule : 

PISA assessment schedule

PISA assessment schedule : 

PISA assessment schedule

Participation in OECD/PISA : 

Participation in OECD/PISA

Definition : 

Definition Reading literacy is understanding, using and reflecting on written texts, in order to achieve one’s goals, to develop one’s knowledge and potential and to participate in society.

Slide 10: 

Retrieving information Interpreting text Reflection & Evaluation

Let’s look at Reading : 

Let’s look at Reading In the activity of reading we can distinguish four main elements: 1 there is a reader 2 there is a text (the author is part of the text, but not present in the activity) Both of these are observable. 3 the reader has a goal (a self-chosen reason for reading the text) 4 the reading activity will have a result (a change in the reader: more knowledge, satisfaction, etc ) These are not observable.

Reading in ‘real’ life : 

Text - Reader - Goal - Result Assessing Reading: Reading in ‘real’ life - Scoring rules - Score End point of process

A Reading item : 

refers to (part of) a text sets task pertaining to the text, requiring particular subskill(s) specifies format for overt response demonstrating application of subskill specifies rules for scoring response A Reading item may require to link information from the text to prior knowledge and experience

Aspects of item difficulty : 

Difficulty of comprehension depends on text characteristics Aspects of item difficulty Text Comprehension is the interaction between text & reader Difficulty of task depends on prior knowledge & experience of candidate Difficulty of item depends on scoring rules Difficulty of item depends on skills required to perform task Difficulty of item depends on response format

PISA 2000: Relation SES & Reading : 

PISA 2000: Relation SES & Reading

Slide 19: 

Mean PISA 2003 scores per educational track in the Netherlands

Slide 20: 

Mean Reading Literacy score for Dutch-born students and for immigrant students

Language proficiency in upper secondary vocational education : 

Language proficiency in upper secondary vocational education CINOP Study using CEF what is the actual language proficiency of students? is there a discrepancy between actual and required language proficiency for further education and professional occupation?

Research design : 

Research design - written questionnaires for language teachers and vocational teachers, N = 210 - self assessment on language proficiency for students, N= 345 - followed by a test for reading, N= 328 - oral interviews with teachers, staff members and management, N= 40

Teachers’ views : 

Teachers’ views 60 to 70 % estimate language proficiency (reading A2-B1) insufficient for school 10 % estimate the language proficiency on a higher level than needed for school 80 % estimate the language proficiency insufficient for professional occupation

Self assessment by students : 

Self assessment by students > 2/3 of students estimate their reading proficiency level at B2 In reality: 7 % at A1 24 % at A2 52 % at A2-B1 17 % at B1-B2

CEF appropriate for description of language of instruction : 

CEF appropriate for description of language of instruction Coherent and workable structure for teachers (contributes to improvement of their professional competences) Competence based descriptions more suitable for language of instruction than linguistic descriptions of L1 Descriptors applicable More transparency and motivating for students (specially in combination with a language portfolio) Offers opportunity for reflection on language learning by students

Shortcomings of CEF in describing language of instruction in vocational education : 

Shortcomings of CEF in describing language of instruction in vocational education Can do statements are geared at academic learning routes: not well adjusted to describing specific tasks in vocational education and specific tasks in the workplace More situations and examples have need to be offered for a range of vocational sectors

“Milestones” : 

“Milestones” 2000 1st PISA Cycle: Reading Major Domain 2002 Publication Reading for Change 2004 CINOP Dutch in Vocational Education 2006 ALARM: Language skills at risk