Simplifying the Language of Tests :Simplifying the Language of Tests Or… on becoming clearer than mud Natalie Taranec Intensive English 28th November 2008


Simplifying the Language of Tests- A Rationale :Simplifying the Language of Tests- A Rationale Dumbing Down Exclusivity Language used to baffle: confuse, exclude, withhold meaning, with loaded meanings


“Yes, Minister”source: http://en.wikiquote.org/wiki/Yes,_Minister , accessed 25 November 2008 :“Yes, Minister”source: http://en.wikiquote.org/wiki/Yes,_Minister , accessed 25 November 2008 Sir Humphrey: Unfortunately, although the answer was indeed clear, simple, and straightforward, there is some difficulty in justifying assigning to it the fourth of the epithets you applied to the statement, inasmuch as the precise correlation between the information you communicated, and the facts insofar as they can be determined and demonstrated is such as to cause epistemological problems, of sufficient magnitude as to lay upon the logical and semantic resources of the English language a heavier burden than they can reasonably be expected to bear. Jim Hacker: Epistemological, what are you talking about? Sir Humphrey: You told a lie.


Found In Translation… :Found In Translation… Not handing the answer to the student, but making knowledge accessible Provide a step-up to teach students how to think and assess for themselves Re-presenting information according to “Plain English” principles


“Plainspeak”:aka Plain English :“Plainspeak”:aka Plain English Never use a metaphor, simile, or other figure of speech which you are used to seeing in print. Never use a long word where a short one will do. If it is possible to cut a word out, always cut it out. Never use the passive where you can use the active. Never use a foreign phrase, a scientific word, or a jargon word if you can think of an everyday English equivalent. Break any of these rules sooner than say anything outright barbarous. Source: "Politics and the English Language," in The Penguin Essays of George Orwell 1946, Penguin Classics, England, 2000 cited on http://www.mtholyoke.edu/acad/intrel/orwell46.htm , accessed 25th November 2008


“Plainspeak”:aka Plain English :“Plainspeak”:aka Plain English How to apply this to re-presenting information in a more accessible way: Providing the Hand-Up (active interventions teachers initiate) Closing the Deal (passive roles students can participate in)


(a). Providing the hand up…. :(a). Providing the hand up…. Readability Index (fog index) Fog indexes measure the complexity of writing samples, and often provide a means of calculating the reading or educational level required to understand a particular passage. Source: "The Technique of Clear Writing," revised edition New York: McGraw-Hill Book Company, 1973. Gunning-Mueller Clear Writing Institute Inc., Santa Barbara, http://extension.missouri.edu/explore/comm/cm0201.htm accessed November 25, 2008


(a). Providing the hand up…. :(a). Providing the hand up…. Divide the total number of words in a paragraph (e.g. 100) by the number of sentences (=the average sentence length) Count the number of words with three or more syllables in the sample. Don't count words (a) that are capitalized; (b) that are combinations of short, easy words (such as "bookkeeper" or "butterfly"); (c) that are verb forms made into three syllables by adding -ed or -es (such as "created" or "trespasses") Divide the total of such words by the number of words in your sample: for example, 15 long words divided by a 100 word-sample long gives you 15% hard words in the passage Add the sentence length and percentage of hard words Multiply this total by 0.4. The answer corresponds to the years of education needed to easily understand the piece of writing. Anything over 17, call "17-plus," meaning above the level of a college graduate. Exercise: working out the readability of Document 1 (over the page)


(a). Providing a hand-up… :(a). Providing a hand-up… Case-study: Document 3 (over the page) Task (5 mins): simplify this text so it can be understood; or create a glossary of terms for difficult language


Sample 1: Glossary (for attachment to test) :Sample 1: Glossary (for attachment to test) Conquered – to win Property – land Bondsmen – the people you employ Land dues – taxes on the land Possessions – things you own District – small area Earl – high official chosen by the king Justice - fairness Table – home Vassals – military leaders Estate – land including any buildings on it Marks – money Bestowed – given Bound – expected to Men-at-arms – soldiers Burden – things you have to do for someone else Service - work


Sample 2: Plain English Text(for attachment to test) :Sample 2: Plain English Text(for attachment to test) King Harald made a law, that all of the lands he won should belong to him, and that all of his workers should pay him land taxes. In every district, he employed an earl to judge, and to collect land taxes and fines. For this each earl received 30% of the taxes and fines as an income. Each earl had four or more vassals. Each earl earnt twenty marks income a year, but had to support twenty soldiers (the earl paid 60 men), from their own expenses. The king had increased the land taxes so much, that each of his earls had greater power and income than any kings before this time. In Throndhjem, many great men joined the king, to work for him.


(a). Providing the hand up :(a). Providing the hand up Create Multiple Choice Questions (sample questions from test) Spot the 5 errors (or room for improvement)


Chapter 9 Multiple Choice Test RevisedFeudal Europe (1 mark per correct question) :Chapter 9 Multiple Choice Test RevisedFeudal Europe (1 mark per correct question) Geographically…. A high desert covers central Europe The Mediterranean Sea borders Europe to the north Europe has many rivers and lakes High mountains form Europe’s western border After Rome collapsed, the system that still continued was.… Charlemagne’s empire The Christian Church The Frankish kingdom The manor system After Charlemagne’s death…. His nephews ruled together His grandsons extended the empire The Romans re-conquered the land The kingdom grew weak Feudalism is based on…. An agreement between serfs and lords The power of the emperor An agreement between lords and vassals The rule of church leaders Serfs…. Were bound to the soil Owned the manor lands Lived in monasteries Led the lord’s armies Lords relied on knights to…. Trade their goods on the market Do their fighting Run their manors Work the land Around 1000 A.D., towns began to grow because: Guilds became weak Powerful lords brought peace and stability Knights became more powerful Serfs were no longer needed to farm the land Japanese and European feudal systems: Both had a strong village network Were both been brought to power by guilds Did not have a strong central government Were brought to power by religious leaders The Samurai warriors followed…. Christianity and Judaism Buddhism and Christianity Shintoism and Christianity Shintoism and Buddhism An early leader of the Franks was…. Charlemagne Grendel Clovis Matuso Basho


(b). Closing the deal :(b). Closing the deal Use a bottom-up approach Explain what students can do to show they are responding to interventions Outline expectations for assessments (6 ‘survival’ skills for non-native English speakers)


(b). Closing the deal6 ‘Survival’ Test Skills for Students  :(b). Closing the deal6 ‘Survival’ Test Skills for Students  Read the instructions Attempt all questions Interpret the weighting of the marks for each question and respond accordingly Go with the gut feeling Identify unknown terminology (try to work out from context, or other use) Understand the multiple choice process (process of elimination, prioritisE the best answer)


Conclusion: Found in Translation!Suggestions for Making it Clearer :Conclusion: Found in Translation!Suggestions for Making it Clearer Find a method to reach a mutual understanding of words, text and instruction Work to be “On the same page” Do the same thing a different way Provide students with the skills so that they can interpret, think and express thoughts independently


Source: Larson, G. The Far Side Gallery, Warner Books, London, 1998 :Source: Larson, G. The Far Side Gallery, Warner Books, London, 1998


And absolutely, irrevocably and finally… :And absolutely, irrevocably and finally… References Nation, P. “Learning Vocabulary in Another Language: A test of teachers' knowledge”, University Of Wellington, New Zealand, 2007 http://www.victoria.ac.nz/lals/staff/paul-nation/vocrefs/testa.aspx Definitions Epistemology [Greek epistemē knowledge + logy study of] n. branch of philosophy that analyses the origin, nature, methods, and validity of human knowledge.


Acknowledgements :Acknowledgements I would like to express a big Thankyou to Terry Burns who suggested this topic in the first place, and helped us focus our presentation in a practical way. Also a big THANKYOU to Gordon Blackie for lending his British accent to the re-enactment of Sir Humphrey, as well as his rendition of the passages of texts. I think everybody found it humorous and interesting! Thankyou to Kostya Osadchuk who provided much needed IT assistance up to and during the presentation. And lastly, a big Thankyou to the two Olyas (Petryna and Golitsyna) for helping hand out the Anti-evil eye stickers (very Feudal Europe!) for respondents .