english placement test on moodle

English Placement Test on Moodle:

English Placement Test on Moodle Tetsuo Kimura Niigata Seiryo University Moodle Moot Hakodate 20 10

Topics:

Topics Backgrounds Why placement tests on Moodle? Moodle-based placement test 2008/2009 What types of questions? Reliability and validity How to develop good tests Item analysis Fit analysis Furthermore: Sharing item banks & equating Moodle-based computer adaptive test

Backgrounds:

Backgrounds Why placement? High diversity in the ability of incoming students Need to stream them into levels Why on Moodle? Easy (?) to make Easy to analyze & revise A good precedent: Hinkelman & Grose (2004) “Placement testing and audio quiz-making with open source software. ”

Moodle-based placement test 2008/2009:

Moodle-based placement test 2008/2009 What types of questions? STEP Eiken 3 rd ~ Pre-1 st Pre-1 st 2 nd Pre-2 nd 3rd Vocabulary & grammar ( Vgm ) 32 2 10 13 7 Listening with dialogue ( Dlg ) 13 0 7 2 4 Listening with monologue ( Mlg ) 19 --- 7 5 7

Vocabulary & grammar (Vgm):

Vocabulary & grammar ( Vgm )

Listening with dialogue (Dlg):

Listening with dialogue ( Dlg )

Listening with monologue (Mlg):

Listening with monologue ( Mlg )

Moodle-based placement test 2008/2009:

Moodle-based placement test 2008/2009 Reliability ( KR-20, Cronbach's alpha ) 2008 2009a 2009b Vocabulary & grammar ( Vgm ) ( i =32) 0.863 (n=193) 0.806 (n=125) 0.903 (n=255) Listening with dialogue ( Dlg ) ( i =13) 0.706 (n=142) 0.607 (n=125) 0.773 (n=253) Listening with monologue ( Mlg ) ( i =19) 0.780 (n=112) 0.704 (n=125) 0.824 (n=236)

Moodle-based placement test 2008:

Moodle-based placement test 2008 Validity r s =.80 r s =.89 n = 55 n = 13

Moodle-based placement test 2008/2009:

Moodle-based placement test 2008/2009 Demonstration http://tetsuo.moodlehub.com Demo student Username: student01 ~ student50 Password : student01 ~ student50 Demo non-editing teacher Username: teacher01 ~ teacher24 Password : jaltcall

How to develop good tests:

How to develop good tests Prepare Qs & sound files MP3 MP3 Divide & Rename Upload Moodle File Folder MP3 M3 Audacity XLS GIFT MCM Import Question bank Multiple Choice Maker

How to develop good tests:

How to develop good tests Pretest to Select Good Items IRT ： 1PLM Fit Index ( Z L value ) Item Analysis & Selection Placement Test / Item Bank CTT ： Point Biserial Correlation Coefficient NTT ： Item Reference Profile Misfit Item ： P.BIS ＜ 0.25 Misfit Person ： Z L ＜－ 1.96

How to develop good tests:

How to develop good tests Person Item Pre-1 st 2 nd Pre-2 nd 3rd Vgm 222 80 25 20 20 15 Dlg 157 47 12 15 10 10 Mlg 119 35 --- 15 10 10 Person Item Pre-1 st 2 nd Pre-2 nd 3rd Vgm 193 32 2 10 13 7 Dlg 142 13 0 7 2 4 Mlg 112 19 --- 7 5 7 misfit elimination

How to Analyze a Quiz on Moodle :

How to Analyze a Quiz on Moodle Within Moodle: Standard Within Moodle: TDAP Block Classical Test Theory R count & R % Difficulty :Facility Index (% Correct) Standard Deviation Discrimination Index Discrimination Coefficient Classical Test Theory Difficulty :Facility Index (% Correct) Discrimination Index Discrimination Coefficient etc. Appropriatness indexes : AENO, ADIF, etc. Reliability : Cronbach‘s α, SEM, etc. Item Response Theory: Rasch Model - PROX Item & Person Difficulty Calibrations Analysis of Fit Save the result to the item bank Make a quiz choosing questions form the item bank, referring to the item traits and IIF & TIF. Simulate a result of the quiz with a hypothetical group of examinees The Other Way Export & Analyze Data Outside of Moodle Download in Excel / Text / ODS format Software for IRT: Winstep Easy Estimation / Easy EstTheta neutet exametrica

Furthermore:

Furthermore Sharing item banks & equating Calibrated items Vgm 32 Dlg & Mlg 32 New items Vgm 96 Dlg & Mlg 96 Calibrated items Vgm 128 － α Dlg & Mlg 128 － α Calibrated items Vgm 128 － α Dlg & Mlg 128 － α New items Vgm 96 Dlg & Mlg 96 Calibrated items Vgm 224 － α － β Dlg & Mlg 224 － α － β 著作権

Furthermore:

Furthermore Sharing item banks & equating Number of item for new tests Test A Test B Test C Test D Calibrated items Vgm 16 16 16 16 Dlg 7 6 6 7 Mlg 9 10 10 9 New items Vgm 16 16 16 16 Dlg 9 10 10 9 Mlg 7 6 6 7 Total Vgm 32 32 32 32 Dlg 16 16 16 16 Mlg 16 16 16 16 Total 64 64 64 64

Equating Design:

Equating Design Items Test A New Items Anchors Anchors Anchors Anchors Test B New Items Anchors Anchors Persons Anchors Anchors Test C New Items Anchors Anchors Anchors Anchors Test D New Items

Furthermore:

Furthermore Moodle-based computer adaptive test CAT module (under development) Simulation Testlet ( Polytomous ) Equating Fit analysis Multi-platform of calibration Rasch model with PROX Rasch model with Bayesian model Latent Rank Theory (Neural Test Theory & Graded NTT) Collaborative R&D with e-learning service

Thank you for listening Tetsuo KIMURA:

Thank you for listening Tetsuo KIMURA If you are interested in English placement test on moodle, or if you have any question, please contact me at kimura@n-seiryo.ac.jp

How to Get Multiple Choice Maker:

How to Get Multiple Choice Maker https://e-learning.ac/moodle-resources/ Need to Register With Academic Domain e-mail e.g. kimura@n-seiryo. ac .jp HERE ! Tools for Moodle

How to Get TDAP block:

How to Get TDAP block https://e-learning.ac/moodle-resources/ Need to Register With Academic Domain e-mail e.g. kimura@n-seiryo. ac .jp HERE ! Plug in – Block

How to Get CAT module:

How to Get CAT module https://e-learning.ac/moodle-resources/ Need to Register With Academic Domain e-mail e.g. kimura@n-seiryo. ac .jp HERE ! Plug in - Module

Slide 23:

Shojima’s Web Site http://www.rd.dnc.ac.jp/~shojima/ntt/ You can download exametirca from here

Reference:

Reference Bond, T. G. & Fox, C.M. (2007). Applying the Rasch Model: Fundamental mesurement in the human sciences. 2 nd ed. Lawrence Erlbaum Associates, Inc. Hinkelman , D., & Grose , T. (2004). Placement testing and audio quiz-making with open source software. Proceedings of CLaSIC 2004, 972-981. Hinkelman , D., & Johnson, A. (2008, November). Digital repository for teaching teams. Paper presented at the 34 th JALT International Conference, Tokyo, Japan. Kimura, T. (2008). Testing Using Moodle and Its Data Analysis. 「 Moodle を使ったテスティングとそのデータ分析」 『 英語教育・英語学習研究現場型リサーチと実践へのアプローチ ― 金谷憲先生還暦記念論文集 ―』 桐原書店 . pp.247-258 ． Kimura, T. (2008, June). Using Moodle’s quiz module in blended learning in freshmen’s English. 「大学１年生の英語授業における Moodle の小テスト・モジュールの活用」 ＬＥＴ関東支部 120 回 (2008 年度 ) 研究大会 Kimura, T. (2008, August). Construction of a Moodle -based Placement Test and Possibility of a Moodle -based Computer Adaptive Test. 「 Moodle を利用したテスト項目分析とアダプティブ・テスト開発の可能性」全国英語教育学会 第 34 回 東京研究大会 Kimura, T. (2008, September). A pilot study to develop a moodle -based in-house English placement test for Japanese university students from an IRT and an NTT perspective. The 12th Annual Conference of the Japan Language Testing Association. 「習熟度別クラス編成のための英語基礎力判定標準化テスト作成の試み」日本言語テスト学会第１２回全国大会 Proceedings Presentation Kimura, T. (2008, December) . Construction and Evaluation of a Moodle -based In-house English Placement Test-- From an IRT and an NTT Perspective. The 29th JLTA Research Meeting in Okinawa. 「 Moodle による英語プレイスメントテストの作成と評価 --IRT と NTT の視点から」日本言語テスト学会第 29 回 研究例会 in 沖縄 Presentation

Slide 25:

Kimura, T. (2009a). Construction and Evaluation of an In-house English Placement Test From a Neural Test Theory Perspective. KATE 23. 「ニューラルテスト理論による英語プレイスメントテストの作成と評価」 関東甲信越英語教育学会研究紀要 23 ． Kimura, T. (2009b). Construction of a Moodle -based Placement Test and Possibility of a Moodle -based Computer Adaptive Test. ARELE 20. 全国英語教育学会研究紀要 20 ． Kimura, T. (2009, September). Neural test theory in practice: A two-step analysis of an English placement test. The 7th Annual Meeting of the Japan Association for Research on Testing (Nagoya, Japan) Session “Neural test theory” Proceedings Presentation Kimura, T. (2009, September). Case studies of stage assessment in language tests: Analyses of an entrance examination and a placement test data. The 13th Annual Conference of the Japan Language Testing Association (Sapporo, Japan) Proceedings Presentation Kimura, T. (200,9 September). 「 Moodle による英語プレイスメントテストのためのアイテムバンク構築」日本教育工学会 25 回全国大会 Kimura, T. & Akiyama, M. (2009). Reinforcing Development of a Moodle-based English Test with Multiple Choice Maker, TDAP block, and CAT module . LET 46. 「 Moodle 上での英語テスト開発を支援するツール： Multiple Choice Maker, TDAP ブロック , CAT モジュール」外国語教育メディア学会紀要 46. Ohtomo , K., & Nakamura, Y. (1996). Test Data Analysis Program (TDAP) Ver. 1.0 [MS-DOS, N88-BASIC, Computer software]. 大友賢二 ,『 項目応答理論入門 』, 東京：大修館書店 . Ohtomo , K., Nakamura, Y., & Akiyama, M. (2002). Test Data Analysis Program (TDAP) Ver. 2.0 [Windows, Computer software]. 大友賢二（監修）・中村洋一（著） ,『 テストで言語能力は測れるのか～言語テストデータ分析入門～ 』, 東京：桐原書店 .

Reference:

Reference Rice, W. H. (2007). Moodle Teaching Techniques: Creative Ways to Use Moodle for Constructing Online Learning Solutions. Birmingham: Packt Publishing Ltd. Rice, W. H. (2008). Moodle 1.9 E-learning Course Development: A Complete Guide to Successful Learning Using Moodle 1.9. Birmingham: Packt Publishing Ltd. Shojima, K. (2008). Neural test theory: A latent rank theory for analyzing test data. DNC Research Note, 08-01. Shojima, K. (July 2009) International Meeting of the Psychometric Society 2009 (Univ. of Cambridge, UK) Neural test theory model for graded response data. P.112. Presentation