NCRT

NRT and CRT:

NRT and CRT Sittie Jalilah T. Abdul- Jalil

Definitions::

Definitions: Assessment -- The process of measuring something with the purpose of assigning a numerical value. Scoring -- The procedure of assigning a numerical value to assessment task. Evaluation -- The process of determining the worth of something in relation to established benchmarks using assessment information.

Evaluation Types::

Evaluation Types: Criterion-referenced evaluation -- student performance is assessed against a set of predetermined standards Norm-referenced evaluation -- student performance is assessed relative to the other students The “curve” -- sometimes a combination of criterion- and norm-referenced processes

Types of Assessments ..Norm-referenced versus criterion-referenced tests ..Traditional, selected- response versus constructed-response (“performance”) tests:

Types of Assessments ..Norm-referenced versus criterion-referenced tests ..Traditional, selected- response versus constructed-response (“performance”) tests

Slide 5:

A norm-referenced test / NRT is a type of test , assessment , or evaluation which yields an estimate of the position of the tested individual in a predefined population, with respect to the trait being measured. This estimate is derived from the analysis of test scores and possibly other relevant data from a sample drawn from the population. [ The term normative assessment refers to the process of comparing one test-taker to his or her peers. Norm-referenced assessment can be contrasted with criterion-referenced assessment and ipsative assessment .

Slide 6:

Criterion-referenced tests are related directly to instructional ..objectives, are based on task analysis, and are designed ..to measure changes in successive performances of an individual. Criterion-referenced tests, therefore, are sensitive to and can be used to measure the effects of instruction.

Assessment Purposes:

Assessment Purposes Norm-Referenced Measures (NRM) Most appropriate when one wishes to make comparisons across large numbers of students or important decisions regarding student placement and advancement. Norm-referenced measures are designed to compare students (i.e., disperse average student scores along a bell curve, with some students performing very well, most performing average, and a few performing poorly).

Slide 8:

Criterion-Referenced Measures (CRM) Most appropriate for quickly assessing what concepts and skills students have learned from a segment of instruction. Criterion-referenced assessments measure how well a student performs against an objective or criterion rather than another student. Criterion-referenced classrooms are mastery-oriented, informing all students of the expected standard and teaching them to succeed on related outcome measures. The "bell curve" in this case is skewed heavily to the right, as all students are expected to succeed. Criterion-referenced assessments help to eliminate competition and may improve cooperation.

Slide 9:

determine individual performance in comparison to others; standardized, comparisons among people items produce great variance in scores, perhaps with less than 50% scoring correctly item analysis used to select those items that were answered correctly by those scoring high on a test but incorrectly by those scoring low on a test (a positively discriminating item) it is inappropriate to use NRMs to determine the effectiveness of educational programs and to provide diagnostic information for individual students; items cover a broad range of content and often represent a mismatch between what is taught locally and what is taught in other states

Slide 10:

determine individual performance in comparison to some standard or criterion items based on standards given to students (i.e., objectives); most students should answer correctly discrimination is irrelevant and should not take place; discrimination may point to errors in instruction

Slide 11:

A Sample NRM: The Graduate Record Exam (GRE) The GRE is taken by college students wishing to enter graduate schools. The test items are included in an actual exam after they are analyzed and determined to discriminate appropriately. The following quote describes the "test development

Slide 12:

A Sample CRM: The Performance Assessment Most appropriate for determining the progress of smaller numbers of students on higher-order learning tasks. For performance assessments, students are tasked with creating or presenting a unique product or solution (paper, design, oral

Norm-referenced Evaluation:

Norm-referenced Evaluation Based upon the assumption of a standard normal (Gaussian) distribution with n > 30. Employs the z score: A = top 10% ( z > +1.28) B = next 20% (+0.53 < z < +1.28) C = central 40% (-0.53 < z < +0.53) D = next 20% (-1.28 < z < -0.53) F = bottom 10% ( z < -1.28)

Norm-referenced Evaluation:

Norm-referenced Evaluation Pros: Ensures a “spread” between top and bottom of the class for clear grade setting. Shows student performance relative to group. Con: In a group with great performance, some will be ensured an “F.” Cons: Top and bottom performances can sometimes be very close. Dispenses with absolute criteria for performance. Being above average does not necessarily imply “A” performance

Criterion-Referenced Eval’s:

Criterion-Referenced Eval’s Based on a predetermined set of criteria. For instance, 90% and up = A 80% to 89.99% = B 70% to 79.99% = C 60% to 69.99% = D 59.99% and below = F

Criterion-Referenced Eval’s:

Criterion-Referenced Eval’s Pros: Sets minimum performance expectations. Demonstrate what students can and cannot do in relation to important content-area standards ( e.g , ILS). Cons: Some times it’s hard to know just where to set boundary conditions. Lack of comparison data with other students and/or schools

Norm and Criterion Compared:

Norm and Criterion Compared Norm-Referenced: Ensures a competitive classroom atmosphere Assumes a standard normal distribution Small-group statistics a problem Assumes “this” class like all others Criterion-Referenced: Allows for a cooperative classroom atmosphere No assumptions about form of distribution Small-group statistics not a problem Difficult to know just where to set criteria

Slide 19:

Dimension Criterion-Referenced Tests Norm-Referenced Tests Purpose To determine whether each student has achieved specific skills or concepts. To find out how much students know before instruction begins and after it has finished To rank each student with respect to the achievement of others in broad areas of knowledge. To discriminate between high and low achievers. Content Measures specific skills which make up a designated curriculum. These skills are identified by teachers and curriculum experts. Each skill is expressed as an instructional objective. Measures broad skill areas sampled from a variety of textbooks, syllabi, and the judgments of curriculum experts.

Slide 20:

Item Characteristics Each skill is tested by at least four items in order to obtain an adequate sample of student performance and to minimize the effect of guessing. The items which test any given skill are parallel in difficulty. Each skill is usually tested by less than four items. Items vary in difficulty. Items are selected that discriminate between high and low achievers. Score Interpretation Each individual is compared with a preset standard for acceptable achievement. The performance of other examinees is irrelevant. A student's score is usually expressed as a percentage. Student achievement is reported for individual skills. Each individual is compared with other examinees and assigned a score--usually expressed as a percentile, a grade equivalent score, or a stanine . Student achievement is reported  for broad skill areas, although some norm-referenced tests do report student achievement for individual skills.

Slide 22:

The End