GIREP6

PHYSAP - PHYS 21 student assessment programme: 

PHYSAP - PHYS 21 student assessment programme Toward a description of physics students’ modelling competency

The PHYS 21 project: 

The PHYS 21 project PHYS 21 employs empirical-mathematical modelling in upper secondary school and may contribute to develop students’ competencies and literacy in the field of physical science: The use of, and interchange between, multiple representations of physical phenomena Skills of scientific reasoning Fruitful learning strategies for gaining understanding in physics; The nature of science (NOS) The role and purpose of experiments in physical science The relationship between mathematics and physics

Purpose: 

Purpose My main task is to ”describe” upper secondary physics students’ modelling competency Methods: classroom observations; focus group interviews; questionnaire; and, achievement test (assess abilities to interchange between multiple representations and reason scientifically)

Slide4: 

Developing a test to assess modelling competency: Developed ca. 100 items (questions) Trialled items to develop a reliable test Items’ psychometrics (p-value, correlation) Main test January 2006 29 items distributed over 7 units

Modelling competency was assessed as ability to interchange between representations and reason scientifically: 

Modelling competency was assessed as ability to interchange between representations and reason scientifically Model: Phenomenon: Learning the model: Mastering the model: Modelling the phenomenon: Interchange between and simultaneous application of the representations constituting the scientific model Interchanging between forms of representation

Distributions of items (%) and formats on test: 

Distributions of items (%) and formats on test

Example: 

Example Unit ”Sea level” Some students wanted to examine how the melting of ice at The South Pole and in the areas around The North Pole influences the sea level. The pupils filled a glass (glass 1) with water. After they had put two ice cubes into the glass, the water level was 5 cm. The students put a stone into an identical glass (glass 2). They put two ice cubes on the stone and filled up with water so the water level was 5 cm there as well. At the North Pole there is no land under the ice, but there is at The South Pole. The stone represents these territories. Assume that the ice is melting with a constant rate. Which mathematical expression describes the water level (y) in glass 1 and glass 2 while the ice melts? A Glass 1: y=b, glass 2: y=ax+b B Glass 1: y=ax+b, glass 2: y=b C Glass 1: y=b, glass 2: y=ax D Glass 1: y=ax, glass 2: y=b

Sea level question 1 analysis: 

Interchange between experimental and mathematical representation Describe physical relationships using a general mathematical expression: reasoning process ”generalization” Distribution of answers: Point biserial = .330** Sea level question 1 analysis

Sea level question 2 with analysis: 

Interchange between conceptual and mathematical representation Analyze problem (reasoning process) Distribution of answers: Point biserial = .338** Sea level question 2 with analysis What does x in the equations above refer to? A The melting speed of the ice B The original water level in the glass C The Temperature of the water in the glass D The time from the ice began to melt

Sea level question 3: 

Sea level question 3 Some students filled cold water (0 0C) in a bottle with a straw through the cap. They put the bottle in a container with warmer water (20 0C). The lower line on the picture shows the initial water level; the upper line shows the final level. The figure below shows the density of water as a function of temperature.

Sea level question 3 analysis: 

Sea level question 3 analysis Interchange between experimental and graphical representation Select from alternative solutions: reasoning process “make decision” Distribution of answers: Point biserial = .410**

Sea level question 4 with analysis: 

Write down how the sea level can change if the average temperature on the Earth increases. Use the information from the experiments with glass 1 and 2 and the flask with the straw: ___________________ ___________________ Sea level question 4 with analysis Interchange between experimental and conceptual representation Make predictions about effects of changes in physical systems: reasoning process ”predict; conclude; communicate”

Proficiency levels: 

Proficiency levels Score on the test is interpreted to represent degree of competency to model physical phenomena. Four proficiency levels were empirically defined on the basis of student performance. The divisions between the proficiency levels were chosen to be as close as possible to the following distribution: 10%; 25%; 30%; 25%; 10% In practice level 2 (intermediate) accounted for 31% of the students. Level 3 (high) and 1 (low) accounted for 24% each. Level 4 (advanced) accounted for 8% “Below level 1” accounted for 13%

Anchoring at proficiency levels: 

Anchoring at proficiency levels Each item was assigned a p-value (percentage correct response) at each of the four proficiency levels. If at least half of the students proficient at a certain level complete an open constructed item, the item is said to “anchor” at that level. Pure guessing makes it 25% probable that a student chooses the key response on a 4-choice MC item. A 4-choice item therefore anchors at a proficiency level if 62.5% (50 + 50/4 = 62.5) of the students tick off the key response Items anchoring at the same level thus lay in the same difficulty interval. The proficiency associated with that level is described as a summary of what these items require.

Anchoring at proficiency levels: 

Anchoring at proficiency levels Sea level question 1 (melting ice cubes in glasses) were quite easy and anchored at proficiency level 2:

Description of proficiency levels: 

Description of proficiency levels

Results: 

Results PHYS 21 students report more frequent use of interchanges between representations during physics lessons Students who are conscious their use of learning strategies in physics are more able to recognise use of interchanges between representations during physics lessons than other students The learning strategy “elaboration” is positively associated with performance on the PHYS 21 achievement test Knowledge about the nature of science (NOS) is also positively associated with performance on the test Empirical-mathematical modelling activities (PHYS 21) may help disadvantaged students overcome the critical level from being “below level 1” to being proficient at level 1 Analysis reveals there being more males among the better performers (proficient at level 3 and 4) and more females among the low performers (proficient at level 1).

Summary: 

Summary Made a reliable test from a test-theoretical point of view Makes it possible to describe physics students modeling competency as operationalised by the PHYSAP instrument Both knowledge about NOS, possessing elaboration strategies, and being conscious the use of interchanges between representations correlate with test score Indicate that aspects of NOS, incitements to use fruitful learning strategies and focus on multiple representations are valuable components of physics teaching Which teaching strategies work best within these components is not clear The empirical-mathematical modeling activities in PHYS 21 were not enough to produce significantly different results on the paper and pencil test in our sample