Measuring Change In Students

Measuring Change In Students: 

Measuring Change In Students What to measure? and How to measure it? Scott W. Brown, Ph.D. University of Connecticut

Why Measure Anything?: 

Why Measure Anything? To demonstrate change / impact. To make changes in things that are not working well. To document effective practices. To enable transfer to other settings. To demonstrate value of products / activities.

How Do I Measure Change?: 

How Do I Measure Change? Determine what impact your project should have. Who? And there can be multiple whos! When? Under what conditions? Determine what impacts are of value Not every one is of value Who values what impacts? Determine when change should occur. When should I start to see change? Be patient! Find instruments and systems to measure and record change. Develop them Use standard ones Have multiple measurements engaged so you don’t miss anything important

Do I only measure changes in students?: 

Do I only measure changes in students? Students as indicators What do students do differently now? Baseline issues – what did they do before How can you demonstrate that? How do students feel? Attitudes Self-efficacy What do students know? How do they know it? How do you know they know it?

Others who are impacted?: 

Others who are impacted? Parents What differences do they see in their children as students Users of educational technology Teachers What do teachers do differently? How do they feel about educational technology Are they using it to promote learning? If so, in what ways? If not, what are the barriers?

2 Systems of Measuring Change: 

2 Systems of Measuring Change Quantitative Measurement scales Frequency counts Time on task – using computers questionnaires Qualitative Interviews Observations Journals Use the right tool for the right job! You can use both, and both are systematic and valuable tools!

Introduction to KABs: 

Introduction to KABs Knowledge Attitudes Behaviors

KABs – topic related!: 

KABs – topic related! What do you know about it? How do you feel about it? Do you believe you can do it? (SE) Do you do the associated behavior? Recall Bloom’s Taxonomy Cognitive Affective Psychomotor

Knowledge: 

Knowledge Computers/ Internet Educational Technology Problem Solving Interests Personal Goal Decision Making

Attitudes and Self Efficacy(Bandura, 1997; sense of personal agency): 

Attitudes and Self Efficacy (Bandura, 1997; sense of personal agency) Use of Computers/Internet School in General Interests Personal Goals Use of Educational Technology Problem Solving Decision Making Working in groups

Computer / Internet Items: 

Computer / Internet Items Do they have a computer at home? Rating of computer ability. Use of Internet? If so, for what? How often?

Attitude items: 

Attitude items Attitudes about computers/Internet Attitudes about school – general Attitudes about working in groups Goals and objectives School topics Math vs. social studies, etc.

Self-Efficacy: 

Self-Efficacy Using computers Solving problems Making decisions Being successful in groups Completing school assignments Working in groups

Behaviors : 

Behaviors Homework assignment changes? How do they use the computer? Level of skill? Class participation? Group work - study groups? Computer usage – time? Solving problems?

Things you can look for indicating change: 

Things you can look for indicating change Classroom participation. Time spent on assignments? Study groups - group work? Student products Changes in teacher activities Changes in the physical classroom environment Changes in the social environment Parental involvement

An Approach: Portfolio Assessment: 

An Approach: Portfolio Assessment Moving from standard portfolio to electronic CD ROMs, discs and web-site of materials Challenges What to include? Best way to present? How to evaluate?

Looking for growth: Farming analogy: 

Looking for growth: Farming analogy KABs KABs KABs KABs KABs

Examples: 

Examples GlobalEd Project Multinational simulation for middle and high school students using the internet Examining Problem solving Negotiation skills Decision making Leadership ability Global citizenship Using KAB’s and both quantitative and qualitative assessments Multiple scales Teacher and student data every 6 months www.globaled.uconn.edu

Classroom of the Sea: 

Classroom of the Sea Studying scientific literacy and skills of deaf middle and high school students Use marine science as the base for a problem based learning environment Studying the potential impact a pipeline proposed to run from Connecticut to Long Island, NY would have on the Long Island Sound harbor seal population. Research trips on the RV Connecticut to study harbor seals, count them and examine water quality issues Assessing changes in students in the classroom and on the ship, now and in the future. Interviews, observations, scales For teachers and students

So what are the keys?: 

So what are the keys? Planning Team work Multiple methods Selecting good and important variables to measure Time To allow for growth

You are NOT ALONE: 

You are NOT ALONE You have a great deal of resources here at UIC You have each other Share ideas and methods Search the web for others measuring similar things They are out there. And you can contact me Sbrown@uconn.edu

Questions?: 

Questions?