IndyAssessPresentati on 11 04 last upade 10 27

Assessment of Technical and General Education Programs at a Two-year Institution Dr. Paulette PopovichAssociate Dean of InstructionDr. Tim VierhellerProfessor of PhysicsThe University of Akron Wayne CollegeOrrville, Ohio: 

Assessment of Technical and General Education Programs at a Two-year Institution Dr. Paulette Popovich Associate Dean of Instruction Dr. Tim Vierheller Professor of Physics The University of Akron Wayne College Orrville, Ohio

Creating and Maintaining and On-going, Faculty-Driven Process: 

Creating and Maintaining and On-going, Faculty-Driven Process Overview of the development of our assessment plan Faculty workshops Assessment committee Coordinators’ role Monitoring collection cycle and record management Assessment-based Program Review Model and preliminary evaluation

Merit and Assessment: Full-time and Adjunct Faculty Issues: 

Merit and Assessment: Full-time and Adjunct Faculty Issues Professional responsibility described on annual evaluation document “Full-Time Faculty Professional Responsibilities Demonstrated commitment to and participation in required faculty initiatives including instructional assessment…” Narrative description of assessment outcomes “Teaching Effectiveness Appropriate instructional assessment activities, analysis, and course or teaching modifications. Please provide a brief narrative (4-5 sentences) describing changes you have or plan to make as a result of assessment results.”

Implementation - Assessment Forms: 

Implementation - Assessment Forms “Part A” College Learning Outcomes Area /Program Learning Outcomes Course Outcomes Assessment Course Objectives Methods Criteria1 1Diamond, R. M. Designing & Assessing Courses & Curricula; Jossey-Bass: San Francisco, 1998, 49.

Implementation - Assessment Forms: 

Implementation - Assessment Forms “Part B” Results Summary and analysis of data Outcomes Achieved Changes based on assessment results1 Description of instructional changes Reflective Critique2 1Ratcliff,Lubinescu,Gaffney How Accreditation Influences Assessment, Jossey-Bass: San Francisco, 2001, 90. 2Glassick, Huber, Maeroff Scholarship Assessed; Jossey-Bass: San Francisco, 1997, 33.

Slide6: 

Part A

Slide7: 

Part B

OPPORTUNITIES FOR IMPROVEMENT IN ASSESSMENT IMPLEMENTATION : 

OPPORTUNITIES FOR IMPROVEMENT IN ASSESSMENT IMPLEMENTATION Simplify - Don’t try to measure more learning outcomes than your instrument ( or you!) can handle! Follow Through - If your form A lists an outcome you hope to measure, then make sure you discuss the data you collected for that outcome on side B. Forms are submitted and reviewed regularly by coordinators.

OPPORTUNITIES FOR IMPROVEMENT IN ASSESSMENT IMPLEMENTATION : 

OPPORTUNITIES FOR IMPROVEMENT IN ASSESSMENT IMPLEMENTATION Justify - When you describe the changes you are planning, explain how and why those changes are due to your assessment activities. Linking - Be sure to link all the learning outcomes on your Form A: course outcomes to program outcomes to college outcomes. Note: Linking is key to Program Assessment and Review

ASSESSMENT INSTRUMENT DESIGNS - 1 : 

ASSESSMENT INSTRUMENT DESIGNS - 1 PRE/POST WITH FINAL EXAM ACTING AS POSTTEST Often included in Math courses PRETEST WITH QUESTIONS ASKED THROUGHOUT THE TERM  Physics ANNOTATED PORTFOLIOS1 Office Health Care and Social Services Programs 1Angelo and Cross Classroom Assessment Techniques; Jossey-Bass: San Francisco, 1993, 208.

ASSESSMENT INSTRUMENT DESIGNS - 2: 

ASSESSMENT INSTRUMENT DESIGNS - 2 IN CLASS EXERCISES  SKILLS ASSESSMENT ATTITUDE AND BEHAVIORAL ASSESSMENT USE OF INTERACTIVE TECHNOLOGY  Example: Biology Professor Emily Rock SELF-ASSESSMENT TECHNIQUES1 1 Oscarsson, M. (1998). “Learner Self-Assessment of Language Skills”. IATEFL TEA SIG Newsletter, Nov. 1998.

NATIONAL INSTRUMENTS : 

NATIONAL INSTRUMENTS PROS THINKING ALREADY DONE FOR YOU QUESTIONS MAY BE SIMILAR TO THOSE ON CERTIFICATION EXAMS MANY COME WITH DATA PACKAGE AND ANALYSIS ALLOWS FOR COMPARISON OF YOUR STUDENTS TO THOSE AT SIMILAR SCHOOLS ESTABLISHED VALIDITY

NATIONAL INSTRUMENTS: 

NATIONAL INSTRUMENTS CONS COST LENGTH LACK OF FLEXIBILITY  QUESTIONS NOT APPROPRIATE FOR YOUR CONTENT WHAT IS A SUCCESSFUL STUDENT?  WHAT CRITERIA MUST BE MET IN ORDER TO SUCCESSFULLY ACHIEVE THE COURSE OUTCOMES?

IN-HOUSE-CREATED INSTRUMENTS: 

IN-HOUSE-CREATED INSTRUMENTS PROS  CONTROL LENGTH  CONTROL WHEN GIVEN  LOWER COSTS  ADJUST CONTENT TO MEET YOUR NEEDS

IN-HOUSE-CREATED INSTRUMENTS: 

IN-HOUSE-CREATED INSTRUMENTS CONS RELIABILTY & VALIDITY  TIME TO DEVELOP 

Workshops Conducted on Reliability and Validity1: 

Workshops Conducted on Reliability and Validity1 Reliability Consistency, stability of scores Error Sources – luck, ambiguity, etc. More easily achieved with objective tests Valuable Resource: Professor Bill Trochim’s (Cornell) website: trochim.human.cornell.edu 1Gay Felix, The University of Akron Wayne College

Workshops Conducted on Reliability and Validity: 

Workshops Conducted on Reliability and Validity Validity Extent to which test measures what it is intended to measure IQ test should measure intellectual potential not achievement or experience Again more easily achieved with objective tests

Levels of Reflective Judgment - Dr. S. Meehan (Wayne College): 

Levels of Reflective Judgment - Dr. S. Meehan (Wayne College) RIGHT ANSWERS EXIST UNCERTAINTY EXISTS THINKING IS IMPORTANT THINKING IS IMPORTANT FOR LIFE THINKING FOR LIFE-LONG LEARNING (Attempts to balance multiple values, beliefs and ideas) 1. King, Patricia M. & Kitchener, Karen S. (1994). Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey-Bass. 2. Kitchener, Karen S.; Lynch, Cindy L.; Fischer, Kurt W.; & Wood, Phillip K. (1993). Developmental range of reflective judgment: Effect of contextual support and practice on developmental stage. Developmental Psychology, 29, 893-906.

Serving the Student : 

Serving the Student Statements explaining importance included on syllabi Faculty can use to create interest in subject Value-added concept

STUDENT’S OPINIONOF THEIR OWN PROGRESS : 

STUDENT’S OPINION OF THEIR OWN PROGRESS WEBSITE: Student Assessment of Learning Gains http://www.wcer.wisc.edu/salgains/instructor/

“Marketing” College and Program Learning Outcomes : 

“Marketing” College and Program Learning Outcomes Visual representations Focus groups (~40 students per year in groups of about 10) Learning outcomes for class and college? What is assessment? Name some assessment methods? What is the importance of assessment? Supportive feedback has been obtained

Wayne College Model forAssessment-Based Program Review of Technical Degree Programs: 

Wayne College Model for Assessment-Based Program Review of Technical Degree Programs

Assessment-based Program Review Model: 

Program’s Contribution to College Mission Program Learning Outcomes Embedded Assessment Assessment Methodology & Summary Assessment Results Programs Learning Outcomes Demonstrating Assessment of College-Level Learning Outcomes SWOT Required Resources Assessment-based Program Review Model

Assessment-based Program Review Model: 

Program’s Contribution to College Mission Career / Job Opportunities Program Learning Outcomes Technical, Logic, & Communication Skills Assessment Methodology & Summary Hands on to Certification Exams Assessment Results Programs Learning Outcomes Demonstrating Assessment of College-Level Learning Outcomes Establishing Links SWOT From process/data collection issues to curriculum changes Required Resources Training / $’s Assessment-based Program Review Model

Slide25: 

Critical analysis and independent thought. Problem-solving ability. Effective interpersonal communication skills. Effective use of technology. Respect for individual differences and personal rights. Responsible and effective citizenship. Wayne’s College Level Outcomes

Making Assessment of College Level Outcomes Pubic: 

Making Assessment of College Level Outcomes Pubic Reexamination of outcomes to include college-wide discussions Systematic community feedback solicited Two-year cycle exists for members of the community to comment on college’s mission, strategic objectives, and stated student outcomes

Measuring College Level Outcomes: 

Measuring College Level Outcomes Systematic approach starting with interpersonal communication skills (written and oral) Teams use rubrics for written and oral assignments (see example) using exit-level work Walvoord, Barbara E. & Anderson, Virginia. (1998). Effective grading: a tool for learning and assessment. San Francisco: Jossey-Bass.

Next Step: Performance-based Assessment in General Education: 

Next Step: Performance-based Assessment in General Education Gen Ed Outcomes defined by parent institution Gen Ed Advisory Committee at parent Institution continues to make progress Wayne College goal is consistency with parent institution & implementing changes based on assessment results Wayne College GE assessments currently course based (See example Form A) This approach to be used for Assoc. of Arts/Science degrees

Thank you!: 

Thank you! Dr. Paulette Popovich popovich@uakron.edu Dr. Tim Vierheller tv@uakron.edu www.wayne.uakron.edu from index go to Faculty Presentations