AIR2002

State Higher Education Assessment Policies: Findings from Case Studies: 

State Higher Education Assessment Policies: Findings from Case Studies Thomas E. Perorazio John J.K. Cole The National Center for Postsecondary Improvement Project 5.1 The University of Michigan Association for Institutional Research 42nd Annual Forum Toronto, Ontario, Canada

Session Goals: 

Session Goals Discuss Policy Process for Assessment Relate Important State Experiences Share Lessons Learned from Cases Relevant to Institutions

The National Center for Postsecondary Improvement: 

The National Center for Postsecondary Improvement Main Project Web Site http://www.stanford.edu/group/ncpi/ University of Michigan - Project Five http://www.umich.edu/~ncpi/

NCPI Project Area 5.1: 

NCPI Project Area 5.1 Examine State & Regional Policies What Policies Are in Existence? Impact on Institutional Assessment Practices Impact on Teaching & Learning Utilize Policy Process Framework Analyze Relationships Among Levels State Government and Regional Accreditation Association Policies of Assessment for Student Learning

Overview of Project 5.1 Research: 

Overview of Project 5.1 Research Benchmarking Assessment Documented Existing Policies SHEAQ Survey Of SHEEO Administrators Literature Review Case Studies

Case Study Research: 

Case Study Research Examine State Policies in Detail Explore Critical Issues with State Officials Trace Evolution and Development Learn about Policy’s Impact on Institutions Infer Lessons about the Policy Process Purposes

States Selected for Study: 

States Selected for Study New York (MSACS) South Carolina (SACS) Washington (NWASC) Missouri (NCA/HLC) Florida (SACS) (with Accreditation Region)

Policy Context: 

Policy Context Historical, Political, Social, & Economic Factors Existing Policies for Accountability, Efficiency, etc. Political Climate in the State for Higher Education Previous Political Action RE: Assessment Governance Structure for Higher Education Relations & Communications between Government, SHEEO, & Institutions Budgetary & Financial Issues for Higher Education

Policy Process Framework: 

Policy Process Framework Five Stages 1.      Problem Formation Recognizing the Need for a State-level Assessment Policy 2.      Policy Formulation Development of Proposed Courses of Action 3.      Policy Adoption Development of Support for a Specific Proposal 4.      Policy Implementation Application of the Policy to the Problem 5.      Policy Evaluation Attempt to Determine the Policy’s Effectiveness

Problem Formation & Policy Origination: 

Problem Formation & Policy Origination SHEEO Seeks to Focus Goals of Institutions on State Priorities Statewide Planning Task Forces Priorities of Quality, Effectiveness, Prestige, & Efficiency Gubernatorial/Legislative Interest in Performance Desires Information on Results/Success Task Force Studies Data Generation & Collection Measures Concern about Public Perceptions of Prestige/Quality

Problem Formation (2): 

Problem Formation (2) Institutional Actions Initiatives to Engage in Assessment Procedures to Improve Program Quality Programs to Enhance Learning Public Opinion Critical for Political Will Change in Political Power

Policy Formulation: 

Policy Formulation Quality Assurance Institutional Data Generation/Collection Performance Indicators/Assessment Reports Make Information Publicly Available Increase Information to Policymakers Accountability State Planning & Coordination Institutions Meet State Goals/Targets Centralized Approach to Data Analysis Results Tied to Budgetary Decisions Options for Consideration

Policy Formulation (2): 

Policy Formulation (2) Institutional Improvement Management & Effectiveness Link Measures to Accreditation Standards Institutional Self-Evaluation Differentiated by Sector Bring Improvement Through Quality Meet Both State and Institutional Goals

Policy Adoption: 

Policy Adoption Legislative Action Mandate Authorize SHEEO & Monitor Work with Institutions for Revision SHEEO Authority Originator of Policy Monitor, Collector, & Distributor of Info Mediator between State, Institutions, & Public Four General Methods

Policy Adoption (2): 

Policy Adoption (2) Task Forces/Blue Ribbon Committees Authority: SHEEO or Legislature Business Leaders, Institutional Presidents Conduct Study/ Make Recommendations State/System Planning Process Produces Actionable Objectives Assessment In Service of Plan Goals

Policy Implementation--Mechanisms: 

Policy Implementation--Mechanisms Reporting Institutional Statistics New York, Washington, & Florida Performance Funding/Reporting South Carolina Missouri -- FFR Florida System Goals & Institutional Improvement Missouri, Washington Accountability Florida, South Carolina, Missouri

Policy Implementation (2): 

Policy Implementation (2) Decentralized State Sets Broad Parameters for Performance Institutions Develop Effectiveness Plans Measurement Defined by Institutions Institutions Report Results up to State State Makes Decisions on Aggregated Data New York & Washington

Policy Implementation (3): 

Policy Implementation (3) Centrally-Guided Prescribed State & Institutional Goals Performance Standards Less Variable Central Data Collection & Analysis Findings Utilized in Budget Decisions South Carolina & Missouri

Policy Implementation (4): 

Policy Implementation (4) Combination Approach State Expectations & Performance Guidelines Institutional Variability for Compliance Institutional Activity for Internal Improvement Florida

Data Collection: 

Data Collection Centralized Databases v. Institutional Data Levels of Aggregation Instruments/Testing Commonality Associated with Centralization Institutional Reporting To SHEEO, Legislature, Public Data is Reported Up the System Emphasis on Data & Data Systems

Data Usage for Decision Making: 

Data Usage for Decision Making Rewards-- For Meeting Targets MO & SC Incentives-- To Achieve State Goals FL Public Knowledge--Consumer Information WA & NY

Links to Teaching/Learning Improvements: 

Links to Teaching/Learning Improvements Making Assessment Institution-centered Public Accountability Institutions Share Data on Learning Revisiting Indicators Regularly Close Information Loop Focused Goals at Different Inst. Levels

Outcomes of Policy: 

Outcomes of Policy Institutional Resistance Disparate Effects “Negative Improvement” Excessive/Burdensome Requirements Indicators Not Useful for Management Institutional Cooperation Focus on Improvement Trust between SHEEO & State Partnership to Develop System

Policy Evaluation: 

Policy Evaluation Revisiting &Adjustment MO & SC Implementation NY & WA Implementation & Evaluation FL

Lessons Learned: 

Lessons Learned Assessment Must Be Incorporated Into Institutional Management Successful Policies Developed in Consultation with Institutions Culture of Institutions Can be Changed If The Process Contributes to Mgt

Lessons Learned (2): 

Lessons Learned (2) Institutions Must Also Be Willing to Form Working Relationships with State Officials Stakeholders at All Levels Must Be Engaged with Assessment Involving Too Many Stakeholders in Development Bogs Process Down

Lessons Learned (3): 

Lessons Learned (3) Sustained Commitment of Leadership Required Political Will for Success Required Policy Process Can Be As, If Not More Important Than Its Results

Successful Policies: 

Successful Policies Have a Clear & Focused Purpose Differentiate by Sector/Mission Emphasize Institutional Improvement Embrace Simpler, Rather than Complex, Indicator/Reporting Mechanisms Incorporate Priorities of Multiple Stakeholders Provide Useful Data for Decision Makers

Important Considerations: 

Important Considerations Context for Assessment Shapes Process Policies Result In Improved Data Systems Process Forces Articulation of Principles New Policies May Not Replace Old Ones Policy Must Be Useful to Instituions