Multiprogram Alignment with Performance Measurement

Views:
 
     
 

Presentation Description

Measuring multiple programs

Comments

Presentation Transcript

Using Performance Measurement in Program Evaluation to Align Multi-Programs:

Using Performance Measurement in Program Evaluation to Align Multi-Programs Presented by: Betty Ann M. Turpin, Ph.D. Turpin Consultants, Inc. September 2015

Table of Contents:

Table of Contents Focus and Purpose Conceptual View of Evaluation and Performance Measurement Synergy Evaluation Project Context Logic Model for Case Example of Flu Prevention Example Strategies related to Prevention of Flu Example - Functional Category by Category Theme Performance Measurement Framework Development Performance Measurement Framework Analysis Architecture Analysis Framework and Database Structure Example of Nested Measurements Key Considerations Dashboard Contact Information

Focus and Purpose:

Focus and Purpose To demonstrate how performance measurement (PM) can contribute to program evaluation Demonstrate the importance of apriori analysis framework development To illuminate the flaw in thinking that prevails surrounding the dichotomy between PM and evaluation, that is: PM is only a management tool PM is only about outputs PM is a corporate office role for reporting

Conceptual View of Evaluation and Performance Measurement Synergy:

Conceptual View of Evaluation and P erformance Measurement Synergy Program lifecycle Ongoing data collection: outputs & outcomes Evaluations: a point in time Program Commences Program Ends or Continues Formative Summative Process Performance Metrics ongoing

Evaluation Project Context:

Evaluation Project Context Implement a multi-department evaluation that focused on one issue (e.g., tax credits programs, health prevention, etc.). Large horizontal government department evaluations can entail few to many Responsibility Centres (RC) that receive funds totalling millions of dollars Stringent reporting requirements: - Report to a central agency that provide the government departmental funding in the form of a summative evaluation

Logic Model for Case Example of Flu Prevention:

Logic Model for Case Example of Flu Prevention Early Detection & Warning Emergency Response Risk Communications * By no means is this a comprehensive program logic model

Example Strategies related to Prevention of Flu :

Example Strategies related to Prevention of Flu Stockpiles supplies: for investments in vaccine readiness clinical trial; rapid vaccine development; and ongoing support for NAS and NESS. Surge Capacity: for investments in organizational capacity for pandemic preparedness, through resources, training, infrastructure Prevention & Early Warning: for investments in surveillance program. Emergency Preparedness: for investments in emergency preparedness and emergency human resources.

Example - Functional Category by Category Theme:

Example - Functional Category by Category Theme Tools & Materials Development & Implementation Engagement Protocols & Guidelines Modeling Testing Capacity Building Mechanisms Networks

Performance Measurement Framework Development:

Performance Measurement Framework Development Developed data collection guidelines and process Consultation with responsibility centers : verify performance indicators, data types and sources; discuss process Consultation with responsibility centers provide support during data entry Data entered by responsibility centers Data Quality Assessment Data analysis, interpretation Data findings verification, discussion with responsibility centers

Performance Measurement Framework Architecture:

Performance Measurement Framework Architecture An analysis framework was developed to facilitate the data consolidation, analysis, and interpretation The PMF analysis framework incorporated the strategy themes, as well as functional categories, and category themes Depending on the Strategy focus, not all functional categories applied to each theme, nor would each category theme apply to each functional category The alignment for each indicator across departments were the outcomes in the logic model Indicators that fell under each stream of strategy-functional category-theme category were grouped to measure against a target

Analysis Framework and Database Structure:

Analysis Framework and Database Structure

Example of Nested Measurements:

Example of Nested Measurements Functional Categories Measurements Responsibility Centers

Key Considerations:

Key Considerations Engagement of responsibility centers (RCs) in the identification of measurements Identification and assessment of existing data for measurement Group measurements across RCs Measure only outcomes, not outputs Set measurement targets Develop a user-friendly database for users to input the measurements

Dashboard:

Dashboard Tell the story for each outcome Rollup the accomplishments by strategy and functional categories Target Balance

Contact Us::

Contact Us: Turpin Consultants, Inc. baturp@sympatico.ca

authorStream Live Help