logging in or signing up MONITORING AND EVALUATION TECHNIQUES 3 aSGuest53822 Download Post to : URL : Related Presentations : Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Uploaded from authorPOINT lite Insert YouTube videos in PowerPont slides with aS Desktop Copy embed code: Embed: Flash iPad Dynamic Copy Does not support media & animations Automatically changes to Flash or non-Flash embed WordPress Embed Customize Embed URL: Copy Thumbnail: Copy The presentation is successfully added In Your Favorites. Views: 5236 Category: Education License: All Rights Reserved Like it (3) Dislike it (0) Added: July 10, 2010 This Presentation is Public Favorites: 2 Presentation Description No description available. Comments Posting comment... Premium member Presentation Transcript MONITORING AND EVALUATION TECHNIQUES : MONITORING AND EVALUATION TECHNIQUES INDICATORS 1 ANDREW MSAMI LEARNING OBJECTIVES : LEARNING OBJECTIVES To identify what indicators are and why they are important How to select indicators Criteria for assessing quality of indicators How and why it is necessary to develop indicators 2 ANDREW MSAMI MONITORING AND EVALUATION COST AND BENEFIT : MONITORING AND EVALUATION COST AND BENEFIT Monitoring is an expensive and time consuming activity hence the need to selectively identify indicators to measure a programme or project change Indicators enables the monitoring and evaluation unit of the organization to reduce the amount of data and information collected, analyzed and reported to its simplest form while generating important information for management decision making. 3 ANDREW MSAMI COMPONENTS OF AN INDICATOR : COMPONENTS OF AN INDICATOR Information Unit of Measurement Documentation of Change Example: Levels of workplace programmes capacity in HIV/AIDS advocacy Unit of measurement Documentation of Change Information 4 ANDREW MSAMI COMPONENTS OF AN INDICATOR : COMPONENTS OF AN INDICATOR Evidence of how much has been or has not been achieved Quantitative: numbers, percentage Qualitative: narrative or observation 5 ANDREW MSAMI IMPORTANCE OF INDICATORS : IMPORTANCE OF INDICATORS Examination of Trends Highlight problems Act as early warning signs Support management corrective actions Evaluate effectiveness of management action Determine achievement of results 6 ANDREW MSAMI INDICATORS ARE NOT : INDICATORS ARE NOT Anything you can think of to measure; Reduce data into its simplest form Objectives, targets, goals or results Words like improved, increased, gained do not normally belong to an indicator Indicators describe information to be measured Example: “increase in knowledge level” is a goal “ change in knowledge” is an indicator 7 ANDREW MSAMI COUNTING MEASURES VS PROXY INDICATORS : COUNTING MEASURES VS PROXY INDICATORS Counting Measures in place of indicators Number of meetings held Number of people trained Number of vehicles purchased The above example should not be used as indicators An indicator for “a number of people trained” could be measured by “level of knowledge” 8 ANDREW MSAMI INDICATOR SELECTION : STEPS IN INDICATOR SELECTION INDICATOR SELECTION 9 ANDREW MSAMI STEP 1 : STEP 1 Clarify the results statements; identify what needs to be measured Precise wording and intentions of the objective Avoid broad results statements Clarity of change implied: situation, a condition, knowledge, attitude, practice, behavior Each type of change is measured by a different indicator Clarity of where change should appear: individuals, groups, communities 10 ANDREW MSAMI STEP 2 : STEP 2 Develop indicators through brainstorming and research Determine what should be measured List possible indicators for what you intend to measure Assess the indicator based on set selection criteria Use a participatory approach to develop indicators 11 ANDREW MSAMI STEP 3 : STEP 3 Assess each possible indicator Criteria for judging appropriateness of indicators 12 ANDREW MSAMI STEP 4 : STEP 4 Select the best indicator Narrow down to the final indicators to be used in your monitoring and evaluation system Be cautious of cost involved in data collection and analysis Limit the number of indicators to track a single object to three 13 ANDREW MSAMI STEP 5 : STEP 5 Draft Indicator protocols Note: protocols are instruction sheet that capture the reason for selecting indicators Describe indicators in precise terms Identify plans for data collection analysis and reporting 14 ANDREW MSAMI STEP 6 : STEP 6 Collect Baseline data Baseline data can be available from your ex-ante assessment (formative evaluation) if they do not exist the conduct a baseline survey Baseline data establish the starting point from which your indicators will be measured 15 ANDREW MSAMI STEP 7 : STEP 7 Refine your indicators and protocols and finalize your selection 16 ANDREW MSAMI RESULTS CHAIN AND INDICATORS : RESULTS CHAIN AND INDICATORS EXAMPLES 17 ANDREW MSAMI PROJECT MONITORING : PROJECT MONITORING LEARNING OBJECTIVE To identify additional monitoring tools useful to project management 18 ANDREW MSAMI MONITORING TOOLS : MONITORING TOOLS Benchmark calendar List of planned key activities arranged by dates Activity Based Budget Estimates of inputs required to undertake benchmarked activities Comparison Chart Compares plans and budgets against occurrence and spending Deliverable schedule Indicates what is due for a project to offer within set time 19 ANDREW MSAMI EVALUATION : EVALUATION LEARNING OBJECTIVES Importance of Evaluation Different types of evaluation Who should be involved in evaluation How to develop a learning agenda 20 ANDREW MSAMI WHAT IS EVALUATION : WHAT IS EVALUATION Monitoring track data and information on progress of the project or programme but it can not tell managers and other stakeholders “why results are being achieved” To answer the question evaluation need to be conducted to collect and analyze data and information on the cause-effects linkages in the project or programme 21 ANDREW MSAMI WHY EVALUATION IS IMPORANT : WHY EVALUATION IS IMPORANT Seeks to understand why changes are occurring Seeks to understand why changes are not occurring Support usage of finding for learning and adaptation Assess relevance, effectiveness and sustainability of project results 22 ANDREW MSAMI EVALUATION AND ADAPTIVE MANAGEMENT APPROACH : EVALUATION AND ADAPTIVE MANAGEMENT APPROACH Decision making approach involving a cycle of planning, implementation, monitoring, evaluation and reflection Action in response to learning Learning and understanding of complex systems in a changing and unpredictable environment Based on double loop of learning and solving problems 23 ANDREW MSAMI TYPE AND PROCESSES OF EVALUATION : TYPE AND PROCESSES OF EVALUATION Formative Evaluation (Ex ante) Mid Term Evaluation Final Evaluation (End of project/programme) Ex Post Evaluation 24 ANDREW MSAMI FORMATIVE EVALUATION (FE) : FORMATIVE EVALUATION (FE) Takes place at the beginning of the project/programme-in the design and concept phase Help organizations understand operational setting of the programme Provide information to define goals, objectives and strategies for a programme 25 ANDREW MSAMI METHODS FOR FE : METHODS FOR FE Literature review Discussion of beneficiaries and partners FGD’s Interviews and surveys Participants observations 26 ANDREW MSAMI MID TERM EVALUATION (MDE) : MID TERM EVALUATION (MDE) Conducted half way the life of project/programme A review of the project/programme it assess Progress Sufficiency of interventions and budgets towards results Identification of barriers to achievement Identify opportunities, unanticipated accomplishment or innovation 27 ANDREW MSAMI METHODS FOR MDE : METHODS FOR MDE Records review Review monitoring reports Key informant interviews Direct observation Surveys 28 ANDREW MSAMI FINAL EVALUATION : FINAL EVALUATION Support process for review of strategic plan Assess the why of positive and negative results Test validity of assumptions underlying results framework Assess how well needs of various customers were met Identify and analyze unintended consequences Examine sustainability of results Supports a learning process 29 ANDREW MSAMI METHODS FOR FINAL EVALUATION : METHODS FOR FINAL EVALUATION Records review Review of monitoring reports Key informants interviews FGD’s Direct observations Surveys 30 ANDREW MSAMI EVALUATION AND LEARNING AGENDA : EVALUATION AND LEARNING AGENDA Evaluation is about learning Development of a learning agenda Determine the learning needs-identification of what to be evaluated Clarify questions to support the learning (What, How and Why) 31 ANDREW MSAMI PARTICIPATORY EVALUATION AND LEARNING : PARTICIPATORY EVALUATION AND LEARNING Capture a wide range of opinion from a wide range of sources Involve as many stakeholders as possible to enhance a sense of ownership and partnership Enables articulation of different stakeholders needs and how they have been met 32 ANDREW MSAMI REPORTING : REPORTING Collection and analysis of monitoring and evaluation data is meant for reporting to various audiences The challenge of reporting is turning raw data and information into useful knowledge 33 ANDREW MSAMI WHAT IS REPORTING : WHAT IS REPORTING A report is a compilation of descriptive information A communication tool to present monitoring and evaluation results as knowledge Reporting covers: progress, problems, successes, lesson learned during programme implementation 34 ANDREW MSAMI WHY IS REPORTING ESSENTIAL : WHY IS REPORTING ESSENTIAL Focus different audiences on results Enable improvement of subsequent work plans Basis for decision making Support organization learning Communicate efficiency and effectiveness of organizations 35 ANDREW MSAMI ELEMENTS OF A GOOD REPORT : ELEMENTS OF A GOOD REPORT Focus on results and accomplishment Assess performance based on established indicators States the extent of achievement Specify actions to overcome problems 36 ANDREW MSAMI REPORTING FORMATS : REPORTING FORMATS PROGRESS REPORTS EVALUATION REPORTS 37 ANDREW MSAMI PROGRESS REPORT FORMAT : PROGRESS REPORT FORMAT INTRODUCTION COMPARISON OF PLANNED VS ACTUAL EVENTS ADMINISTRATIVE REVIEW FINANCIAL REVIEW-EXPENDITURE REPORT INDICATOR DATA LESSONS LEARNT ISSUES REQUIRING IMMEDIATE SUPPORT ANNEXES 38 ANDREW MSAMI EVALUATION REPORT FORMAT : EVALUATION REPORT FORMAT EXECUTIVE SUMMARY INTRODUCTION METHODS USED RESULTS/IMPACT SUMMARY OF PROSPECTS FOR SUSTAINABILITY REVIEW OF DELIVERABLE FINANCIAL REPORT LESSONS LEARNED ANNEXES 39 ANDREW MSAMI You do not have the permission to view this presentation. In order to view it, please contact the author of the presentation.
MONITORING AND EVALUATION TECHNIQUES 3 aSGuest53822 Download Post to : URL : Related Presentations : Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Uploaded from authorPOINT lite Insert YouTube videos in PowerPont slides with aS Desktop Copy embed code: Embed: Flash iPad Dynamic Copy Does not support media & animations Automatically changes to Flash or non-Flash embed WordPress Embed Customize Embed URL: Copy Thumbnail: Copy The presentation is successfully added In Your Favorites. Views: 5236 Category: Education License: All Rights Reserved Like it (3) Dislike it (0) Added: July 10, 2010 This Presentation is Public Favorites: 2 Presentation Description No description available. Comments Posting comment... Premium member Presentation Transcript MONITORING AND EVALUATION TECHNIQUES : MONITORING AND EVALUATION TECHNIQUES INDICATORS 1 ANDREW MSAMI LEARNING OBJECTIVES : LEARNING OBJECTIVES To identify what indicators are and why they are important How to select indicators Criteria for assessing quality of indicators How and why it is necessary to develop indicators 2 ANDREW MSAMI MONITORING AND EVALUATION COST AND BENEFIT : MONITORING AND EVALUATION COST AND BENEFIT Monitoring is an expensive and time consuming activity hence the need to selectively identify indicators to measure a programme or project change Indicators enables the monitoring and evaluation unit of the organization to reduce the amount of data and information collected, analyzed and reported to its simplest form while generating important information for management decision making. 3 ANDREW MSAMI COMPONENTS OF AN INDICATOR : COMPONENTS OF AN INDICATOR Information Unit of Measurement Documentation of Change Example: Levels of workplace programmes capacity in HIV/AIDS advocacy Unit of measurement Documentation of Change Information 4 ANDREW MSAMI COMPONENTS OF AN INDICATOR : COMPONENTS OF AN INDICATOR Evidence of how much has been or has not been achieved Quantitative: numbers, percentage Qualitative: narrative or observation 5 ANDREW MSAMI IMPORTANCE OF INDICATORS : IMPORTANCE OF INDICATORS Examination of Trends Highlight problems Act as early warning signs Support management corrective actions Evaluate effectiveness of management action Determine achievement of results 6 ANDREW MSAMI INDICATORS ARE NOT : INDICATORS ARE NOT Anything you can think of to measure; Reduce data into its simplest form Objectives, targets, goals or results Words like improved, increased, gained do not normally belong to an indicator Indicators describe information to be measured Example: “increase in knowledge level” is a goal “ change in knowledge” is an indicator 7 ANDREW MSAMI COUNTING MEASURES VS PROXY INDICATORS : COUNTING MEASURES VS PROXY INDICATORS Counting Measures in place of indicators Number of meetings held Number of people trained Number of vehicles purchased The above example should not be used as indicators An indicator for “a number of people trained” could be measured by “level of knowledge” 8 ANDREW MSAMI INDICATOR SELECTION : STEPS IN INDICATOR SELECTION INDICATOR SELECTION 9 ANDREW MSAMI STEP 1 : STEP 1 Clarify the results statements; identify what needs to be measured Precise wording and intentions of the objective Avoid broad results statements Clarity of change implied: situation, a condition, knowledge, attitude, practice, behavior Each type of change is measured by a different indicator Clarity of where change should appear: individuals, groups, communities 10 ANDREW MSAMI STEP 2 : STEP 2 Develop indicators through brainstorming and research Determine what should be measured List possible indicators for what you intend to measure Assess the indicator based on set selection criteria Use a participatory approach to develop indicators 11 ANDREW MSAMI STEP 3 : STEP 3 Assess each possible indicator Criteria for judging appropriateness of indicators 12 ANDREW MSAMI STEP 4 : STEP 4 Select the best indicator Narrow down to the final indicators to be used in your monitoring and evaluation system Be cautious of cost involved in data collection and analysis Limit the number of indicators to track a single object to three 13 ANDREW MSAMI STEP 5 : STEP 5 Draft Indicator protocols Note: protocols are instruction sheet that capture the reason for selecting indicators Describe indicators in precise terms Identify plans for data collection analysis and reporting 14 ANDREW MSAMI STEP 6 : STEP 6 Collect Baseline data Baseline data can be available from your ex-ante assessment (formative evaluation) if they do not exist the conduct a baseline survey Baseline data establish the starting point from which your indicators will be measured 15 ANDREW MSAMI STEP 7 : STEP 7 Refine your indicators and protocols and finalize your selection 16 ANDREW MSAMI RESULTS CHAIN AND INDICATORS : RESULTS CHAIN AND INDICATORS EXAMPLES 17 ANDREW MSAMI PROJECT MONITORING : PROJECT MONITORING LEARNING OBJECTIVE To identify additional monitoring tools useful to project management 18 ANDREW MSAMI MONITORING TOOLS : MONITORING TOOLS Benchmark calendar List of planned key activities arranged by dates Activity Based Budget Estimates of inputs required to undertake benchmarked activities Comparison Chart Compares plans and budgets against occurrence and spending Deliverable schedule Indicates what is due for a project to offer within set time 19 ANDREW MSAMI EVALUATION : EVALUATION LEARNING OBJECTIVES Importance of Evaluation Different types of evaluation Who should be involved in evaluation How to develop a learning agenda 20 ANDREW MSAMI WHAT IS EVALUATION : WHAT IS EVALUATION Monitoring track data and information on progress of the project or programme but it can not tell managers and other stakeholders “why results are being achieved” To answer the question evaluation need to be conducted to collect and analyze data and information on the cause-effects linkages in the project or programme 21 ANDREW MSAMI WHY EVALUATION IS IMPORANT : WHY EVALUATION IS IMPORANT Seeks to understand why changes are occurring Seeks to understand why changes are not occurring Support usage of finding for learning and adaptation Assess relevance, effectiveness and sustainability of project results 22 ANDREW MSAMI EVALUATION AND ADAPTIVE MANAGEMENT APPROACH : EVALUATION AND ADAPTIVE MANAGEMENT APPROACH Decision making approach involving a cycle of planning, implementation, monitoring, evaluation and reflection Action in response to learning Learning and understanding of complex systems in a changing and unpredictable environment Based on double loop of learning and solving problems 23 ANDREW MSAMI TYPE AND PROCESSES OF EVALUATION : TYPE AND PROCESSES OF EVALUATION Formative Evaluation (Ex ante) Mid Term Evaluation Final Evaluation (End of project/programme) Ex Post Evaluation 24 ANDREW MSAMI FORMATIVE EVALUATION (FE) : FORMATIVE EVALUATION (FE) Takes place at the beginning of the project/programme-in the design and concept phase Help organizations understand operational setting of the programme Provide information to define goals, objectives and strategies for a programme 25 ANDREW MSAMI METHODS FOR FE : METHODS FOR FE Literature review Discussion of beneficiaries and partners FGD’s Interviews and surveys Participants observations 26 ANDREW MSAMI MID TERM EVALUATION (MDE) : MID TERM EVALUATION (MDE) Conducted half way the life of project/programme A review of the project/programme it assess Progress Sufficiency of interventions and budgets towards results Identification of barriers to achievement Identify opportunities, unanticipated accomplishment or innovation 27 ANDREW MSAMI METHODS FOR MDE : METHODS FOR MDE Records review Review monitoring reports Key informant interviews Direct observation Surveys 28 ANDREW MSAMI FINAL EVALUATION : FINAL EVALUATION Support process for review of strategic plan Assess the why of positive and negative results Test validity of assumptions underlying results framework Assess how well needs of various customers were met Identify and analyze unintended consequences Examine sustainability of results Supports a learning process 29 ANDREW MSAMI METHODS FOR FINAL EVALUATION : METHODS FOR FINAL EVALUATION Records review Review of monitoring reports Key informants interviews FGD’s Direct observations Surveys 30 ANDREW MSAMI EVALUATION AND LEARNING AGENDA : EVALUATION AND LEARNING AGENDA Evaluation is about learning Development of a learning agenda Determine the learning needs-identification of what to be evaluated Clarify questions to support the learning (What, How and Why) 31 ANDREW MSAMI PARTICIPATORY EVALUATION AND LEARNING : PARTICIPATORY EVALUATION AND LEARNING Capture a wide range of opinion from a wide range of sources Involve as many stakeholders as possible to enhance a sense of ownership and partnership Enables articulation of different stakeholders needs and how they have been met 32 ANDREW MSAMI REPORTING : REPORTING Collection and analysis of monitoring and evaluation data is meant for reporting to various audiences The challenge of reporting is turning raw data and information into useful knowledge 33 ANDREW MSAMI WHAT IS REPORTING : WHAT IS REPORTING A report is a compilation of descriptive information A communication tool to present monitoring and evaluation results as knowledge Reporting covers: progress, problems, successes, lesson learned during programme implementation 34 ANDREW MSAMI WHY IS REPORTING ESSENTIAL : WHY IS REPORTING ESSENTIAL Focus different audiences on results Enable improvement of subsequent work plans Basis for decision making Support organization learning Communicate efficiency and effectiveness of organizations 35 ANDREW MSAMI ELEMENTS OF A GOOD REPORT : ELEMENTS OF A GOOD REPORT Focus on results and accomplishment Assess performance based on established indicators States the extent of achievement Specify actions to overcome problems 36 ANDREW MSAMI REPORTING FORMATS : REPORTING FORMATS PROGRESS REPORTS EVALUATION REPORTS 37 ANDREW MSAMI PROGRESS REPORT FORMAT : PROGRESS REPORT FORMAT INTRODUCTION COMPARISON OF PLANNED VS ACTUAL EVENTS ADMINISTRATIVE REVIEW FINANCIAL REVIEW-EXPENDITURE REPORT INDICATOR DATA LESSONS LEARNT ISSUES REQUIRING IMMEDIATE SUPPORT ANNEXES 38 ANDREW MSAMI EVALUATION REPORT FORMAT : EVALUATION REPORT FORMAT EXECUTIVE SUMMARY INTRODUCTION METHODS USED RESULTS/IMPACT SUMMARY OF PROSPECTS FOR SUSTAINABILITY REVIEW OF DELIVERABLE FINANCIAL REPORT LESSONS LEARNED ANNEXES 39 ANDREW MSAMI