CFGB Results Framework Evaluation Presentation

Logical Framework – Plan for Evaluation!: 

Logical Framework – Plan for Evaluation! UAB Center for Educational Accountability Brian F. Geiger, EdD, FAAHE Marcia R. O’Neal, PhD Scott Snyder, PhD Tel. 205/ 975-5388; fax 205/ 975-5389 Web: http://www.ed.uab.edu/cea/index.htm

Learning objectives: 

Learning objectives At the end of the session, you will be able to: Define key evaluation terminology Select indicators to monitor and evaluate your project Construct a project logical model (worksheet) Describe process to evaluate short- and medium-term outcomes Understand CFGB reporting requirements 2

PowerPoint Presentation: 

Result #1: Children are successful along the education pipeline Strategies : Increase high quality early learning opportunities birth to 8; Decrease high school dropout rates CFGB introduced a new approach to drive positive change… 4 Results and related Strategies 3 By Señor Codo

CFGB introduced a new approach to drive positive change…4 Results and related Strategies: 

CFGB introduced a new approach to drive positive change…4 Results and related Strategies Result #2: People can lead healthy lives Strategies : Improve nutrition/healthy food access and increase opportunities for physical activity; Improve care access for vulnerable populations 4 U.S. Department of Agriculture. ChooseMyPlate.gov

PowerPoint Presentation: 

Result #3: Communities are sustainable, livable and vibrant Strategies: Expand and improve public green space; Further develop vibrant city center; Improve natural environment; Expand access to arts and cultural opportunities 5 Acnatta’s photostream CFGB introduced a new approach to drive positive change… 4 Results and related Strategies

PowerPoint Presentation: 

Result #4: Individuals and families are economically secure Strategies: Improve housing stability; Increase public policy changes and direct services that positively impact low income individuals and families 6 Leff’s Photostream & Randy Le’Moine Photography CFGB introduced a new approach to drive positive change… 4 Results and related Strategies

Project Logic Model: 

Project Logic Model 7

Important Terms: 

Important Terms Inputs - personnel, funds, facilities, supplies, equipment necessary for project implementation What specific resources does your agency or organization bring to the table? 8

Important Terms: 

Important Terms Activities are aimed at realizing short- medium- and long-term outcomes, e.g. vocational training for adults, presenting a preschool curriculum, offering healthy cooking classes to older adults Participation – expectations to engage intended audience of children, youth and adults in project activities 9

Project Logical Framework: 

Project Logical Framework A planning and evaluation tool used to: analyze the existing situation during planning establish sequence of resources and activities to attain objectives identify potential risks to achieve objectives and reach short- and medium-term outcomes establish how activities, participants, and outcomes are best monitored and evaluated during implementation present a visual summary of the project 10

Project Logic Model: 

Project Logic Model 11

Logic Model: 

Logic Model 12

Logic Model: 

Logic Model 13

Logic Model: 

Logic Model 14

Sample Logic Model for Abstinence Education (Adapted from HUD Center for Faith-Based and Neighborhood Partnerships): 

Sample Logic Model for Abstinence Education (Adapted from HUD Center for Faith-Based and Neighborhood Partnerships) 15 Inputs Activities Participation Outcomes - Impact What resources does your organization have ? What will your project do to effect change? Who will participate? What will they do? What short- and medium-term changes are expected? How much change? How will you measure performance? Abstinence education curriculum Process to train peers as mentors Experienced staff members Trusted relationship with community residents

PowerPoint Presentation: 

16 Inputs Activities Participation Outcomes - Impact What resources does your organization have ? What will your project do to effect change? Who will participate? What will they do? What short- and medium-term changes are expected? How much change? How will you measure performance? Abstinence education curriculum Process to train peers as mentors Experienced staff members Trusted relationship with community residents Recruit students and peer mentors Implement abstinence education classes in 3 neighborhood schools Train teens as peer mentors

PowerPoint Presentation: 

17 Inputs Activities Participation Outcomes - Impact What resources does your organization have ? What will your project do to effect change? Who will participate? What will they do? What short- and medium-term changes are expected? How much change? How will you measure performance? Abstinence education curriculum Process to train peers as mentors Experienced staff members Trusted relationship with community residents Recruit students and peer mentors Implement abstinence education classes in 3 neighborhood schools Train teens as peer mentors 100 students will complete 6 afternoon training sessions 50 students will be matched with peer mentors within 2 months Mentors will assist peers with healthy decision-making

PowerPoint Presentation: 

18 Inputs Activities Participation Outcomes - Impact What resources does your organization have ? What will your project do to effect change? Who will participate? What will they do? What short- and medium-term changes are expected? How much change? How will you measure performance? Abstinence education curriculum Process to train peers as mentors Experienced staff members Trusted relationship with community residents Recruit students and peer mentors Implement abstinence education classes in 3 neighborhood schools Train teens as peer mentors 100 students will complete 6 afternoon training sessions 50 students will be matched with peer mentors within 2 months Mentors will assist peers with healthy decision-making 80% of the students who are mentored will report abstinence Counselors will administer surveys to students, parents, and mentors to rate program effectiveness Program adjusted, as needed

Important Terms: 

Important Terms Outcomes are measurable short- medium- and long-term changes as a result of the project. Outcomes are reflected in your statement of PURPOSE Project evaluation will focus on assessing short- and medium-term changes identified in the logic model 19

Chain of Outcomes: 

Chain of Outcomes Short-term outcomes Achieved during program timeframe Within program control “expect to see” Medium-term outcomes Long-term outcomes Logic Model Workbook ©Innovation Network , Inc. 20

Chain of Outcomes: 

Chain of Outcomes Short-term outcomes Medium-term outcomes Achieved at the end / beyond program timeframe Follow short-term outcomes “want to see” Long-term outcomes Logic Model Workbook ©Innovation Network , Inc. 21

Chain of Outcomes: 

Chain of Outcomes Short-term outcomes Medium-term outcomes Longer-term outcomes Achieved after program timeframe Outside direct program control “hope to see” Logic Model Workbook ©Innovation Network , Inc. 22

Important Terms: 

Important Terms Long-term Outcomes relate to CFGB Strategies These effects for the population at large may not be realized within period of project funding e.g. changes in disease incidence/ prevalence, changes in life expectancy and quality of life, changes in economic impact, rates of employment 23

Grant Purpose: 

Describe how CFGB funds will be used – (this is not the budget – it generally describes personnel funded, resources acquired, activities conducted) Tell what audience(s) will benefit – (target population) Identify planned contributions of other groups/agencies in the project – (who they are, what they will do) 24 Grant Purpose

Supportive Evidence: 

Supportive Evidence Briefly summarize any literature, research, data, and/or best practices that provide evidence that your proposed strategies will impact medium- and long-term outcomes of your program/project.  Note: Your evidence must be relevant to the Result Strategy you plan to address. 25

CFGB Project Logic Model: 

CFGB Project Logic Model 26

PowerPoint Presentation: 

Monitoring is assessment of… Project implementation and management Staff performance Achievement of goals and objectives Resource requirements (i.e. staffing and cost) 27 Important Terms

Common terminologies: 

Common terminologies Process Monitoring is a best practice recommended for your project administrator(s) Monitoring includes description, counting and tracking of project activities and participation Answers the questions What was done? Where did it happen? When did it occur? and How much or how many? 28 Important Terms

Sample Process Monitoring Plan: 

Sample Process Monitoring Plan Activity Responsible Person for Data Collection Timeline Measures What activity will be completed? Who will collect the data? When will data be gathered? What information will be collected to document the activity? Example : orient agencies to goals and eligibility criteria for women’s training and employment program Program coordinator Twice per calendar year Agenda, r oster of present staff members, participant completion survey 29

Sample Process Monitoring Plan: 

Sample Process Monitoring Plan Activity Responsible Person for Data Collection Timeline Measures What activity will be completed? Who will collect the data? When will data be gathered? What information will be collected to document the activity? Example: train three groups of young women on money management skills Vocational trainer Quarterly at conclusion of training session Practical a ssessment of money management skills 30

Common terminologies: 

Common terminologies Process Monitoring is a form of quality assurance It will not answer Is it effective? or Why is it effective? Important to assess if something is being done, if it is performed as intended, where and when intended, and at the level intended 31 Important Terms

PowerPoint Presentation: 

32 Baseline Data – data relevant to an outcome gathered BEFORE beginning project activities e.g., Unemployment rates in metro Birmingham from 2007-summer 2011

Important Terms: 

Important Terms 33 New Data – data relevant to an outcome gathered AFTER project activities begin and compared to Baseline Data to determine if change has occurred e.g., employment rate among graduates of a vocational training program supported by CFGB

PowerPoint Presentation: 

Evaluation is the p rocess of comparing what has changed as a result of specific activities aimed at improving long-term outcomes (CFGB Results and Strategies) Which outcomes are directly attributable to the project intervention (i.e. inputs, activities, participation)? Evaluation is a tool to assess effectiveness of a project 34 Important Terms

Project Outcome Indicators: 

Project Outcome Indicators Specify how you will measure and verify achievement of objectives Used by planners to monitor progress (delivery of activities and participation) Enables you to answer the question: How do I know whether this project makes a difference? 35

Develop S-M-A-R-T Outcome Indicators: 

Develop S-M-A-R-T Outcome Indicators S – specific , i.e. project objectives should specify what will be achieved M - measurable in the program setting A – achievable R – realistic given available resources and constraints T - time-framed 36

PowerPoint Presentation: 

Outcome Measuring the Outcome Data Collection Strategy Baseline Change What will change because of your project? What data for this outcome will you collect to measure change? How will the data be collected? Describe your measurement tool. What data do you have or will you gather at the beginning of the project? Who will be responsible to collect data? When will you gather new data to monitor outcomes? Who will be responsible to collect data? You will complete and submit to CFGB annually! 37 Project Evaluation Plan

PowerPoint Presentation: 

Outcome Measuring the Outcome Data Collection Strategy Baseline Change 80 % of children served by program will have decreased number of disciplinary referrals during the school year Disciplinary referral record for each child served by the program at end of school year Program staff will obtain from the local education agency records of disciplinary referrals Records of student past disciplinary referrals compiled by the local education agency. Program staff will request records from the child’s school with parental permission. Records will be gathered from the local education agency at end of each grading period by program staff with parental permission. What will change because of your project? What data for this outcome will you collect to measure change? How will the data be collected? Describe your measurement tool. What data do you have or will you gather at the beginning of the project? Who will be responsible to collect data? When will you gather new data to monitor outcomes? Who will be responsible to collect data? 38 Sample Project Evaluation Plan

PowerPoint Presentation: 

Outcome Measuring the Outcome Data Collection Strategy Baseline Change 80% of children served by program will have decreased number of disciplinary referrals during the school year Disciplinary referral record for each child served by the program at end of school year Program staff will obtain from the local education agency records of disciplinary referrals Records of student past disciplinary referrals compiled by the local education agency. Program staff will request records from the child’s school with parental permission. Records will be gathered from the local education agency at end of each grading period by program staff with parental permission. What will change because of your project? What data for this outcome will you collect to measure change? How will the data be collected? Describe your measurement tool. What data do you have or will you gather at the beginning of the project? Who will be responsible to collect data? When will you gather new data to monitor outcomes? Who will be responsible to collect data? 39 Sample Project Evaluation Plan

PowerPoint Presentation: 

Outcome Measuring the Outcome Data Collection Strategy Baseline Change 80% of children served by program will have decreased number of disciplinary referrals during the school year Disciplinary referral record for each child served by the program at end of school year Program staff will obtain from the local education agency records of disciplinary referrals Records of student past disciplinary referrals compiled by the local education agency. Program staff will request records from the child’s school with parental permission. Records will be gathered from the local education agency at end of each grading period by program staff with parental permission. What will change because of your project? What data for this outcome will you collect to measure change? How will the data be collected? Describe your measurement tool. What data do you have or will you gather at the beginning of the project? Who will be responsible to collect data? When will you gather new data to monitor outcomes? Who will be responsible to collect data? 40 Sample Project Evaluation Plan

PowerPoint Presentation: 

Outcome Measuring the Outcome Data Collection Strategy Baseline Change 80% of children served by program will have decreased number of disciplinary referrals during the school year Disciplinary referral record for each child served by the program at end of school year Program staff will obtain from the local education agency records of disciplinary referrals Records of student past disciplinary referrals compiled by the local education agency. Program staff will request records from the child’s school with parental permission. Records will be gathered from the local education agency at end of each grading period by program staff with parental permission. What will change because of your project? What data for this outcome will you collect to measure change? How will the data be collected? Describe your measurement tool. What data do you have or will you gather at the beginning of the project? Who will be responsible to collect data? When will you gather new data to monitor outcomes? Who will be responsible to collect data? 41 Sample Project Evaluation Plan

PowerPoint Presentation: 

Outcome Measuring the Outcome Data Collection Strategy Baseline Change 80% of children served by program will have decreased number of disciplinary referrals during the school year Disciplinary referral record for each child served by the program at end of school year Program staff will obtain from the local education agency records of disciplinary referrals Records of student past disciplinary referrals compiled by the local education agency. Program staff will request records from the child’s school with parental permission. Records will be gathered from the local education agency at end of each grading period by program staff with parental permission. What will change because of your project? What data for this outcome will you collect to measure change? How will the data be collected? Describe your measurement tool. What data do you have or will you gather at the beginning of the project? Who will be responsible to collect data? When will you gather new data to monitor outcomes? Who will be responsible to collect data? 42 Sample Project Evaluation Plan

Why Evaluate?: 

Why Evaluate? To determine project effectiveness To focus activities aimed at results To ensure accountability to the funder and public To inform decisions on new investment plans and programs 43

Preliminary Steps: 

Preliminary S teps Identify needed baseline data and measures of change Complete the Project Logic Model Complete the Project Evaluation Plan 44

Now what?: 

Now what? Access the Full Proposal Form on the CFGB Online Grant Application System (available 21 Feb.) Complete all narrative portions Upload all required documents including your Project Logic Model and Project Evaluation Plan 45

Now what?: 

Now what? Full Proposals must be completed AND submitted by midnight Thu. 15 March, 2012 CFGB will contact you to arrange a site visit or information meeting Review Team members include CFGB staff, committee members and community volunteers 46

Questions about the proposal???: 

Questions about the proposal??? James McCrary, VP of Grants and Evaluation Community Foundation of Greater Birmingham Tel. 205 327-3812 direct Email: jmccrary@foundationbirmingham.org 47