Final Seminar 28.02.2005

Slide 1: 

New Opportunities for PE and Sport Initiative Evaluation Seminar

Slide 2: 

Introduction NOPES programme Evaluation –why? Programme evaluation Big Lottery Fund Monitoring Self-evaluation Baseline and Annual Surveys Summary and Conclusions Structure of Seminar

Slide 3: 

Introduction to NOPES

Slide 4: 

Why Evaluate?

Slide 5: 

Why Evaluate? Demonstrate the impact of an initiative Identify ways to improve existing practice Influence future policy

Slide 6: 

Why Evaluate?

Slide 7: 

Why Evaluate?

Slide 8: 

Why Evaluate?

Slide 9: 

Why Evaluate?

Slide 10: 

New Opportunities for PE and Sport Initiative Programme Evaluation

Slide 11: 

NOPES Evaluation Team Loughborough Partnership Institute of Youth Sport Institute of Sport and Leisure Policy Centre for Research in Social Policy PE and Sport Pedagogy Group British Heart Foundation National Centre Centre for Developing and Evaluating Lifelong Learning, Nottingham University Led by Dr Mary Nevill, Director, Institute of Youth Sport Three Research Associates at Loughborough University

Slide 12: 

Programme Evaluation Three key dimensions Partnership effectiveness Impact on participation Impact of participation on wider social change

Slide 13: 

Programme Evaluation Methods 7 Fast Track projects 11 Core Case Study Telephone Case Studies Baseline and Annual Surveys

Slide 14: 

Programme Evaluation Dissemination of Results Reports to the Big Lottery Fund Evaluation Newsletters Evaluation Website www.nopesevaluation.org.uk

Slide 15: 

Findings – Fast Track projects Partnership Operational realities taken precedence over strategic aims and objectives Role seen as being instrumental in achieving longer term goals

Slide 16: 

Findings – Fast Track projects Participation Dramatic impact on both quality and quantity of PE recognised by both staff and young people Very positive impact on the provision of extra-curricular activities Impact on the community limited to date and presents a much greater challenge

Slide 17: 

Findings – Fast Track projects Impact on Wider Social Behaviour Links with wider social outcomes are based on very general notions about the benefits of sport and exercise Few specific programmes designed to target wider social behaviour

Slide 18: 

New Opportunities for PE and Sport Initiative Big Lottery Fund Monitoring

Slide 19: 

New Opportunities for PE and Sport Initiative Self-Evaluation

Slide 20: 

Self – Evaluation – Why? Task 1: In pairs or small groups consider: What are the main benefits of monitoring and evaluating the projects within your portfolio?

Slide 21: 

Self – Evaluation – Why? Demonstrate the impact of NOPES on local schools and communities Identify ways to improve existing practice within your LEA and within your schools Influence the future direction of PE and Sport policies for your LEA and schools

Slide 22: 

Self – Evaluation – Purpose To consider how your local authority will demonstrate the impact of your NOPES portfolio To consider how the projects within your portfolio can be supported in undertaking self-evaluation

Slide 23: 

Gathering evidence – Evaluator’s role New Opportunities for PE and Sport Self- Evaluation Handbook Includes: Survey guidance Step by step guide

Slide 24: 

Self – Evaluation – Step by Step Step 1 Identify aims, objectives and success indicators Step 2 Collect and record data   Step 3 Make sense of your data   Step 4 Share your findings

Slide 25: 

STEP 1 – Aims, Objectives and Success Criteria Aims - Aims are general statements you intend to achieve Objectives - More specific measurable outcomes

Slide 26: 

NOPES – Outcomes 1- 3 1.Improved physical education and sport in schools  2. Higher standards across the whole school through PE, sport and other forms of structured activity 3. Better opportunities to increase the levels of physical activity among the school age population and, more generally local communities

Slide 27: 

NOPES – Outcomes 4 - 6 4. Improved collaboration, co-operation and partnership between schools and their communities 5. Promotion of social inclusion through access to, and use of, sports and outdoor adventure facilities by all groups in society 6. Innovation and best practice in the design and management of facilities including consideration of the principles of sustainable construction and development.

Slide 28: 

STEP 1 – Aims, Objectives and Success Criteria Success indicators - Criteria to determine whether the process is successful or not in achieving the aims originally identified. Success indicators need to be SMART (Specific, Measurable, Attainable, Relevant and Time-limited)

Slide 29: 

SMART Success Criteria Task 2 Using the following aim and objectives taken from a development plan identify possible SMART Success Criteria. SMART = Specific, Measurable, Attainable, Relevant and Time-limited Aim: NOPES Outcomes 3 and 4 Objective: Develop inter-school competitions organised and umpired by students

Slide 30: 

SMART Success Criteria Task 2 - Examples of SMART Success criteria Quantitative measures of impact Summer term 2005 - 5 Year 10 students involved in JSLA to organise a one day ‘kwik cricket’ event for 4 local primary schools in Summer term using the new facility. Also need to consider: Quality of provision How will future policy and practice be informed?

Slide 31: 

STEP 2 – Collect and record data Data collection What data need to be collected? How will the data be collected? How often will the data be collected?

Slide 32: 

STEP 2 – Collect and record data Task 3 In pairs or small groups consider: How will the required data be collated within your projects? What assistance will they need in order to do this? Is any support available?

Slide 33: 

STEP 3 – Making sense of the data Has your project or portfolio achieved the intended aims and objectives? Do the results match your expectations? What implications does this examination of the data have for the future of the project? What will need to be done to ensure the project achieves the aims and objectives identified?

Slide 34: 

STEP 4 – Sharing the findings Audiences Stakeholders and partners Young people Parents Colleagues Reporting the findings Posters Newsletters Reports

Slide 35: 

Sharing the findings – Evaluator’s role Sharing examples of effective practice and evaluation findings through: Newsletters Website - www.nopesevaluation.org.uk Evaluation progress Resources Effective practice

Slide 36: 

New Opportunities for PE and Sport Initiative Baseline and Annual Surveys

Slide 37: 

Development Extensive development phase Piloted with fast track projects Baseline and Annual Surveys Big Lottery Fund Monitoring

Slide 38: 

Purpose Quantitative Data Change Over Time Baseline and Annual Surveys Tools Baseline Survey – NOPES funding under £125,000 and / or not based at school site Baseline Survey (Extended) – NOPES funding over £125,000 based at school site

Slide 39: 

Baseline and Annual Surveys Distribution Directly to Schools / Projects Follow Up Annual Survey LEA Role Access to returned surveys Loughborough Report (50% of surveys returned)

Slide 40: 

Baseline Survey Distribution - England

Slide 41: 

Baseline Survey (Extended)

Slide 42: 

Section 2 : Background Information Baseline Survey (Extended) Quality of existing facilities Can be used as evidence for Outcome 3 & 6 Q5

Slide 43: 

Section 2 : Background Information Baseline Survey (Extended) Importance of other initiatives Can be used as evidence for Outcome 4 Q6

Slide 44: 

Section 3 : Typical Week Baseline Survey (Extended) A week in which there is minimal disruption to school timetables and plans as a result of activities weeks, poor weather, exams and tests, plays, visits or concerts. In a typical week, pupils are familiar with the timetable and pattern of the day.

Slide 45: 

Section 3 : Information about the pupils at your school Q11 Baseline Survey (Extended) Curriculum PE Time How to collect Can be used as evidence for Outcome 1

Slide 46: 

Section 3 : Information about the pupils at your school Q12 Baseline Survey (Extended) Pupils participating in 2 hours PE and Sport How to collect Can be used as evidence for Outcome 1

Slide 47: 

Section 3 : Information about the pupils at your school Q14 Baseline Survey (Extended) Availability of extra-curricular activities Participation in extra-curricular activities How to collect? Can be used as evidence for Outcome 1

Slide 48: 

Section 4 : Usage by other schools and community Q16 Baseline Survey (Extended) Other school’s use Definition How to collect? Can be used as evidence for Outcomes 3 & 4

Slide 49: 

Section 5 : Range of Activities Q19 Baseline Survey (Extended) Available Activities How to collect? Can be used as evidence for Outcomes 1 & 3

Slide 50: 

Section 6 : Target Users Q20 Baseline Survey – Larger Projects Identify Target Groups Target groups users How to collect? Can be used as evidence for Outcome 5

Slide 51: 

Self-Evaluation Handbook Website www.nopesevaluation.org.uk Phone and email I.A.Lindsey@lboro.ac.uk / C.L.J.Mason@lboro.ac.uk 01509 226379 / 01509 226378 Support for Completion

Slide 52: 

Why evaluate? NOPES programme evaluation Self-evaluation Baseline and Annual Surveys Summary and Conclusions

Slide 53: 

New Opportunities for PE and Sport Initiative Evaluation Seminar