01 uvalopen Evolution ppt

Evolution of Uval's work program: from support to evaluation : 

Evolution of Uval's work program: from support to evaluation Italian Evaluation Unit Open Days 3 – 5 July 2006 Laura Raimondo

Until April 2006 Ministry of Economy and Finance: 

Until April 2006 Ministry of Economy and Finance

Department for Development Policies: 

Department for Development Policies CSF MA

UVAL’s tasks : 

UVAL’s tasks UVAL founded in 1982 Reformed in 1998 Re-reformed in 2002 Main tasks: Promotion of data, information and knowledge for public policies Support to administrations to modernize and innovate public action, Evaluation of programs and capacity building for evaluation Ex-ante assessement of projects

UVAL’S work program: 

UVAL’S work program Activities (not staff) are organized in four areas Methods and studies for public policy Modernization and administrative innovation Program evaluation Project appraisal 104 activities Support to other parts of the Department and to the Network of the Italian Evaluation Units is still a large part of work (e.g., project appraisal, support to programming for 2007-2013)

The Evaluation Unit (UVAL): 

The Evaluation Unit (UVAL)

Program evaluation: 

Program evaluation Evaluation capacity development Started with Mezzogiorno & structural funds Now expanding to whole country & all development program Huge task: guidelines, groups, observation of practice & processes, common work Evaluations Slowly growing Legitimity (who asks for evaluations) Resources (financial and staff)

Evaluation: where we started from: 

Evaluation: where we started from A “continental” public service: No orientation to results Absence of social partners and policy makers Push for evaluation from EU Structural Funds: No national reinterpretation of general framework Predominance of EC evaluative needs Rigid framework Compliance at best purely formal No interactions Low quality of evaluations Little (if any) use of evaluations

Evaluation capacity strategy initially relied on:: 

Evaluation capacity strategy initially relied on: EU evaluation requirements + New context of policy making More ambitious development policy More emphasis on local knowledge More emphasis on change More responsibilities to regional governments Greater importance to administrative capacity

Close links between administration capacity and development policy: 

Close links between administration capacity and development policy Development policy requires administrative capacity Evaluation capacity is part of administrative (and political) capacity Evaluation results improve development policies if useful for key actors More than one way to get there

A piecemeal strategy: 

A piecemeal strategy developed over time gradually introducing innovations knowledge from day-to-day work taking into account achievements and disappointments We are still working on it

Some evaluation capacity building activities UVAL worked on: 

Some evaluation capacity building activities UVAL worked on Drafting national Guidelines (2001-2002) Efforts to open up the market (2001) Setting up networks and discussion groups (from 2001) Creation of evaluation units (2000-2002) Meta-evaluation activities (from 2002) Self assessment exercises (2002-2003) Utilization of evaluation, evaluative knowledge, and implicit knowledge in discussing strategic lines for next programming period and for nat’l interventions (2005) Orientation to ex-ante evaluation process and products and organization of training and discussion groups (2006)

UVAL role in the evaluation of Ob.1 CSF: 

UVAL role in the evaluation of Ob.1 CSF Organizing the evaluation: Suggested four segments (main contract, environment, cultural resources, macro model) Evaluation questions: Collected MA’s point of view Drafted Terms of Reference Dealt with EC, evaluator, MA Steering Group: Read and commented report drafts and reports Informal dealings, formal meetings and feedback Protected evaluators’ independence

A part of the Network of Italian Evaluation Units: 

A part of the Network of Italian Evaluation Units Not ALL evaluation units: Evaluation Units dealing with Development Policies Within executive branch Some Ministries and all Regions There are others: Regional elected assemblies, Universities, other Ministries. Mix of tasks (actually evaluation only a small part) ex ante analysis, project appraisals, feasibility studies direct evaluations (only one so far) management of mid-term evaluations Various roles in programming activities Interaction with local authorities and coalitions on projects UVAL plays a proactive role in the Network, but has no formal “central” role.

The National Evaluation System: 

The National Evaluation System UVAL – Ministry of Economy and Finance (Coordinator) INEA – Ministry for agricultural policies ISFOL – ESF Evaluation Unit – Ministry of Labour Evaluation Units of MA’s Initially built to deal with evaluation of Objective 1 regions’ strategy and OPs Now on the way to be extended to all Italian Regions’ regional policies

Slide16: 

External : Mid-term evaluation of CSF and OP Internal: Evaluation of the performance reserve system Thematic evaluations (social inclusion, urban development, research) Self-evaluation Different ways to organise evaluation

Slide17: 

EVALUATION OF EVALUATIONS, MID-TERM EVALUATION, UP-DATE Design Early Reports Fieldworks Intermediate Products and in-depth studies Fieldworks Evaluation Questions Organization of evaluation update EC reads Evaluation reports SNV reads reports for lessons and utilization Fieldworks Updating Mid-Term Evaluation Reports SNV reads intermediate evaluation products Working in team with Ob.1 Units (Questionnaire and Meeting in Oct). EVALUTION OF EVALUATIONS MID-TERM EVALUATION OF OP and CSF Mid-Term Evaluation Report Up-dating Mid- term : Strategy of CSF SNV reads preliminary products Work with Ob.1 Units Methodological talks SNV – Evaluators – MA’s – Units SNV makes reports to CSF’s MC