evidence and policy: where are we now?

Evidence and policy: Where are we now? :

Evidence and policy: Where are we now? Annette Boaz King’s College London 11 th May 2011

Outline:

Outline Evidence based policy in the UK Understanding the role of research in policy and practice The ‘three generations’ approach 2

Do we have evidence based policy?:

Do we have evidence based policy? Practice change from knowledge transfer activities is only about 8-15% Grimshaw et al 2001; Crowley et al 2004 3

Slide 4:

“Social science should be at the heart of policy making… we need social scientists to help determine what works and why, and what types of policy initiatives are likely to be most effective.” David Blunkett , Secretary of State for Education “No one with the slightest common-sense could take seriously suggestions by University researchers that homework is bad for you.” David Blunkett , Secretary of State for Education 4

Evidence based policy has::

Evidence based policy has: 5 Infiltrated our language Affected staffing and budgets in government Influenced policy development Influenced planning/ change programmes Led to new agencies and initiatives

With implications for research practice::

With implications for research practice: 6 Renewed interest in RCTs New interest in systematic reviews Mega and Meta evaluations Interest in research quality Interest in policy transfer

Is this changing?:

Is this changing? 7 ‘Government cuts millions from research and evaluation budgets.’ ‘Scrapped projects worth £8.9m include evaluation of academy schools and survey of disabled children's services’ Guardian, Saturday 26 th February 2011

And even at the best of times, research... :

And even at the best of times, r esearch ... competes with many other factors isn’t valued as an information source isn’t relevant isn’t easy to use 8 ‘Well, obviously there is a political context to almost everything that we do. Ministers want to be seen to be taking action and sometimes any action is better than no action... There are often political commitments that lead you in directions that the evidence doesn’t necessarily strongly support.’ (Campbell et al 2007)

And even at the best of times, research... :

And even at the best of times, r esearch ... competes with many other factors isn’t valued as an information source isn’t relevant isn’t easy to use 9 “..You’ve [the analytical expert] been researching this area for ten years, have you got any ideas? It can be quite irritating sometimes when somebody then just shrugs their shoulders and says ‘that’s not my job, i’m just an analyst.’ You think, ‘yes, take your analysis a bit further and give us some ideas about if these are all the problems which might some of the solutions be?’ ” (Campbell et al 2007)

Slide 10:

Why is improvement so hard? Factors relating to the intervention Factors relating to the environment Factors relating to the people involved Martin Marshall, Health Foundation Or to look at another way...

Slide 11:

11 Notion of evidence based policy has led to renewed assumptions that decisions follow directly on from evidence

But...:

But... 12 The use of evidence is complex Evidence is about more than research Governments are interested in more than ‘what works’

Different situations where you might need evidence :

Different situations where you might need evidence 13 Stable policy fields Policy fields in flux Inherently novel policy fields Mulgan, 2002

Promoting the use of evidence in policy requires::

Promoting the use of evidence in policy requires: 14 Features of success Research translation Enthusiasm Contextual analysis Credibility Leadership Support Integration Nutley et al 2003

Evidence and argument:

Evidence and argument As politicians know only too well, but social scientists too often forget, public policy is made of language. Whether in written or oral form, argumentation is central to all stages of the policy process’ (Majone 1989) 15

Slide 16:

16 Skills in argumentation Scope for partnerships (with those interested in knowledge transfer) Managing engagement Writing for different audiences (with professional writers and editors)

Policy choices are made on the basis of what people::

Policy choices are made on the basis of what people: believe (ideologies) want (interests) know (information) and how they can get it (institutions) 17

The Four ‘I’s that shape policy:

The Four ‘I’s that shape policy 18 Weiss 1995

Ideologies includes::

Ideologies includes: 19 philosophies, principles, values, world views, political orientation.. range from coherent to homely… are as much rooted in emotion as reason…

Interests (i.e. Stakeholders) have self interest::

Interests (i.e. Stakeholders) have self interest: 20 e.g. politicians want re-election… bureaucrats want career advancement… professions want respect… lobbies want influence… enterprises want resources…

Information (or evidence ) is very diverse::

Information (or evidence ) is very diverse: 21 in subject in detail in source in reliability

Types of evidence:

Types of evidence 22 “Expert knowledge; published research; existing statistics; stakeholder consultations; previous policy evaluations; the Internet; outcomes from consultations; costings of policy options; output from economic and statistical modelling.” (Cabinet Office 1999)

Institutions have::

Institutions have: 23 structures and procedures skills and expertise budgets cultures and a history

Institutions provide:

Institutions provide opportunities for and constraints on the influence of ideologies, interests and information 24

Ideologies/interests and information interact:

Ideologies/interests and information interact 25 e.g. an ideology will predispose giving weight to particular information new information may lead to a reformulation of interest

All four ‘I’s are dynamic:

All four ‘I’s are dynamic 26 contentious information ideological fashion coalitions of interest institutional change

Where are we now?:

Where are we now? Three generations of thinking about knowledge transfer/ evidence into practice: Linear models Relationship models Systems models Best and Holmes (2010) 27

Systems dimensions of interest in promoting knowledge transfer::

Systems dimensions of interest in promoting knowledge transfer: 28 Evidence and knowledge Leadership Networks Communications Best and Holmes (2010)

My contact details: Annette.boaz@kcl.ac.uk:

My contact details: Annette.boaz@kcl.ac.uk Some useful references: Best, A and Holmes, B (2010) Systems thinking, knowledge and action. Evidence and Policy. 6 (2): 145-149 -Boaz, A, Grayson, L, Levitt, R and Solesbury, W (2008) Does Evidence Based Policy Work? Evidence & Policy. 4 (2): 233-253 Campbell, S, Benita, S, Coates, E, Davies, P and Penn, G (2007) Analysis for policy: evidence based policy in practice. London: HM Treasury. Crowley, W.F et al (2004) Clinical research in the United States at a crossroads: proposal for a novel public-private partnership to establish a national clinical research enterprise. JAMA. 291: 1120-6 Grimshaw, J et al (2001) Changing provider behaviour: An overview of systematic review interventions. Medical Care . 38: II-2-II-4 Majone , G (1989) Evidence, Argument and Persuasion in the Policy Process. New Haven, Conneticut : Yale University Press. Nutley S, Walter I, and Davies H (2007) Using evidence: How research can inform public services. Bristol: The Policy Press. Weiss, C (1995) The four ‘I’s of school reform. How interests, ideology, information and institutions affect teachers and principles. Harvard Educational Review . 65 (4): 571-592

Slide 30:

30 Managing Editors: Annette Boaz, Division of Health and Social Care Research, King's College London, UK and David Gough, Institute of Education, University of London, UK Volume 6, 2010, 4 issues: January, May, August and November http://www.policypress.co.uk/journals_eap.asp