Evidence-based policy
Encyclopedia
Evidence-based policy is public policy
informed by rigorously established objective evidence
. It is an extension of the idea of evidence-based medicine
to all areas of public policy. An important aspect of evidence-based policy is the use of scientifically rigorous studies such as randomized controlled trial
s to identify programs and practices capable of improving policy relevant outcomes. However, some areas of knowledge are not well serviced by quantitative research, leading to debate about the methods and instruments that are considered critical for the collection of relevant evidence. Good data, analytical skills and political support, as such, are seen as the important elements.
Although evidence-based policy can be traced as far back as the fourteenth century, it was more recently popularized by the Blair Government
in the United Kingdom
. The Blair Government said they wanted to end the ideological led-based decision making for policy making. For example, a UK Government white paper published in 1999 ("Modernising Government") noted that Government "must produce policies that really deal with problems, that are forward-looking and shaped by evidence rather than a response to short-term pressures; that tackle causes not symptoms".
Evidence-based policy is associated with Adrian Smith because in his 1996 presidential address to the Royal Statistical Society, Smith questioned the current process of policy making and urged for a more “evidence-based approach” commenting that it has “valuable lessons to offer”.
The term “evidence-based policy” has evolved from “evidence-based medicine“, in which research findings are used as the support for clinical decisions and evidence is gathered by randomized controlled trials (RCTs), which is comparing a treatment group with a placebo group to measure results. In 1993, the Cochrane Collaboration was established in the UK, and works to keep all RCTs up-to-date and provides “Cochrane reviews“ which provides primary research in human health and health policy. There was then an increase in research and policy activists pushing for more evidence-based policy-making which lead to the formation of the sister organization to Cochrane Collaboration, the Campbell Collaboration in 1999. The Campbell Collaboration conducts reviews on the best evidence that analyzes the effects of social and educational policies and practices.
The Economic and Social Research Council (ESRC) became involved in the push for more evidence-based policy making with its 1.3 million pound grant to the Evidence Network in 1999. The Evidence Network is a center for evidence-based policy and practice and is similar to both the Campbell and Cochrane Collaboration.
The form of methodology used with evidence-based policy fit under the category of a cost-benefit framework and are created to estimate a net payoff is the policy was to be implemented. Because there is a difficulty in quantifying some effects and outcomes of the policy, it is mostly focused broadly on whether or not benefits will outweigh costs, instead of using specific values.
has pioneered RAPID Outcome Mapping Approach (ROMA) over the past five years as a means to help aid donors and partners better transform research into policy initiatives.
These lessons show that the relationship between research, policy and practice is complex, multi-factoral, non-linear, and highly context specific. What works in one situation may not work in another. Developing effective strategies in complex environments is not straightforward. Simple tools such as cost–benefit analysis, logical frameworks, traditional project management tools and others may not work on their own, as they fail to take into account the existing complexity.
Public policy
Public policy as government action is generally the principled guide to action taken by the administrative or executive branches of the state with regard to a class of issues in a manner consistent with law and institutional customs. In general, the foundation is the pertinent national and...
informed by rigorously established objective evidence
Evidence
Evidence in its broadest sense includes everything that is used to determine or demonstrate the truth of an assertion. Giving or procuring evidence is the process of using those things that are either presumed to be true, or were themselves proven via evidence, to demonstrate an assertion's truth...
. It is an extension of the idea of evidence-based medicine
Evidence-based medicine
Evidence-based medicine or evidence-based practice aims to apply the best available evidence gained from the scientific method to clinical decision making. It seeks to assess the strength of evidence of the risks and benefits of treatments and diagnostic tests...
to all areas of public policy. An important aspect of evidence-based policy is the use of scientifically rigorous studies such as randomized controlled trial
Randomized controlled trial
A randomized controlled trial is a type of scientific experiment - a form of clinical trial - most commonly used in testing the safety and efficacy or effectiveness of healthcare services or health technologies A randomized controlled trial (RCT) is a type of scientific experiment - a form of...
s to identify programs and practices capable of improving policy relevant outcomes. However, some areas of knowledge are not well serviced by quantitative research, leading to debate about the methods and instruments that are considered critical for the collection of relevant evidence. Good data, analytical skills and political support, as such, are seen as the important elements.
Although evidence-based policy can be traced as far back as the fourteenth century, it was more recently popularized by the Blair Government
Tony Blair
Anthony Charles Lynton Blair is a former British Labour Party politician who served as the Prime Minister of the United Kingdom from 2 May 1997 to 27 June 2007. He was the Member of Parliament for Sedgefield from 1983 to 2007 and Leader of the Labour Party from 1994 to 2007...
in the United Kingdom
United Kingdom
The United Kingdom of Great Britain and Northern IrelandIn the United Kingdom and Dependencies, other languages have been officially recognised as legitimate autochthonous languages under the European Charter for Regional or Minority Languages...
. The Blair Government said they wanted to end the ideological led-based decision making for policy making. For example, a UK Government white paper published in 1999 ("Modernising Government") noted that Government "must produce policies that really deal with problems, that are forward-looking and shaped by evidence rather than a response to short-term pressures; that tackle causes not symptoms".
Evidence-based policy is associated with Adrian Smith because in his 1996 presidential address to the Royal Statistical Society, Smith questioned the current process of policy making and urged for a more “evidence-based approach” commenting that it has “valuable lessons to offer”.
History of evidence-based policy
The earliest form of evidence-based policy was tariff-making which was required under legislation to be educated by the public report issued by the Tariff Board. These reports were initially only reporting on the impacts but changed to also report on the effects of industries and the economy.The term “evidence-based policy” has evolved from “evidence-based medicine“, in which research findings are used as the support for clinical decisions and evidence is gathered by randomized controlled trials (RCTs), which is comparing a treatment group with a placebo group to measure results. In 1993, the Cochrane Collaboration was established in the UK, and works to keep all RCTs up-to-date and provides “Cochrane reviews“ which provides primary research in human health and health policy. There was then an increase in research and policy activists pushing for more evidence-based policy-making which lead to the formation of the sister organization to Cochrane Collaboration, the Campbell Collaboration in 1999. The Campbell Collaboration conducts reviews on the best evidence that analyzes the effects of social and educational policies and practices.
The Economic and Social Research Council (ESRC) became involved in the push for more evidence-based policy making with its 1.3 million pound grant to the Evidence Network in 1999. The Evidence Network is a center for evidence-based policy and practice and is similar to both the Campbell and Cochrane Collaboration.
The Methodology of evidence-based policy
There are many methodologies for evidence-based policy but they all share the following characteristics:- It tests a theory as to why the policy will be effective and what the impacts of the policy will be if it is successful
- Inclusion of a counterfactual: what would have occurred if the policy had not been implemented
- They incorporate some measurement of the impact
- Examines both direct and indirect effects that occur because of the policy
- Separates the uncertainties and controls for other influences outside of the policy that may have an effect on the outcome
- Should be able to be tested and replicated by a third party
The form of methodology used with evidence-based policy fit under the category of a cost-benefit framework and are created to estimate a net payoff is the policy was to be implemented. Because there is a difficulty in quantifying some effects and outcomes of the policy, it is mostly focused broadly on whether or not benefits will outweigh costs, instead of using specific values.
Evidence-based development policy
The Overseas Development InstituteOverseas Development Institute
The Overseas Development Institute is one of the leading independent think tanks on international development and humanitarian issues. Based in London, its mission is "to inspire and inform policy and practice which lead to the reduction of poverty, the alleviation of suffering and the achievement...
has pioneered RAPID Outcome Mapping Approach (ROMA) over the past five years as a means to help aid donors and partners better transform research into policy initiatives.
Key lessons of evidence-based policy-making
Six key lessons have been developed, which are:- Policy processes are complex and rarely linear or logical and simply presenting information to policy-makers and expecting them to act upon it is very unlikely to work. Policy processes are not purely linear as they have various stages that each take varying lengths of time to complete and may in fact be conducted simultaneously. Strategies must be fluid.
- Policy is often only weakly informed by research-based evidence due to information gaps, secrecy, the need for speedy responses, political expediency and the fact that policy makers are rarely scientists.
- Research-based evidence can contribute to policies that have a dramatic impact on lives. Success stories quoted in the UK's Department for International Development's (DFID) new research strategy include a 22% reduction in neonatal mortality in Ghana as a result of helping women begin breastfeeding within one hour of giving birth, and a 43% reduction in deaths among HIV positive children using a widely available antibiotic.
- The need for a holistic understanding of the context in which the policy is to be implemented.
- Policy entrepreneurs need additional skills to influence policy. They need to be political fixers, able to understand the politics and identify the key players. They need to be good storytellers, able to synthesise simple compelling stories from the results of the research. They need to be good networkers to work effectively with all the other stakeholders, and they need to be good engineers, building a programme that pulls all of this together.
- Policy entrepreneurs need clear intent – they need to really want to do it. Turning a researcher into a policy entrepreneur, or a research institute or department into a policy-focused think tank involves a fundamental re-orientation towards policy engagement rather than academic achievement; engaging much more with the policy community; developing a research agenda focusing on policy issues rather than academic interests; acquiring new skills or building multidisciplinary teams; establishing new internal systems and incentives; spending much more on communications; producing a different range of outputs; and working more in partnerships and networks.
These lessons show that the relationship between research, policy and practice is complex, multi-factoral, non-linear, and highly context specific. What works in one situation may not work in another. Developing effective strategies in complex environments is not straightforward. Simple tools such as cost–benefit analysis, logical frameworks, traditional project management tools and others may not work on their own, as they fail to take into account the existing complexity.
RAPID Outcome Mapping Approach
ROMA approach takes these lessons into account has been field tested through more than 40 workshops and training courses worldwide. It is an eight-step approach for each of which the ODI has developed resources and policy tools to ensure each step is comprehensively addressed:- Define a clear, overarching policy objective.
- Map the policy context around that issue and identify the key factors that may influence the policy process. The RAPID framework provides a useful checklist of questions.
- Identify the key influential stakeholders. RAPID’s Alignment, Interest and Influence Matrix (AIIM) can be used to map actors along three dimensions: the degree of alignment (i.e. agreement) with the proposed policy, their level of interest in the issue, and their ability to exert influence on the policy process.
- Develop a theory of change - identify the changes needed among them if they are to support the desired policy outcome.
- Develop a strategy to achieve the milestone changes in the process - Force Field Analysis is a flexible tool that can be used to further understand the forces supporting and opposing the desired policy change and suggest concrete responses.
- Ensure the engagement team has the competencies required to operationalise the strategy.
- Establish an action plan for meeting the desired policy objective - useful tools include the RAPID Information matrix, DFID’s log frame and IDRC’s Outcome Mapping Strategy Map among them.
- Develop a monitoring and learning system, not only to track progress, make any necessary adjustments and assess the effectiveness of the approach, but also to learn lessons for the future.
Results
This has resulted in:- Over 50 case studies on successful evidence-based policy engagement have been compiled, a network
- Development and facilitation of the evidence-based policy in Development Network (ebpdn), which links more than 20 institutional partners and thousands of practitioners working on evidence-based policy processes.
- Creating an array of practical toolkits designed with civil society organisations, researchers and progressive policy makers in mind.
- Direct support to civil society organisations (CSOs) to provide training in policy influencing and strategic communication.
- Strengthening the capacity for the UK Department for International DevelopmentDepartment for International DevelopmentThe Department For International Development is a United Kingdom government department with a Cabinet Minister in charge. It was separated from the Foreign and Commonwealth Office in 1997. The goal of the department is "to promote sustainable development and eliminate world poverty". The current...
(DFID) to influence other actors.