Evidence-based policy: evidence-based policy

What is evidence-based policy?

Evidence-based policy is an approach to the development of public policy in which decisions are based on systematic analysis of data and research results, rather than on ideological preferences, intuition, or lobbying pressure.

The concept originated in medicine (evidence-based medicine in the 1990s) and then spread to public policy. Proponents include Tony Blair (“What works”, United Kingdom, 1990s), Barack Obama (emphasis on rigorous evaluation), and a number of Scandinavian governments.

Tools of evidence-based policy

Randomized Controlled Trials (RCT)

The "gold standard" of the evidence base. Subjects are randomly assigned to groups—those receiving the intervention and a control group.

Application in policy:

Experiments with conditional cash transfers (Progresa/Oportunidades in Mexico): RCTs showed a reduction in poverty and an increase in school attendance
Experiments with basic income (Finland 2017–2018: 2,000 unemployed received €560/month unconditionally)
Microfinance programs: RCTs by Abhijit Banerjee and Esther Duflo (Nobel Prize 2019) showed a limited impact of microloans on the incomes of the poor

Limitations of RCTs in policy:

Ethical: is it acceptable to randomly deprive some people of a potentially beneficial intervention?
Problem of generalization: if it works in Mexico—will it work in Germany?
Political horizon: RCTs require time, politicians want results quickly

Quasi-experimental methods

When randomization is impossible:

Difference-in-differences (DiD): Comparing changes in a group that received an intervention to a control group before and after. For example: the impact of increasing the minimum wage in one state compared to a neighboring state.

Regression Discontinuity Design: Using “thresholds” (for example, eligibility for a benefit at an income below X) as a source of quasi-random assignment.

Synthetic control: Creating a “synthetic counterfactual”—a weighted combination of control units closest to the treated unit before the intervention.

Systematic reviews and meta-analysis

Cochrane Collaboration (medicine), Campbell Collaboration (social policy) are organizations specializing in the systematic synthesis of studies.

Instead of reading one study, a policymaker receives a meta-analysis of 30 studies with an assessment of effect size and heterogeneity.

What Works Centers (UK)

The UK government has established a series of specialized “What Works Centers”:

Education Endowment Foundation (what works in education)
What Works Centre for Wellbeing
NICE (National Institute for Health and Care Excellence)—recommendations for the NHS
Centre for Homelessness Impact

Application of evidence-based policy in business regulation

Impact Assessment

Before adopting a new regulatory act, the EU requires the conduct of a Regulatory Impact Assessment (RIA):

Defining the problem
Assessment of the baseline scenario (“do nothing”)
Analysis of alternative policy options
Assessment of economic, social, and environmental impacts
Monitoring plan

REFIT (EU): A program to assess existing regulation for its relevance and proportionality. The goal is to reduce regulatory burden without compromising policy objectives.

Regulatory Sandbox

An evidence-based policy tool: a new product/service is tested on a limited scale with temporary relaxation of regulatory requirements.

Examples:

FCA (UK): the world’s first fintech regulatory sandbox (2016). Hundreds of fintech companies tested products in a controlled environment.
DIFC (Dubai): Innovation Testing Licence—a similar mechanism for fintech and proptech.
MAS (Singapore): Fintech regulatory sandbox.

For startups and innovators: the sandbox lowers the barrier to entry for new products, allowing a body of evidence to be built before full regulatory approval.

Limitations of evidence-based policy

The problem of knowledge use in policy

Even with a good evidence base, policymakers do not necessarily use it:

Political incentives: Reforms that bear fruit in 10 years are not attractive to a politician focused on the next election
Ideological filters: Research contradicting beliefs is rejected
Lobbying pressure: Industry groups may challenge inconvenient results

The “reproducibility crisis”

Many results in the social sciences are not reproduced in repeated studies. According to the Open Science Collaboration (2015), only 36% out of 100 psychology studies were successfully reproduced.

The problem of scaling

What works in a pilot may not work at full scale (general equilibrium effects: a larger-scale program affects prices and behavior, which did not occur in the pilot).