Using evidence to influence policy: experience from less-developed countries - a discussion with Professor Dean Karlan

Yale University Professor Dean Karlan is an internationally-recognised expert in development economics and using scientific evidence to influence policy. He is a board member of Abdul Latif Jameel Poverty Action lab (J-PAL), and founded its sister NGO Innovations for Poverty Action, which both run research in dozens of countries and advise governments and others on the basis of the results. Professor Karlan will discuss the opportunities and barriers to uptake of evidence by governments and others in the developing world. This builds on the Institute for Government’s earlier work on evidence and evaluation in policy making.

Speaker: Professor Dean Karlan

Chair: Jill Rutter, Institute for Government

Watch the video of the event here

Professor Karlan notes that is it not always in the nature of academics to work proactively with those charged with implementing their suggestions.  Many academics see their role as to test theories, and not to consider whether their research will see the light of day outside academic circles.  This often affects the nature of their research questions which do not always lead to a workable policy solution.  This disconnect between the need for quality evidence to inform policy and the nature of the evidence often generated by academics inspired Professor Karlan to found Innovations for Poverty Action (IPA) in 2002, in order to bridge the gap between the researcher and the policymaker. 

Professor Karlan summarised his approach towards evidenced-based solutions with a quote from Richard Thaler, ‘grandfather’ of the Nudge Unit: ‘If you want to encourage some activity, make it easy’

This statement was true not only of trying to encourage individuals to act in a particular way, but also of encouraging governments to make use of evidence-based policy.  He recommended a three-step process to solving problems effectively and ‘making it easy’ for governments to put recommendations into practice;

1. What is the problem? Figure out which questions need answering

2. Design a programme and tests to answer that question

3. Tell people about the evidence and make sure it is used

Researchers needed to ensure that their evidence is based on robust and fair tests.  Part of the work of the IPA has been to encourage the use of randomised controlled trials (RCTs) to test interventions in policy areas where this method has not been used before, such as combatting corruption.

Professor Karlan’s third step for tackling problems effectively was to ‘make it easy’ to make use of evidence.  This could involve presenting evidence using non-technical language, providing easily digestible policy summaries, and working with stakeholders to help implement a policy.  Finally, he noted that it was important to test interventions for multiple outcomes, to identify which aspect of holistic interventions produced the desired effect.  Testing the ‘multiple arms’ of a policy would allow policymakers to allocate resources effectively.

Professor Karlan described the ‘three I’s’ - ignorance, inertia and ideology - which could prevent the uptake of evidence in policy making.  He explained that, as well as dispelling ignorance, robust evidence could be used to grind down inertia and ideological resistance, underlining the importance of the three steps towards robust, reliable and implementable research evidence.

Q & A session

Speaking from the Chair, Jill Rutter asked Professor Karlan about the ethical objections that were often raised against RCTs.  Professor Karlan replied that it is important not to let emotions prevent us from doing important research.  An intervention that had not been tested in an RCT will simply be tested on the next generation when that intervention becomes policy.  If the goal were  to minimise experimentation, then testing an intervention on a small control group before rolling it out across a larger population is the way to achieve this goal. 

Professor Karlan also took questions from the audience on the challenges of ensuring that research is neutral and not led by policy concerns; the different challenges faced by researchers working with governments in the developed and developing worlds in terms of encouraging the use of evidence-based policy; and the ethical implications of using ‘nudge’ tactics to implement policy. 

Robyn Munro