Working to make government more effective

Comment

Making an evidence check work

Today sees the House of Commons Science and Technology Committee launch an ‘evidence check’ exercise, where both members of the public and experts are being asked to comment on the evidence base underpinning government policy decisions. Jen Gold explores the importance of this initiative, and looks at how it can reach its full potential.

It’s often fiendishly difficult to find out what evidence (if any) underpins the policy announcements that ministers make – something we documented in our Show Your Workings report last October. Despite a government commitment in 2013 to publish more of its evidence base, little progress has been made and the structure of the policies section on the gov.uk website makes the documentation that is available hard to locate.

But as my colleague Jill Rutter recently pointed out: “Government showing its workings to those who elected them… is an essential building block of an effective democracy.” The Institute for Government has long argued that parliamentary select committees play an important role in encouraging departments to disclose the evidence behind their policy decisions. Today’s science policy evidence check – which follows a similar initiative conducted by the Education Select Committee at the end of the last parliament – makes a welcome contribution.

How are committees carrying out ‘evidence checks’?

The Science and Technology Committee have followed a similar process to the Education Select Committee, by choosing a number of policy areas and asking the Government to provide a statement on the evidence base it has used to inform its decision making. These statements are then published on a series of web forums and members of the public, academics, and practitioners are invited to comment on the strength of the evidence outlined. In today’s evidence check, the Science and Technology Committee have released the first batch of statements on smart meters, accelerated access to healthcare, digital government, and flexible working (other policy areas will follow). And the statements produced by the government are now available on the committee’s website for comment until 29 January.

If the committee continues to follow the Education Committee’s process then they may hold oral evidence sessions on some policy areas. The Education Committee illustrated the impact that such sessions can have – the Government’s decision to enable summer-born children to delay the start of formal schooling followed hot on the heels of the Committee’s evidence hearing on school admission policies.

What did we learn from the Education Select Committee’s initial evidence check?

Transparency is a first step to assessing quality…

The public’s ability to comment on the strength of the evidence used by the Department for Education was heavily dependent on how the Department responded to the Committee’s request. More detailed statements – on the use of phonics in the classroom, for instance – enabled readers to engage directly with the evidence cited and to highlight contrasting evidence, selection biases, and gaps in the existing evidence base. Yet the submission on the work of the National College for Teaching and Leadership finally amounted to little more than a description of the creation, purpose, and aims of the college. As a result, respondents resorted to offering personal stories of their encounters with the college.

…so committees need to issue departments with a framework to structure their responses

The Department for Education wasn’t given a framework to guide the drafting of its statements, meaning that there was no minimum standard of disclosure they were required to meet and no guidelines for how to showcase best practice.

This time, the Science and Technology Committee has decided to give departments a framework – the evidence transparency tool that the Institute for Government developed in partnership with the Alliance for Useful Evidence and Sense About Science at the beginning of this parliament. We are planning to use this tool later this year to benchmark the performance of government departments on evidence transparency, but initial indications from the Science and Technology Committee’s evidence check suggests it has provided a useful way of organising departmental responses.

How is the Science and Technology Committee evidence check shaping up?

Even a cursory glance at the four evidence statements issued by the government reveals that many are fairly weak when it actually comes to discussing the evidence underpinning decision making. The submission on digital government, in particular, is barely more than a promotional statement on the Government’s ICT strategy. In other words, it’s difficult to see what members of the public or experts can usefully respond to. Across the four submissions, there’s little acknowledgement of gaps in the available evidence base or discussion of the alternative options considered and dismissed. Yet the Department of Health should be commended for making a much more comprehensive attempt to discuss of the evidence base behind accelerated access to healthcare. It is also clear that the exercise is more effective when a committee narrows its focus to a specific policy question rather than go on a broad fishing expedition across a wide field, which is likely to result in less than useful responses from government.

As the crowdsourcing phase of this evidence check gets underway, the Science and Technology Committee needs to bear in mind that any such exercise requires active outreach. We saw with the Education Committee evidence check that academics and professional organisations were slow to respond to the web forums. Meanwhile campaign groups and individual parents dominated the online discussion – and many chose to focus on criticising government policy rather than discussing the quality of the evidence cited.

As the period for submitting evidence wore on, the Education Committee became better at engaging practitioners and the research community through a targeted use of social media, writing guest blogs, and leveraging the networks of organisations such as the Education Media Centre and Royal Society. The Science and Technology Committee will likely need a similar strategy.

Evidence checks are an important new approach for committees that have the potential to drive better government decision-making. These exercises shouldn’t be one-off endeavours. Rather, they need to become a routine part of committee inquiries. Of course, there will be teething problems – but committees should look to share their learning with one another as they refine their methods and identify what works.

Related content