Making Policy Better Series - Armchair evaluators: How can citizen feedback drive better policy

This was the third in a series of seminars the Institute for Government is organising, in partnership with the National Institute of Economic and Social Research and NESTA, and in association with the Alliance for Useful Evidence. This followed up key recommendations in our report Making Policy Better. This seminar explored the role of citizens/private operators in holding public services and government to account.

Panel discussion with:

• Nadhim Zahawi MP - MP for Stratford-on-Avon, Co founder & CEO of YouGov PLC 2000 – February 2010, Co author of “Masters of Nothing”
• Cathy Newman - Channel 4 News presenter who runs the FactCheck blog.
• Kate Ebbutt - from Patient Opinion the UK's leading independent non-profit feedback platform for health services
• Hadley Beeman - Technology Strategy Board and founder of LinkedGov

Chair:

• Geoff Mulgan - Chief executive, NESTA

Audio

Armchair evaluators audio recording (MP3, 01:09:34)

Geoff Mulgan introduced the event by saying that the provision of more public data spawns a society of ‘armchair evaluators,’ who are hard for politicians and policy makers to ignore, and who could influence both the supply of and the demand for data.

Hadley Beeman focussed on the implementation of the Coalition’s plan to publish all public sector data for use by external bodies. The challenges are for the public sector to learn how to redact and anonymise data, what to do with data that was previously charged for and how to deal with questions around national security. There are also issues with the varying quality and formats of data generated by different bodies across the public sector, which makes them hard for outsiders to work with. What is surprising about the open data policy agenda is how much bigger it had become than anyone expected; it has generated a great deal of interest and a lot of input into government about how it should be done.  

Kate Ebbutt saw Patient Opinion as a powerful tool to hold services to account. It feeds stories from service users back to staff, who can then make improvements to the services.  In one example, a substance abuser had fed back on the consequences of missing a Friday appointment: with clinics closed on the weekend that had big implications for service on Monday. That had led to service redesign – an example of co-production. The culture change brought on by the web represents a huge shift the transfer of power away from government and to citizens. The initial reaction from the NHS was to be negative – with feedback channelled to patient Liaison Services and staff not allowed to react. Now staff and managers are much more engaged and prepared to react to feedback – with potential to save on litigation. As a tool it is very scalable; Patient Opinion alone aims to collect 100,000 stories per year, which at the current rate would lead to 10,000 changes in NHS services per year.

Nadhim Zahawi MP cautioned that the UK is notoriously poor at citizen feedback; it tends to focus on lobbyists and special interest groups, not ordinary people. He acknowledged that it is hard to get the opinion of ordinary people without just hearing those who are angry and shout the loudest. What is required, in order to get the largely silent majority to share their experiences, is to keep the way feedback is collected simple. He pointed to new methods of collecting feedback, for example interactive survey formats, which are based on responding to the citizen, going further and deeper into issues. YouGov had for example done some pioneering work which took a panel of citizens through the process of thinking through regional assemblies – effectively mimicking a campaign: that turned them from supporters into sceptics.

Cathy Newman explained that Channel 4’s Factcheck helps the ordinary citizen scrutinise the claims of people in public office, by cutting through political spin and government obfuscation, adjudicating between conflicting claims during election campaigns, and explaining technical and complex subjects. People get involved with Factcheck by suggesting facts to be checked, but also by helping the Factcheck team with their enquiries. Cathy cited the work experience row, where a reader sent in a letter from DWP to Factcheck which used the word ‘mandatory’ despite Ministerial denials. Ministers had changed the letters – and thus the policy – as a result.   

Geoff pointed out that these were all overlapping aspects of an emerging “ecology” of citizen feedback and scrutiny. In discussion concerns were raised over possible biases in open source data, particularly the access the elderly have to the internet, though the panel thought these issues could be overcome. Another set of questions were about the consequences for those working in the public sector – increased workload and whether they had the skills to deal with the amount of feedback they could receive from armchair evaluators, as government became more accessible. But this was a rapidly changing area and there was no consensus on what this means for government over the next five years.

Rosie Shorrocks, Institute for Government