It has been another politically tumultuous year.

The Prime Minister Theresa May lost her parliamentary majority in a snap general election. Revelations about ministers’ inappropriate conduct resulted in three Cabinet resignations. Preparations for Brexit have been disrupted by the election, by turnover in personnel and by difficulties in parliamentary management.[1] The Government faces challenges in key public services, notably hospitals, prisons and adult social care.[2]

This Whitehall Monitor annual report – our fifth – finds that:

  • The political situation following the early election constrained the Prime Minister’s political authority and created challenges for the Government’s legislative programme and management of public services, major projects and Brexit.
  • The civil service is growing, in terms of size and morale, but should be more diverse.
  • Government is less open than it was after 2010, and is not using data as effectively as it should.

This summary includes some key charts which can also be found – with more analysis – in the relevant chapters. Our website has more charts and deeper analysis on many of the subjects covered.

Politics: governing with a minority presents new challenges

Composition house of commons

By triggering an election three years earlier than required, Theresa May hoped to strengthen her ability to get parliamentary support for Brexit. Instead, the Government lost its parliamentary majority, leaving the Prime Minister in a difficult political situation and without a free hand to choose her Cabinet ministers.

Cabinet ministers new to post after election

Following the Prime Minister's post-election reshuffle, only a quarter of Cabinet ministers were new to their posts, fewer than after any other election or change of prime minister since at least 1997. Cabinet stability is welcome – it is better to avoid moving ministers just as they are getting to grips with their department – but in this case the limited change appeared to reflect the delicate balance of ministerial views on Brexit within the Cabinet.

This relative stasis was not matched at junior ministerial levels: 44% of all ministers across government were new to their roles after the post-election reshuffle. This upheaval came less than a year after May’s first set of government appointments in 2016, when 11 Cabinet attendees left government, three new departments were created, and only at the Ministry of Defence did more than half of ministers stay in post.[3]

Chart: when ministers were appointed to current posts

The January 2018 reshuffle caused even more upheaval. Moves at Cabinet level were relatively limited – 31% of attendees were new to their roles, and media attention focused on mooted moves which failed to materialise. However, the Ministry of Justice (MoJ) and Department for Digital, Culture, Media and Sport (DCMS) each welcomed their sixth Secretary of State since 2010. Junior ministerial turnover was high: 38.5% of all government ministers were new to their roles, including the entire ministerial team at the Cabinet Office and three quarters at the MoJ. 71% of ministers have been appointed to their roles since the 2017 general election.

Turnover at the Department for Exiting the European Union (DExEU) has been notable. Only the Secretary of State, David Davis, and one other minister (Robin Walker) have remained in place since the department was created in July 2016. Half of DExEU’s ministers changed at the June 2017 reshuffle. The Department’s Lords minister, a critical role given the challenge of navigating Brexit legislation through the upper house, has changed three times since June 2017. DExEU had two permanent secretaries in 2017, and the National Audit Office has reported that DExEU’s staff turnover is 9% a quarter, when most departments average 9% a year.[4] More positively, given all this churn, DExEU's ministerial team survived the January 2018 reshuffle intact (indeed, the department gained an extra minister).

Chart: size of government defeats in House of Commons

The Government would have found preparing the statute book for Brexit challenging under any circumstances, with a heavy workload of primary and secondary legislation to be passed within a vanishing timetable. Lacking a majority, it will also need to work hard to avoid defeats in the Commons, which are much more likely under minority governments: Labour in the late 1970s was defeated more than 30 times between 1976 and 1979. Avoiding defeats could be made more difficult by a high level of turnover in the Commons Whips’ Office: there have been two chief whips and three deputy chief whips since November 2017, and more than 70% of whips were new to their posts following the January 2018 reshuffle.

The early election disrupted select committee inquiries midstream, delayed government responses to committee reports, and stymied scrutiny as committees had to wait to be re-established. It took until November 2017 for the chair of the Liaison Committee to be elected and for the Intelligence and Security Committee to be set up.[5]

The early election also scuppered a planned revision of Single Departmental Plans, which were supposed to identify political priorities but initially failed to do so. A month before the election was called, the Government agreed the plans could be improved and promised to update them in June 2017. The election disrupted the planned revision and added numerous new political priorities in a manifesto that was the second-longest from the Conservatives since at least 1945.[6]

The Single Departmental Plans published in December 2017 were an improvement on their predecessors, many of which had not been updated since February 2016. None had previously existed for DExEU, the Department for International Trade (DIT) and the Department for Business, Energy and Industrial Strategy (BEIS). The new plans show a greater sense of prioritisation, a more consistent format and some performance data to track progress. But they should still prioritise further, and link priorities with spending and workforce strategy, giving a sense of whether the civil service has the right skills and experience to deliver them.[7]

Chart: red-green-amber rating of projects in government major projects portfolio

When it comes to the Government’s portfolio of projects, the number of projects remains as high as last year (143). This is before Brexit-related projects, in areas like customs and immigration, begin in earnest; the National Audit Office estimates there are 14 projects ‘critical for immediate implementation after EU Exit’ that might need to enter the Government’s portfolio.[8]

The Infrastructure and Projects Authority’s confidence that the portfolio of major projects will be delivered on time and on budget has fallen: only one fifth now have the highest green or amber/green confidence rating, compared with nearly half in 2013. More costly projects have lower confidence ratings: a third of projects worth over £1bn are rated amber/red or red.

Chart: percentage change in planned 'day-to-day' spending of government departments

The Government also faces challenges in delivering public services. Many departments are expected to cut their day-to-day spending budgets (Resource Departmental Expenditure Limits (DEL), in the jargon) over the next few years, including some – like the MoJ and Department for Communities and Local Government (DCLG, renamed the Ministry of Housing, Communities and Local Government in January 2018) – that have already absorbed significant cuts since 2010. Finding ways to control spending further without affecting the quality of public services will be challenging, given that most ‘belt-tightening’ options, like pay freezes and staff cuts, have already been tried. The Institute for Government’s Performance Tracker estimates that over a five-year period from 2015/16, the Government has already committed to spending £10bn on emergency cash injections to alleviate pressures on public services. To avoid this in future, it needs to change how services operate.[9]

The civil service: increasing numbers and morale – but not diversity

Departments are preparing for Brexit by hiring more staff, and civil servants are as motivated as ever. But considerable challenges remain in improving the diversity of the civil service.

Chart: civil service staff numbers

After staffing hit its lowest level since the Second World War in June 2016, the civil service has added more than 8,000 staff to reach just over 392,000 in September 2017. Some of this increase is Brexit-related: as well as DExEU and DIT building up their capacity, the Department for Environment, Food and Rural Affairs (80% of the work of which is framed by EU legislation) and BEIS (thought to have more Brexit workstreams than any other department) are also increasing staff numbers. This upward trend looks set to continue, with the Home Office currently recruiting 1,200 additional immigration caseworkers and 300 border staff.[10]

Chart: percentage change in civil servants at each grade

But Brexit is only part of the story. The numbers of civil servants in the second most senior grade, grades 6 and 7, have been increasing since 2012. (Numbers have also been increasing in the senior civil service, though some of this is due to machinery of government changes rather than recruitment.) Only at the most junior administrative level – AA/AO (administrative assistants and officers), including prison officers and people administering benefits and staffing job centres – have numbers continued to fall. This means reductions have been driven by cutting administrative staff who tend to be located in the English regions, while more senior, usually London-based policy officials are actually more numerous than in 2010.

Chart civil service people survey theme scores

Staff morale is a vital sign of administrative health. According to the Civil Service People Survey, morale has reached a record high across the civil service. The annual engagement index (an average of five questions asking civil servants about pride in, advocacy of, attachment to, inspiration from and motivation by their departments) is at 61%, up two points on 2016. Engagement scores rose across all but one of the other ‘themes’ in the survey, in which civil servants are asked about everything from leadership to how their work fits in with their organisation’s objectives. The exception was pay and benefits. Despite austerity and whatever might be happening at a political level, civil servants’ experience of their own work is more positive than ever.

Gender balance of each civil service grade

The story on diversity is less positive. While there has been progress on gender diversity – more than two fifths of all senior civil servants (and equivalent grades) are now women, a record high – women are still underrepresented at the top. Women outnumber men in the more junior grades but the percentage decreases with each step up in grade, with women remaining in the minority at grades 6 and 7 and the senior civil service. Of those appointed permanent secretaries in 2017, as many were men with the surname Rycroft as were women (two in each case).

Percentage of civil servants that are BAME and disabled

The picture is worse for ethnic minority and disabled civil servants at the most senior levels: representation is lower in the senior civil service than across both the civil service as a whole and the general population, with little recent progress. The civil service leadership is taking these issues seriously, most recently promising specific action on underrepresented groups at senior civil service level through a new diversity and inclusion strategy. This includes pledges to set and monitor civil service-wide targets from April 2018, which we look forward to analysing – although a lot of diversity data remains unavailable as individuals choose not to disclose them. The Cabinet Office has for the first time started gathering data on and working out new measures for another important characteristic: socio-economic background. This is a welcome step, but it would be better still if these data were published to enable the public to get a sense of the situation and provide an external stimulus to improve matters.

Data and transparency: government less open, especially DExEU

Openness is important for accountability: Parliament, press and the public should all be able to understand what government is doing and how well it is doing it. But openness also matters for government effectiveness. Publishing data can foster improvements in its use and quality, and widen the audience that might draw insights. It also has consequences for the economy in terms of businesses that might utilise open data.

There have been some positive steps over the past year. The Prime Minister’s race disparity audit shone a light on difficult truths. The Conservative manifesto promise to ‘continue the drive for open data’ was converted into Budget announcements of a commission on geospatial data use and a centre for data ethics. The UK remains at or near the top of global open data rankings. We have also seen progress in the publication of government grants data, the development of ‘registers’ (single reliable lists, of anything from countries recognised by the UK to local authorities), and in December 2017, the Government Digital Service issuing new guidance for publishing transparency data.[11]

But, again, the political situation has had an impact on transparency. The Government’s own assessment of its Open Government Partnership commitments – on everything from Freedom of Information to contract transparency – notes slower delivery because of ‘institutional change’. One fifth of these commitments are behind schedule.[12] Despite the promises of the 2017 Government Transformation Strategy, no chief data officer has been appointed.[13] Without such leadership or political sponsorship, it’s not always obvious who has lead responsibility for data in government. Compiling this report is not as easy as it should be: there are inconsistencies in how departments are labelled, gaps in data, a lack of clarity and explanation, and a sense that data users have not been considered, with data veiled in vexing formats.

Percentage of Freedom of Information requests withheld

Departments continue to be opaque in their responses to Freedom of Information requests. In late 2010, they responded to 39% of requests by withholding information in part or in full; by the third quarter of 2017 they were doing so 52% of the time.

DExEU has a particularly poor record. In the four quarters to Q3 2017, it was the least likely of all departments to release information in response to requests. On ministerial correspondence, it has the sixth-worst response rate, despite giving itself longer to respond than most departments. On parliamentary questions, it has a below-average response rate. This poor performance reflects more than the growing pains of a new department, and is only part of a broader picture of opacity: ministers resisted the release (and then denied the existence) of assessments of the impact of Brexit on various economic sectors, and have been opaque about their negotiating objectives even as the EU has used transparency as a tool to set the agenda and get what it wants.[14]

Being open isn’t only about responding to requests, but also about proactive publication. In 2010, David Cameron mandated departments to publish datasets including spending items over £25,000 (monthly), organograms of their internal organisation (every six months), and the meetings attended and hospitality received by ministers (quarterly).

Departmental publication of £25000 spending

Publication of this data continues to be patchy. On spending over £25,000, fewer than a quarter of releases were published on time; a fifth have not been published at all. On organograms, more than half of departments have failed to publish their March 2017 data, and the departments created in 2016 – BEIS, DExEU and DIT – have never published any (although DExEU has published a few ‘senior team organogram’ diagrams). Departments have become better at publishing hospitality releases for their ministers within the three-month limit, but even these could be published more quickly.

Why does this patchy publication matter? It might imply a lack of political drive towards openness, or reflect a smaller civil service with fewer staff and other priorities. But it also suggests that few people in government are using the data within these releases.[15] This is also evident from the quality of some published data, the difficulties in linking and comparing different datasets, and the effort involved in turning data into information and insight. Information that should be everyday and essential, such as how departments spend their money, instead has to be chased across different publications and stitched together in spreadsheets, to give us even a chance of understanding what’s happening in government. As the Institute’s Performance Tracker shows, the same applies to data on everything from crime rates (where different sources contradict one another) to how neighbourhood services are performing.[16]

Government should ask what data it needs to inform decisions on its governing agenda and drive improvements in its administration. Parliament should be asking what data it needs to hold government accountable for those decisions. We will all benefit from a government that uses data to govern more effectively, and publishes data so we can understand them better. The alternative is making decisions – and leaving those outside government – in the dark.