Using parliamentary data

There is much about Parliament’s work and role that cannot be quantified – the quantity of parliamentary activity should not be confused with its quality.[1] As this report shows, context and nuance matter when it comes to Parliament. Nevertheless, parliamentary data is still useful. It can help those within Parliament to understand how they are working, and whether there are processes or procedures that might be improved. More than this, data can help Parliament to explain to the public what it does, and why it matters.

There is no lack of data on Parliament and its work: this report largely draws on publicly available information, brought together from a range of sources. But while there is a wealth of data available, using it is harder than it could or should be.

Parliamentary data is often fragmented, with multiple sources of data that don’t quite match up – making it difficult to draw a clear and consistent picture. And the data itself is published in formats that make analysis harder. These issues are not peculiar to parliamentary data: the Institute has previously identified similar problems with government data.[2] But differences in the ways in which the two Houses and different parliamentary bodies and departments produce and use data seems to make these problems particularly acute. Addressing these issues will take time, but doing so will help Parliament to make better use of data to understand its own work and effectiveness – and to show the public what it does, and why it matters.


Data is fragmented – and doesn’t always match up

A key challenge with parliamentary data is its fragmentation. Data can be drawn from a number of sources: from Hansard to publications by each House’s libraries; from the website to the sessional returns and business statistics published by the Commons and Lords. Data may be contained in select committee webpages or in committees’ reports. It can also be found through the search function on Parliament’s website. These data sources may count or define the same thing differently, in ways that are not always obvious. All of this means that a vast array of data exists – but it is data that does not always match up.

Data on secondary legislation is a good example of this fragmentation, as the Hansard Society have previously found.[3] Some data focuses on all secondary legislation, broadly defined, while other data is only concerned with secondary legislation subject to parliamentary procedure. Some data sources count pieces of secondary legislation on a calendar year basis, while others use parliamentary sessions as their time period. The range of different approaches to the data makes it difficult to piece together the progress through Parliament of a particular piece of secondary legislation.

There are some good reasons why the data on secondary legislation – and parliamentary data more broadly – is so patchy. Key to this is the structure of Parliament itself. There are two Houses, which fulfil different roles and have different rules and procedures. This means that each House publishes its own statistics on its business each session, and its own annual report and accounts – and these are not necessarily directly comparable. And even within each House, there are users and producers of data who may approach the same topic from different perspectives, due to their differing roles or interests. Without any central control, different sources of data have evolved over time.

The Parliamentary Digital Service, which works across the two Houses, can help play a positive role in providing more consistent data. Its development of the website is an important step towards bringing data together in one place, and its recent work on a new Statutory Instrument tracker will help bring some greater order to the data on secondary legislation.

Although it will take time and effort, there is potential for cross-parliamentary thinking about why and how data is produced and used, to bring currently fragmented data sources together into a more coherent whole.


The presentation of data can make it harder to analyse

A second challenge of using parliamentary data lies in the way it is published and presented.

Key data on the activity of both Houses is published at the end of each session, in sessional returns and business statistics. But this means that collecting and analysing data at other points – for example, in the middle of a two-year session – is difficult. Obtaining data on an ongoing basis, for example on the daily sitting times of each House – requires manual collation, which is laborious and increases the risk of error. The search functions of various data sources do not consistently return accurate results. While it is possible to request data from Parliament, doing so takes up the time of staff.

Publishing much data at the end of each session makes sense, and is a valuable service. But the difficulties of working with data on an ongoing basis make it harder for those outside Parliament to keep track of what it is doing. It also raises questions about the ability of those inside Parliament to analyse and assess their work as a session progresses. Focusing data collection and publication at the end of the session risks implying that data collection is an additional task to be completed, rather than something that is useful for Parliament in its day-to-day work. It would be helpful to consider whether there are ways in which some pieces of data could be regularly updated in easily accessible formats – for example, as with Parliamentary Information Lists compiled by the House of Commons Library, that include spreadsheets of data on proceedings such as emergency debates, which are updated on an ongoing basis.

The specific format in which parliamentary data is published can also make it harder to analyse. Much data, including sessional returns, is published in PDF format, rather than in spreadsheets that can be downloaded. This may seem a minor point, but putting data in these formats adds an extra, time-consuming hurdle to analysis. It can also increase the opportunity for error, as the data has to be transferred from one format into another. The site – which allows data to be downloaded into spreadsheet format – is a step in the right direction, as is the Commons Procedure Committee’s publication of spreadsheets of data on Parliamentary Questions, but more could be done in many other areas to publish data in more usable formats.


Parliamentary data can be further improved

The quality, consistency and usability of parliamentary data need to improve. But a change in attitude to data is also required across Parliament. The more that those responsible for producing data value it, and recognise its utility, the better the data is likely to be. There are many positive signs, from the ongoing work of the Parliamentary Digital Service, to the efforts of individual teams across Parliament to use data to improve their effectiveness. In subsequent editions of Parliamentary Monitor, we look forward to highlighting further improvements in the quality of parliamentary data.