Learn from failure: the key lesson that Universal Credit should take from agile

5 September 2013

At its outset there was considerable hope that Universal Credit would signal a break with the large Government IT failures of the past. It was held up by DWP as an example of the department adopting new ‘agile’ ways of working. But as the new NAO report makes clear, serious improvements need to be made if Universal Credit is to fly.

Introducing Universal Credit was always going to be a “courageous” undertaking. There were the technical and managerial challenges of simplifying complex benefits, together with large scale change to IT and working processes. Furthermore the project aims were also set very high – to change culture among benefit recipients while making sure the most vulnerable people did not slip through the net. And all of this was to be implemented at a time when budgets, and staff resources, were being dramatically reduced.

Some were sceptical from the start – not least the Treasury. But this was the project that Secretary of State Iain Duncan-Smith came into government to implement – and it was driven on a wave of political momentum.

The government’s track record of IT disasters meant that there were warning lights flashing from the start as it was a project that depended on major IT change. There was a real push inside and outside government to change the way government ran IT projects, as we set out in our 2011 report, System Error. We, along with others, welcomed the news that Universal Credit would be run differently to past projects, using agile techniques to deliver iterative, modular functionality that was tightly focused on the needs of users. So should today’s NAO report force us to reconsider the potential for agile? And are their other wider lessons to be learned about policy implementation in government?

After a brief blooming of openness about how Universal Credit would be run, things then went quiet. Despite intermittent rumblings of problems surfacing from time to time in the media, the department continued to inform Parliament that everything was proceeding as planned (Tony Collins provides a full account of this on his blog). Then, last year the Major Projects Authority (MPA) graded the programme Amber/Red. According to the MPA this rating meant: “Successful delivery of the project is in doubt, with major risks or issues apparent in a number of key areas. Urgent action is needed to ensure these are addressed, and whether resolution is feasible.”

Yet the department’s responding narrative was entirely positive and made no reference to why it had received this rating. Nor did it refer to the reset that was immediately carried out to try and get the programme back on track. The NAO’s new report confirms difficulties and sheds further light on their causes. DWP has responded that they are “committed to delivering Universal Credit on time by 2017 and within budget, and under new leadership we have a plan in place that is achievable”. Time will tell whether the actions taken will be sufficient but it is important that we learn the right lessons from experiences to date.

Why did no one see there was a problem earlier?

One of the key failures that the NAO identify is a lack of transparency and information on key elements of the programme. It states: “The board did not have adequate performance information to challenge the programme’s progress” and also describes a ‘good news’ reporting culture. There was also a lack of information to support financial governance and the assessment of supplier performance. This is something that the programme’s latest chief has acknowledged and intends to reform. In the Institute’s Improving Decision Making report, we argue that demand for information is a key driver of quality information. So the board, the centre, Parliament and those holding government to account also have a role to play in demanding better information.

So did agile fail in Universal Credit?

The NAO suggests that DWP struggled to incorporate agile into its existing contract, governance and approval structures. DWP then decided instead to go for a hybrid ‘Agile 2.0’ before switching again to a ‘phased approach’. Given a core component of agile projects is regular results being presented and feedback openly feeding into the next iteration, the NAO’s remarks about lack of information and challenge suggest that agile practices were not really happening. The NAO concludes: “The Cabinet Office does not consider that the Department has at any point prior to the reset appropriately adopted an agile approach to managing the Universal Credit programme.” So it seems more that Universal Credit failed to be agile, rather than agile failing Universal Credit.

Does this mean agile can’t work in government? Quite simply, no. Agile is extensively and openly used by the Government Digital Services (GDS) to improve a range of government services. GDS are even now working with DWP to see if they can get help get things back on track in the department.

Other problems

But other problems have bedevilled Universal Credit’s development. Our report on the Olympics suggested lessons government should learn if it wanted to improve its track record on delivering big, complex projects:
• Bring together the right people, with the right skills and experience in effective and stable teams. DWP have been unlucky, but they seem now to be nearer to creating the team they need.
• Get the scope clear at the start – even if it appears to delay the project – and then be transparent about milestones, budget and performance against them. This was crucial to rebuilding public confidence in the Olympics – and this is what Universal Credit needs now.
• Devote time, effort and money to good project management and assurance. The Olympics did this and Universal Credit needs to do so too.
The Olympics showed it was possible to recover from a false start. Universal Credit needs to do that rapidly if it is not simply to become another exhibit in the policy failure hall of fame.

Accountability

Finally, the story of Universal Credit shows how difficult it is to disentangle accountability. Was the policy “feasible”? Should the Accounting Officer have demanded a direction from a determined and new secretary of state? Or was the failure all one of delivery for which the Civil Service should be castigated – as Iain Duncan-Smith has done today? These issues will be addressed in our forthcoming report on accountability.

Share this post:

Comments (7)

  1. Chris Wheeler on 5 September 2013 at 8:47 pm

    Every time a major business change is described instead as an IT project, you know it’ll struggle, whether in public service or private companies. IT can only be an enabler in these kind of changes.

    IT projects are things like upgrade networks, upgrade email systems, replace one system with another. They rarely have the same struggles.

    What something like Universal Credit needed was simplification of the regulations, so that they use the same measures for delivering benefits, then harmonisation of processes, then develop the IT to deliver them more efficiently. IT cannot be a goal in itself.

  2. Tim Blackwell on 6 September 2013 at 10:33 am

    It would be a kindness for the Cabinet Office to define what they mean by “appropriately agile”, as this is not obvious.

    ‘Agile’ methods may have value in themselves – and over and above that as provocations to schlerotic institutions, but Agile is not a panacea and ought never to be the object of infatuation. I’m sometimes reminded of Flann O’ Brien’s Dublin Gaels, who were never so truly Gaelically Gaelic as when they spoke at length in truly true Gaelic Gaelic about the Gaelic language.

    In particular Agile promotes continuous iteration: wrongly applied this can make for a horrible user experience. System interfaces too should not be iterated beyond necessity. They should support change, by being tightly delineated, loosely coupled and properly versioned. But they should change as rarely as possible.

    I don’t know the inside story of Universal Credit, but I wonder if Fred Brookes’ Mythical Man Month might be more apposite than the Agile Manifesto – not that the two are entirely opposed. Brookes talks about getting the architecture right before starting coding. Architecture has deservedly gotten a bad reputation – because it panders to our desire to live in castles of abstraction, and because more than a page or two of UML exhausts the will to live, but it’s still a good idea to know what we’re going to build before we start digging. Perhaps a construction engineer is the right person after all.

  3. DT on 6 September 2013 at 1:39 pm

    It’s frustrating that the problems identified in the report were listed by the NAO and OGC as common causes of project failure many years ago. This keeps happening.

    Particularly egregious is not having a “detailed view of how Universal Credit is meant to work” through 2011 and the first half of 2012.

  4. Tim Blackwell on 6 September 2013 at 4:27 pm

    “What something like Universal Credit needed was simplification of the regulations”

    The regulations have been hugely simplified compared to those which apply to the benefits which Universal Credit replaces. Some people think they are too simple – perhaps inadequate to cover the full diversity of claimant circumstances.

    “IT can only be an enabler in these kind of changes.”

    It can also be a canary.

  5. Andrew Lainton on 13 September 2013 at 9:04 am

    Interesting post – you state ‘Get the scope clear at the start – even if it appears to delay the project’ but isnt this the very definition of a waterfall approach to project management rather than an agile approach? Any project with a complex web of outsourcing isn’t suitable for agile – rather agile has to be done in house working with those at the front line in teams so business processes can be changed alongside the coding. Would appear to be another example of failure of outsourcing where the specification of what was being outsourced as not understood but it was not tested to be achievable.

  6. Tim Manning on 13 September 2013 at 10:26 am

    In response to the last comment. By “scope” we need to mean high level design. It’s a misconception that agile means that you can just start building stuff, without any idea about what (architecturally) the finished design is intended to look like.

    As mentioned in the NAO report and at the PAC session this week, the “reset team” are busy ‘reverse engineering’ (my phrase) a “blueprint/target operating model”, presumably out of the various pieces that have been designed, built and partially deployed to-date.

    This exercise should provide a firm high level design (of hopefully the scheme and service operating model) against which the programme can continue; and also identify what should be kept and what will need to be thrown away.

    With this firm foundation in place, then and only then, can an incremental and iterative development approach be applied – the only way to deliver something of this nature.

  7. Nik Silver on 13 September 2013 at 10:28 am

    Andrew Lainton makes some interesting comments…

    I think “getting the scope clear from the start” isn’t necessarily anti-agile. You need to decide what is in and out of scope in order to determine your end goal and your interim milestones. It’s then okay (and sensible) to change that in the light of continuous feedback — with appropriate governance, of course. Determining scope is fine. Fixing scope — particularly at the low level — is bad.

    Outsourcing definitely makes agile more difficult, because you introduce the complexity of contracts, and all the problems associated with programme goals versus contractual goals, which it seems Universal Credit has demonstrated. I wouldn’t say outsourcing is wholly incompatible with agile, but it certainly adds another layer of difficulty.

Blog: Permalink