The problem(s) with public sector IT


This post discusses the frequently poor IT provision and projects which seem to dog the public sector. Some tentative explanations are offered as to why these failures might keep happening, and some suggestions are made as to how public sector organisations might benefit from top-notch IT provision in future.

The history of government IT projects is littered with futile mass expenditure, from the NHS (“Abandoned NHS IT system has cost £10bn so far” – Guardian 18 Sept 2013), to the criminal justice system (“Anger at bill for scrapped penal agency” – Observer, 27 Jan 2008). It is no exaggeration to state that billions of pounds have been wasted on ill-conceived and mismanaged IT projects with grand aims. These projects have generally either been scrapped after massive budget and deadline overruns, or have been reduced in scale to the point where they are a fleeting shadow of their former intentions.

My own experience in this area is surprisingly extensive – I am not a software developer per se, though I do develop data analysis tools – my experience in an (arms length) government department, a research and management consultancy, and as a freelance consultant, have frequently been affected by poor quality IT systems, and ambitious but fundamentally flawed attempts at replacing them.

Legacy systems

It seems that the public sector is constantly playing catch-up in terms of the modernity of the software that it uses. I believe that there are three main reasons for this:

  • the scale of required changes;
  • fear of disruption/failure; and
  • contractual tie-ins.

Some of the biggest cancellations in recent years have been related to the largest public sector organisations in the UK (e.g. the NHS and the criminal justice system), which have huge numbers of staff and hundreds of locations in the UK. Not only does any system necessarily need to involve a vast network deployment, but it also needs to meet very stringent security standards.

These organisations provide critical public services. Any failure in IT may result in severe reductions in service level and quality, and could even cause crises or additional mortality. As a result (and reinforced by the long history of failed IT projects), public sector organisations are reticent to change from systems that may be slow and basic, but are proven reliable.

There is a long history of public sector organisations being contractually locked in to poor IT. This is another manifestation of the desire for secure and reliable provision, but has frequently resulted in organisations being stuck with outdated, slow, restrictive, and expensive IT.

Use of standard and off-the-shelf software

As a result of being tied-in to restrictive legacy systems, many public sector organisations end up either improvising with general office software or buying off-the-shelf software that is an improvement, but doesn’t quite meet their needs. Inevitably, this creates inefficiency in business processes, and can sometimes result in catastrophic data failures.

My own experience in this area is telling. I once developed a database case management system, designed in MS Access, because the organisation needed a stop-gap until a new, fancy, bells and whistles government IT system was in place. Of course, the project to put this new system in place massively overran, and it was eventually cancelled. The Access database, which was designed to be used for a year or two at the most, was still in place years later!

Anyone familiar with MS Access knows it should not be used for extended periods for critical processes. It can be temperamental, is prone to corrupting data, and does not manage well with large storage requirements and large numbers of users. Other issues occur with standard software when user error is involved. Many times in the past I have come across distraught administrators who have messed up an entire dataset by sorting their data in MS Excel incorrectly and saving the file!

New systems and project creep

One of the other main issues with attempts at developing new systems in the public sector relates to extreme project creep. Because the systems tend to have large scope – being used by very large numbers of staff, in a multitude of teams and organisations, with different requirements in different locations – the development of a new system can easily start to resemble a chaotic shopping list.

The cause of this problem is usually the mismatch between the world of the operational staff, and the world of the developers. Operational staff want particular things from a new system, but cannot easily translate their requirements into a specification for a good software product. In contrast, IT developers do not generally have a deep understanding of the roles and work of all of the teams of staff in the client organisation, and sometimes do not interrogate and examine the stated requirements sufficiently to reach an optimal solution.

It can be like having a toddler in a supermarket asking for everything, and the mother just letting the kid fill the trolley: the end result is a terrible diet and poor health! For IT systems to be developed well, there needs to be a project manager involved who can speak both languages – someone who understands the IT side of things and is also experienced in the organisation’s work and processes; a carer who knows what’s available, knows their child well, and can moderate their demands to reach what’s best for them.

The way forward

We now live in a world where IT development is required for all organisations and services. The software giants (Google, Microsoft, Amazon, etc.) are increasingly providing extensive ‘software-as-a-service’ solutions, which provide powerful, scalable architecture at incredibly competitive prices, with no need for the organisation to have its own servers.

It’s clear that traditional government IT suppliers have failed to provide cost-effective, top-notch solutions. Some of the best software is now developed by small IT companies with small teams of highly skilled developers. The skills of these developers is what matters – with expertise comes speed and high quality. Because these companies are small, they have low overheads, need to compete to survive, and offer far better value-for-money than the large, traditional government IT suppliers. These skilled developers can choose where to work, and they are choosing to work for themselves.

Taking all the above into consideration, I believe there are three elements which would lead to successful IT development in the public sector:

  • the public sector would do best by seeking the services of these new, small IT companies who are at the forefront of web software development;
  • contracts need to be designed to give the public sector ongoing support and reassurance, ultimate ownership of its IT, and modern systems which are regularly refreshed and move with the times; and
  • project managers with extensive sector and IT experience need to be involved in translating between the developers and the public sector.

We are living in a time of change, where the whole ecology of IT and web development is constantly on the move. In order to make best use of what’s available, it is necessary to be as ‘agile’ as this ecology. Public sector organisations need to be on the ball, move with the times, and take advantage of what’s now available. With support, this is entirely feasible; after all, it’s not rocket science!