As census failure blame points at IBM, why we shouldn't be surprised by its failings

The Australian online census, which failed so spectacularly last week has still not succeeded in having enough people complete the process for it to be rescued from irrelevance. Fewer than half of Australia’s households have so far completed the survey.

As previously reported, IBM is being held responsible in a large part for the major failings in preparation that led to a panic move by both it, and the Australian Bureau of Statistics (ABS), to shut the servers down.

For IBM, what is ultimately at risk is the business that they carry out for Australian Federal Government departments which total AU $2.4 billion.

IBM has already been banned from being allowed to work with Queensland State departments after its AU $1.25 billion payroll failure in 2013.

The fact that IBM may be at fault in this particular software project should not come as a surprise. Firstly, software projects are generally hard and there is evidence that a majority will fail for a variety of reasons, but a principle one being failure to capture all of the requirements correctly. This was certainly the cause in the poor preparation for the eCensus project.

A more important observation about IBM however is that this is a company that is still in the throws of trying to reinvent itself whilst fighting declining revenues as it shifts from selling hardware to software and services. IBM’s year-over-year sales have declined consistently over the past 4 years.

IBM’s battle of reinvention was catalysed when it jettisoned its PC business in 2005, selling the division to Chinese PC company Lenovo in that year. Most of IBM’s server hardware business followed to Lenovo in 2014. There are rumours that IBM will even exit the mainframe business, potentially selling that to Japanese firm Hitachi.

Instead of building hardware, IBM has been developing its cloud business and a major initiative around what it calls “cognitive computing”. This latter aspect of the business is based on the artificial intelligence capabilities developed around its Watson) system. The service, selling for millions of dollars, allows customers to interrogate unstructured data such as that typically found in documents and spreadsheets. It has been used in healthcare to allow doctors , for example, to search research papers relating to a particular clinical trials and provide insight into particular cancers.

IBM is trying to grow its cloud, cognitive and analytics capabilities by rapidly buying other companies. It has made 11 acquisitions so far this year for a total of US $5 billion, including The Weather Company, along with all of the weather data it has amassed.

One of these acquisitions was SoftLayer, the cloud computing company that hosted the eCensus services that in part failed last week.

IBM’s success in convincing government departments to award it contracts relies on its ability to trade off its reputation as a large, solid and trustworthy company that has access to a network of futuristic functionality provided through Watson and its acquisitions. In other cases, such as the ABS, it relies in part on the use of IBM’s software, such as products like Lotus Notes and Websphere. The ABS built its eCensus system around Websphere software which also used elements of Lotus Notes.

The end result however is not always coherent, and at the end of the day, still relies largely on the capabilities of the individual IBM employees assigned to projects and less to the collection of technologies IBM is trying to sell, some of whose benefits may be more hype than reality. From the perspective of writing software that works, IBM doesn’t have a magic formula for success, nor can it control the quality of the clients that it has to engage with, especially in the civil service. There is pressure for it to bid for government work and to potentially underbid others, even if this means cutting corners to save money, as may have been the case in the decision not to take up additional protection from denial of service attacks for the eCensus project. This means that it will fail in software projects just like any other consulting firm.

What IBM may have to accept blame for however is that knowing that all software projects can fail, and knowing its own limitations, it didn’t have a mitigation strategy in place in the event of failures. For a project like the online census which had such a high profile, that is far less excusable.