ABS's night of disaster as servers crash and millions fail to complete the census

The Australian census has proved a disaster tonight as the Australian Bureau of Statistics (ABS) threw up its hands and declared that its online service would not be available on the night that the census was due to be completed and people should try again in the morning.

After struggling to convince the Australian public that its most private data was safe in their hands, the crash of its online service couldn’t have come at a worse time.

Again, to be fair to the ABS, the fault is shared with IBM who won the AU $9.6 m contract to host the census this year.

The ABS had tested the application at 150 percent of the expected load and paid Revolution IT AU $470,000 to carry out these tests. This lead to the claim that the website would be able to handle 1 million form submissions every hour. This sounds impressive but it actually equates to only 277 a second. When nearly 11 million people are accessing the site at roughly the same time, this would actually be a very small number.

Another issue with load testing is that it is hard to replicate the exact nature of how millions of users will actually try and access the site. Load testing usually just ramps up the number of accesses to the site evenly whereas the demand from real users is unlikely to be this even.

What the ABS didn’t count on is the fact that slow response times and failures cause people to constantly refresh or resubmit their forms and this just makes matters worse.

it seems inconceivable that a company like IBM would underestimate the expected load on their servers so badly. One can only assume that there is more to the story than is apparent at this time. IBM is still tarnished from its association with the AU $1.2 billion failed Queensland Health payroll project.

For the ABS however, they have not only damaged their own reputation and their ability to convince anyone to take seriously any of their technical claims, but they have brought into question the ability of any government agency to be able to run technology projects of this scale. This is specifically relevant given the recent discussions about running elections online.

From a technical perspective, it is not clear whether the failure was simply a result of the load on the services or actual bugs in the underlying system. The fact that the servers are now offline and won’t be restored until tomorrow suggests that the issues were more fundamental than just being overloaded.

It is also possible that given the publicity involved, the sites may have been subjected to some specific effort to crash them, including the possibility that they were targeted by hackers running a denial of service attack.

In any event, the ABS has proved that its ability to communicate with the Australian public in an effective way has been as deficient as its technology. There is also a clear question as to whether the entire census will need to be rerun as it has already been severely compromised.

Would anyone actually still trust the results that come from the census as it has been run so far?