Daily BulletinDaily Bulletin

The Conversation

  • Written by Simon Williams, Lecturer in Mathematics, Flinders University

Since it announced a crackdown on outstanding debt in June last year, Centrelink has sent debt recovery letters to thousands of Australian welfare recipients. Early reports indicated that around 20% were issued in error, although the true number may be substantially higher.

This led to the appointment last week of a Senate committee to investigate why so many erroneous letters were sent.

While there has been some controversy over the government’s reliance on big data to save taxpayers’ money, none of the commentary has come close to understanding the complexities hidden in the notion of the error rate.

The system that Centrelink employs is an example of artificial intelligence, and the problems it faces are intrinsic to all decision systems.

Working with error

Mathematicians, computer scientists and engineers have long grappled with these problems and developed ways of quantifying the errors, as well as determining the point that balances their relative importance.

The same mathematics is used to describe radar systems, disease screening programs, and speech recognition systems such as Siri and Google Cloud Speech.

But is it an error in itself to talk of a single error rate for such systems. That assumes there is only one way to make an error, or that all of the different types of errors are equally bad, which is almost never the case.

For example, there are at least two ways the Centrelink system can make a mistake in its job of sending letters to debtors. It can send a letter to someone without a debt or it can overlook sending a letter to someone who does have a debt.

The former error is a false positive, and the rate at which a system avoids this type of error is called the sensitivity. The latter error is a false negative, and the associated rate is called the specificity.

image How many of those radar blips are not really aircraft? Flickr/NATS Press Office, CC BY-NC

Radar systems, which use radio waves to detect aircraft, are where this notion of multiple sources of error was first formalised. It is important not to miss a detection of aircraft (false negative) but the avoidance of mistaken detections is equally important.

Having to constantly deal with incorrect reports is fatiguing for the operator and may cause them to discount valid detections – like a modern version of The Boy Who Cried Wolf.

Screening errors

Another example of the importance of multiple error mechanisms is in breast cancer screening programs.

While it is good to catch cancers early, it is also important to correctly clear women without breast cancer due to the psychological harm caused by the stress of an incorrect diagnosis.

These systems use a type of machine learning called classification. A classification system uses all of the available data to decide into which class a particular case falls. We have already seen that for a two-class problem (Centrelink: letter or no letter) there are two ways to make an error.

But it gets worse: for a three-class problem (a cancer test that decides between clear, benign and malignant) that number rises to six. The more decisions a system makes, the more ways it can fail.

Nobody’s perfect

It is not possible to make a perfect system. Errors in data entry, natural variability and human error all play their part in blurring the line between debt and no debt, malignant or benign, detection or false alarm.

The following sequence shows how a mathematician approaches the decision process using a simplified model of the Centrelink system, where we cross-reference someone’s declared income against their actual income. The following sequence uses simulated data that assumes only that debtors’ actual income is higher than their declared income.

image An animated explanation of how mathematicians understand how to measure the performance of a simplified version of the Centrelink robot-debt system.

But the intrinsic uncertainty in the data, and the freedom we have to choose the decision line, shows the impossibility of simply defining the error rate.

What we can do is summarise the performance of the system using a method known as receiver operating characteristic (ROC) where we move the decision line and record the true and false positive rates for each position.

image This chart shows that as the number of debtors correctly identified increases, so too does the number of non-debtors who are incorrectly identified as debtors. Author created

The true positive rate is the percentage of debtors sent a letter, and the false positive rate is the percentage of people incorrectly sent a debt letter. This graph is generated by marking all true positive/false positive pairs for each possible decision line against the same simulated data used earlier.

There are two things we can see from this graph: if you want to identify all of the debtors (100% true positive) you will also catch 90% of the debt-free in your net. This point is marked A on the curve. Furthermore, if you do not want to write any unnecessary letters (0% false positive) then you will only find 38% of those who should pay back a debt (marked B).

As usual, a trade off is unavoidable, and the sensible way lies somewhere in between. That’s why you have to accept that there will always be some errors in your system. The challenge is how to balance them, and how to deal with them whey they are occur.

Why now?

One puzzle remains for the Centrelink system. The data matching algorithm has been in use for several years in a manual form, but only now is it making headlines.

The answer lies in the expansion of its application. The automated system sends out 20,000 letters a week, a 50-fold increase on the the manual system. Broadening the population to which a system is applied has diminished its effectiveness.

Imagine we have a test for breast cancer that is 99% accurate at identifying cancer, and clears those without cancer 99% of the time. This sounds like a great test, but when we use it for mass screening, such as applying it to every Australian, things will go wrong.

The prevalence of breast cancer in Australia is 61 per 100,000 people. This means the test will correctly identify all but 14 of the approximately 1,400 new cases in the country each year.

But by the same token, 230,000 people will be incorrectly diagnosed with cancer and recalled unnecessarily for further expensive and invasive testing to clear them completely. That’s not to mention the stress and worry these incorrect diagnoses would cause. It would rightly be denounced as a disaster.

The actual screening program avoids this trap by narrowing its application to women over 50, which is a small enough population for whom the prevalence is high enough to make screening useful.

Lessons to learn

The Human Services Minister, Alan Tudge, says that sending a letter to a non-debtor is not an error.

But, as we have seen, downplaying these errors and concentrating only on identifying more and more debtors magnifies the number of this second type of error.

Also, expanding the data-matching system and removing the human element from the case-selection process has undermined the system’s performance even further.

By underestimating both the number of errors possible and the effects of their interaction, Centrelink is left dusting itself off after having been hit by a big steaming pile of data.

Authors: Simon Williams, Lecturer in Mathematics, Flinders University

Read more http://theconversation.com/errors-in-centrelinks-debt-recovery-system-were-inevitable-as-in-all-complex-systems-71409

What's in a name? Well, quite a bit if your name is Karen (or Jack, John, Jeff, Dolly, Biddy, Meg ...)


Don't know what day it is or who said what at the last meeting? Blame the coronavirus


Secondary school textbooks teach our kids the myth that Aboriginal Australians were nomadic hunter-gatherers


The Conversation


Scott Morrison: the right man at the right time

Australia is not at war with another nation or ideology in August 2020 but the nation is in conflict. There are serious threats from China and there are many challenges flowing from the pandemic tha...

Greg Rogers - avatar Greg Rogers

Prime Minister National Cabinet Statement

The National Cabinet met today to discuss Australia’s COVID-19 response, the Victoria outbreak, easing restrictions, helping Australians prepare to go back to work in a COVID-safe environment an...

Scott Morrison - avatar Scott Morrison

Prime Minister Interview with Tracy Grimshaw

TRACY GRIMSHAW: Prime Minister, thank you for your time.    PRIME MINISTER: Great to be here. Thank you for the opportunity.    GRIMSHAW: A month or so ago, you probably thought that today's...

Scott Morrison - avatar Scott Morrison

Business News

Instant Steel Solutions Review

Are you keen on having the right guidance, knowledge and information about the right kind of steel purchases for your industries? If yes, then you are in the right place. There is no doubt that ...

a Guest Writer - avatar a Guest Writer

Everything You Need To Know About Waste Removal Services 

Waste is capable of posing threats to the environment and general public health. So, if you want to live a healthy life, you need to take care of your waste products. Proper collection and dispo...

News Company - avatar News Company

How Forklift Rental Can Assist During Unexpected Upturns

Although some companies might balk at the prospect of hiring a forklift for the work they need to get done rather than buying it, hiring a forklift can fill a very specific need: being able to m...

News Company - avatar News Company

News Company Media Core

Content & Technology Connecting Global Audiences

More Information - Less Opinion