# Here's how to make opinion polls more representative and honest

• Written by Adrian Barnett, Professor of Statistics; President of the Statistical Society of Australia, Queensland University of Technology

In 2012, US statistician Nate Silver correctly predicted the results of all 50 states in the US presidential election with 100% accuracy.

Compare this with the shock result of the 2016 Trump election, and now the 2019 Australian election, when nearly all Australian opinion polls incorrectly predicted that Bill Shorten’s Labor Party would defeat Scott Morrison’s Coalition on a two-party preferred basis.

How did the pollsters get it so wrong? More importantly, how can we ensure we get a more accurate result next time?

## What opinion polls must do

A good opinion poll must do two difficult things:

1. it must find a large and representative sample of Australian voters
2. it must get those voters to honestly divulge who they will vote for.

If both of these things happen, statistical theory gives us confidence that the poll will be reliable.

To add to the difficulty, we don’t just need good opinion polls we need excellent opinion polls, because even a small error could mean predicting the wrong winner.

For example, if a government ministry wanted to know what percentage of people had done volunteer work in the last year, it probably wouldn’t matter for their purposes if they commissioned two polls which gave estimates of 48% and 52%. But this variation is unacceptable for electoral opinion polls, which need a small margin of error to accurately predict the winner.

## How to get a representative sample

A good representative sample will be the whole Australian voting population, but in microcosm. If there are 16 million eligible voters, then a representative sample of 1,600 would mean that each sampled voter represents 10,000 people just like them.

Using relatively small samples is a cheap way to estimate the bigger picture of the election, without resorting to expensive surveys like the Australian same-sex marriage survey.

With a large and representative sample, statistical theory tells us that we will get close to the real result with a narrow margin of error. But the opinion polls in this election consistently predicted the wrong winner, meaning there was something wrong with the samples.

## The problem of disruptive technology

Opinion polls are usually implemented by pollsters calling landlines, sometimes calling mobile phones, robo-dialling and internet surveys.

In the recent past when almost every Australian had a landline phone, it was far easier to get a representative sample of voters by randomly calling numbers or randomly sampling from the White Pages. The randomness is key, because it destroys bias by giving every eligible voter the same chance of being surveyed. This in turn avoids favouring particular groups.

Modern methods of communication have spoiled the representativeness of landlines – 36% of Australian adults now use a mobile only. And because older Australians are more likely to keep their landline, this biases any sample that uses landlines alone.

Far more Australians have a mobile phone than a landline – so why not use these instead? One reason is that some people have two mobiles – business and personal – so the poll would over-represent working Australians. And with fewer older people having mobiles, this approach would also over-represent younger Australians.

Even if we had a perfectly representative sample of phone numbers, we can expect a large number of people not to answer or hang up, which creates another bias because the poll then over-represents people with more time and those who are more politically engaged.

## How to get honest answers

Opinion polls have been wrong before. Two examples are the 1992 and 2015 elections in the UK, where the polls wrongly predicted a good result for Labour. The post-mortem of these failings came up with the theory of the “Shy Tory factor”, where people were too embarrassed to admit to pollsters that they were voting for the Tories (the Conservative right-wing party).

Plenty of Australians are not embarrassed to tell anyone who they will vote for, but even the proportion of shy voters is small, then they could still derail the accuracy of the poll.

A number of Australian voters have admitted to giving fake answers to telephone polls because they were annoyed by the call. Even a small percent of annoyed people could ruin an otherwise well-designed opinion poll.

Another theory – put forward by Australian Nobel Laureate Brian Schmidt – is that the pollsters listened to each other more than they did the public. This is known as “herding”, where poll results that seemed to far from the norm were shepherded towards a groupthink.

An analysis of the numbers does show a striking lack of variability in the many recent polls.

## How to get better polls

Better opinions polls are more expensive because pollsters need to spend more effort getting a representative and honest sample of voters. Some of the quick and easy polls that are currently used are primarily aimed at generating news stories, with accuracy as a secondary goal.

Poll accuracy could be increased by paying people to participate, and paying them more if, as a group, they get the result right. This is an untested idea, but it could create a strong incentive for people share their true intentions.

The same argument could also improve the pollsters’ strategies. That is, create financial rewards for getting things right, or penalties for getting them wrong. However, with shrinking budgets at news outlets to finance polling, there is no reason to believe that this problem is going to fix itself.

Read more: Going up. Monday showed what the market thinks of Morrison

## Polling is a difficult job

An opinion poll targets the shifting opinions of an ever-changing population. Voters are often difficult to reach or recalcitrant. Yet much of the population is likely unhappy with the polls. They may never pay heed to polls again, or refuse to participate in them, which will ironically make the pollster’s job even harder.

Election opinion polls are like penalty shoot-outs at a World Cup final: there’s huge pressure to get it right and we remember the big misses most of all. But come the next World Cup we’re back watching the penalties, because nobody has come up with a better system to assess the national mood.

Authors: Adrian Barnett, Professor of Statistics; President of the Statistical Society of Australia, Queensland University of Technology

Coronavirus disrupted my kid's first year of school. Will that set them back?

arrow_forward

What are manufactured home estates and why are they so problematic for retirees?

arrow_forward

Things to Ask To Your Removalists

arrow_forward

### Politics

#### Did BLM Really Change the US Police Work?

The Black Lives Matter (BLM) movement has proven that the power of the state rests in the hands of the people it governs. Following the death of 46-year-old black American George Floyd in a case of ...

a Guest Writer

#### Scott Morrison: the right man at the right time

Australia is not at war with another nation or ideology in August 2020 but the nation is in conflict. There are serious threats from China and there are many challenges flowing from the pandemic tha...

Greg Rogers

#### Prime Minister National Cabinet Statement

The National Cabinet met today to discuss Australia’s COVID-19 response, the Victoria outbreak, easing restrictions, helping Australians prepare to go back to work in a COVID-safe environment an...

Scott Morrison

### Business News

#### Ten tips for landing a freelance transcription job

Transcription jobs are known to be popular in the field of freelancing. They offer fantastic job opportunities to a lot of people, but there are some scammers who wait to cheat the freelancers. ...

News Company

#### How To Remove Rubbish More Effectively

It can be a big task to remove household rubbish. The hardest part is finding the best way to get rid of your junk. It can be very overwhelming to know exactly where to start with so many option...

News Company

#### 4 Tips To Pass Skills Certifications Tests

Developing the right set of skills is valuable not only to your career, but for life in general. You can get certified in these skills through obtaining a license. Without a certified license, y...

News Company

# News Company Media Core

Content & Technology Connecting Global Audiences

More Information - Less Opinion