Daily Bulletin

The Conversation

  • Written by David Tuffley, Senior Lecturer in Applied Ethics and SocioTechnical Studies, School of ICT., Griffith University

Social media is viewed as a potential goldmine of information. The key is to work out how to mine this abundant source of public sentiment.

Linking social media sentiment with human behaviour is a relatively new and evolving field of study. It has a lot of potential – we successfully used it to predict the result of the 2016 US election.

But we got it wrong with Australia’s same-sex marriage survey, and here’s why.

Read more: What the numbers say (and don’t say) in the same-sex marriage survey

We crunched the numbers

We carefully sampled the sentiment of 458,565 anonymised Australian tweets that made reference to same-sex marriage. We found 72% overall support for Yes. This was averaged out from the whole month of October.

But we noticed that some Twitter accounts had sent more than 1,000 tweets related to same-sex marriage. The number of unique users was down to just 207,287.

It seemed wise to minimise the influence of these bulk tweets because by the time they were sent, many of the votes had already been cast. Discounting the influence of the bulk tweets brought Yes support down to 57%.

Once we adjusted another 8% for the under-representation of the over-55 demographic in the Twitter sample, we concluded that the overall support for Yes was down to 49%.

With the benefit of hindsight

In previous successful trials we had assumed that all tweets are equal. If we had made the same assumption in this trial and did everything else the same, then – re-crunching the numbers – our prediction for Yes would have been 59.08%, which is close to the official result of 61.6%.

We made the incorrect assumption that the bulk tweeting would not be influential because the voting was spread across several weeks.

In our previous article we acknowledged the influence of bulk tweeting. We said that campaign tweets would have influenced public opinion to some degree, but we anticipated it to a much lower extent.

So there are lessons to be learned from this for any future analysis.

Success stories

So far we’ve talked mainly about when we were wrong and why. But what about those times when the Big Data and Smart Analytics Lab got it right?

The Lab correctly predicted no less than 48 out of 50 US State elections held at the same time as the 2016 presidential election, which we also correctly called.

We called the Coalition’s win in the 2016 Australian federal election. And our method gave a clear indication that “Brexit” would prevail over “Bremain”, contrary to the polling before Britain’s referendum on European Union membership.

In all of these cases, we were sampling the social media sentiment leading up to a specific election day when all would be decided. The election result is a snapshot of how the voters feel on that day.

With the same-sex marriage survey, the voting was spread across several weeks, making it difficult to know what proportion of the vote took place on a particular day or even week.

Even with this uncertainty, it was possible to make reasonably accurate predictions provided that the underlying assumptions are correct, such as all tweets being equally influential.

Twitter isn’t the only source

With 328 million active users worldwide, and many more inactive users who nonetheless read the tweets of others, Twitter is an excellent source of information on people’s views and intentions.

But it is good to have multiple sources of data when doing big data analytics.

In diverse projects, ranging from tourist satisfaction to environmental changes, the Big Data and Smart Analytics Lab uses combinations of Twitter, Flickr, Instagram, public Facebook pages, and even the Chinese social media platform Weibo. It is all grist to the mill.

Facebook is by far the dominant social media channel in the world. Only public pages are accessed by our analytics. But with two billion users and growing, we still have plenty of data to work with.

Twitter has evolved into a more news- and opinion-oriented channel, with people sharing newsworthy items with like-minded others. Celebrities and politicians use it as a direct channel to their audience, bypassing the established media channels altogether.

Brevity of tweets was enforced by a 140-character limit until recently, when the length restriction was doubled to 280 characters. The extra characters make tweets an even richer source of information for data mining.

The power of social media

The fact remains that people say things on social media that they would not say out loud. Many trolls and hecklers in the online world turn out to be mild-mannered individuals in the real world. It can be surprising.

Read more: How the same-sex marriage survey got us talking about – and trusting – data

Whose opinion is more interesting to the analyst? Is it the social persona who has responsibilities to the community and is generally polite? Or is it the private persona who only vents their true feelings to their closest confidants and on social media.

Both are interesting, but arguably it is the latter whose opinion determine the outcome of social issues.

The lesson to be learned from our error with same-sex marriage survey is that every social media post counts. Social media is indeed a powerfully democratising force.

Authors: David Tuffley, Senior Lecturer in Applied Ethics and SocioTechnical Studies, School of ICT., Griffith University

Read more http://theconversation.com/we-learn-from-our-mistakes-how-to-make-better-predictions-from-tweets-87599

Writers Wanted

How To Find The Right Emergency Plumber Lismore


Delivery rider deaths highlight need to make streets safer for everyone


The Conversation


Prime Minister Interview with Ben Fordham, 2GB

BEN FORDHAM: Scott Morrison, good morning to you.    PRIME MINISTER: Good morning, Ben. How are you?    FORDHAM: Good. How many days have you got to go?   PRIME MINISTER: I've got another we...

Scott Morrison - avatar Scott Morrison

Prime Minister Interview with Kieran Gilbert, Sky News

KIERAN GILBERT: Kieran Gilbert here with you and the Prime Minister joins me. Prime Minister, thanks so much for your time.  PRIME MINISTER: G'day Kieran.  GILBERT: An assumption a vaccine is ...

Daily Bulletin - avatar Daily Bulletin

Did BLM Really Change the US Police Work?

The Black Lives Matter (BLM) movement has proven that the power of the state rests in the hands of the people it governs. Following the death of 46-year-old black American George Floyd in a case of ...

a Guest Writer - avatar a Guest Writer

Business News

Nisbets’ Collab with The Lobby is Showing the Sexy Side of Hospitality Supply

Hospitality supply services might not immediately make you think ‘sexy’. But when a barkeep in a moodily lit bar holds up the perfectly formed juniper gin balloon or catches the light in the edg...

The Atticism - avatar The Atticism

Buy Instagram Followers And Likes Now

Do you like to buy followers on Instagram? Just give a simple Google search on the internet, and there will be an abounding of seeking outcomes full of businesses offering such services. But, th...

News Co - avatar News Co

Cybersecurity data means nothing to business leaders without context

Top business leaders are starting to realise the widespread impact a cyberattack can have on a business. Unfortunately, according to a study by Forrester Consulting commissioned by Tenable, some...

Scott McKinnel, ANZ Country Manager, Tenable - avatar Scott McKinnel, ANZ Country Manager, Tenable

News Co Media Group

Content & Technology Connecting Global Audiences

More Information - Less Opinion