Daily Bulletin

News

  • Written by Alan Finkel, Australia’s Chief Scientist, Office of the Chief Scientist
'There is a problem': Australia's top scientist Alan Finkel pushes to eradicate bad scienceAustralia's Chief Scientist Alan FinkelAAP

In the main, Australia produces high-quality research that is rigorous and reproducible, and makes a significant contribution towards scientific progress. But that doesn’t mean we can’t do it better.

In the case of the research sector here and abroad, we need to acknowledge that as good as the...

In the main, Australia produces high-quality research that is rigorous and reproducible, and makes a significant contribution towards scientific progress. But that doesn’t mean we can’t do it better.

In the case of the research sector here and abroad, we need to acknowledge that as good as the research system is, there is a problem.

There are a significant number of papers that are of poor quality, and should never have made it through to publication. In considering why this might be the case, I have found myself reflecting on the role of incentives in the research system.

Because incentives matter, as we have seen through the findings of the Royal Commission into the banking sector led by Kenneth Hayne.

The commission shone a light on how the sector incentivises its employees. And there are some incentives in the research community that, in my view, need to be looked at.

We may be inadvertently encouraging poor behaviour. And to ensure research remains high-quality and trustworthy, we need to get the incentives right.

Lessons from the banking Royal Commission

'There is a problem': Australia's top scientist Alan Finkel pushes to eradicate bad science Parallels can be drawn between inappropriate incentives in both the banking and research sectors, Dr Finkel says. AAP

The commission showed that over the past decade or two, the banking sector moved from salary-based to bonus-based remuneration. But those bonuses have been mapping to the wrong values: to sales and profit instead of compliance with the law and net benefit to customers.

Similarly in the research sector, we can’t ignore that there are many incentives pushing some to cut corners and lower their standards.

The competition for funding is fierce and is increasing every day. The temptation to judge a researcher’s performance through simple metrics, such as the number of published research papers, is strong. These metrics are incentives that drive behaviour - not all of it good.

Read more: Finkel: students, focus on your discipline then you’ll see your options expand

We all know of instances of poor research practice. Selective publication of results to support a hypothesis. HARKing: hypothesising after results are known. Manipulating data and research methods to achieve statistical significance.

If we can focus on improving the quality of research in general, we can achieve broad and long-lasting benefits. And I think the best way to do this is to look at the incentives.

Quality should trump quantity

Publication is a principal criterion for career advancement in the research sector. And I don’t want to change that. However, the institutionalisation of performance metrics has created incentives for researchers to publish as many papers as possible.

There shouldn’t be an incentive for a researcher to salami-slice their results into three or four separate publications, rather than one meaningful publication. If the purpose of publication is to share your results in a way that can be built on by other researchers, this kind of practice completely defeats that purpose.

'There is a problem': Australia's top scientist Alan Finkel pushes to eradicate bad science Perverse incentives can encourage the manipulation of research results, Dr Finkel says. AAP

One model that places the focus on quality over quantity is the “Rule of Five”. With this rule, a researcher’s performance for grant funding or promotion is judged on their best five publications over a five year period, accompanied by a description of its impact and the researcher’s individual contribution.

The exact number of publications or years isn’t important, as long as it is less than ten.

Of course, there are disciplinary differences that may need to be taken into account. But what matters is the emphasis on the significance of the research.

Researchers must undergo integrity training

Unlike other professions, there are no national competencies and no national recognition of education and training in research integrity. While many institutions in Australia do provide training programs for their PhD students, these programs vary in quality, content and reach.

And, to the best of my knowledge, no Australian institutions have a training requirement for their existing research workforce.

Read more: The science is clear: we have to start creating our low-carbon future today

I strongly believe the overall quality of research in Australia would be strengthened by research integrity training for all researchers.

Training puts a spotlight on expectations for the whole community and encourages consistent behaviour. It also removes that old chestnut of plausible deniability: “Honest, officer, I didn’t know it was wrong!”

The training must be accredited, and must be high quality. It should not be a “tick the box” exercise. And if we circle back to incentives, the best way to encourage researchers to undertake the training is to make proof of training a requirement for obtaining a grant.

To those naysayers who say it will never happen, let me tell you that it already has. The Irish Health Research Board has recently implemented such a scheme.

'There is a problem': Australia's top scientist Alan Finkel pushes to eradicate bad science Finkel: research integrity training should be mandatory for all researchers, and tied to grant funding. AAP

‘Predatory, evil’ scholarly journals

Finally, I am concerned that the incentives in the research system are not just driving bad behaviour for researchers, but are also creating a market for criminals to enter scholarly publishing.

What is motivating the crooks is the pay-per-page system that has come with the introduction of open access publishing.

Now, open access publishing has many benefits and I support the move towards it. But I remain concerned that it has opened the door for predatory, evil, crooked journals.

It is just too easy to set up a journal and a website with a highfalutin title, and appropriate the biographies of leading researchers for the editorial board – without their knowledge or permission. Before you know it, huge numbers of papers are being published without any rigour.

And there are researchers who are knowingly paying to publish in journals that have no peer review, even though they claim to. Journals that have no ethics. Not even an editorial team to consider the submitted paper.

These researchers might just be naïve, but we have to acknowledge that the current incentives reward this behaviour.

While this is not a major problem in Australia, emerging research nations are really struggling with this.

'There is a problem': Australia's top scientist Alan Finkel pushes to eradicate bad science Finkel: a new publishing standard should be introduced to weed out unscrupulous research journals. AAP

In my conversations with senior research leaders around the world, they are looking for ways to improve performance metrics in a way that does not drive their researchers to these journals.

I propose a rigorous quality assurance system, designed to inform stakeholders that a particular journal’s processes for assessing a paper meets agreed publishing standards. I like to call it Publication Process Quality Assurance, or PPQA.

Compliance with PPQA would indicate to researchers, research institutions, libraries and granting agencies that the journal follows internationally accepted guidelines for the publication process.

Read more: Science isn't broken, but we can do better: here's how

Granting agencies are best placed to provide the incentive for researchers to only publish in PPQA-compliant journals by enforcing it through their grant application process.

Follow the money

You might have picked up by now a common thread; that in each of my three recommendations, I am looking to take the responsibility back to the granting agencies. It’s a concept referred to by others as “follow the money”.

If the granting agencies put in place these measures, they will ripple through into the research institutions and mitigate the ongoing risks of poor quality research.

It will change the culture and ensure that last century’s academic rigour continues for the 21st century research workforce and beyond.

This is an edited version of a speech Dr Finkel will deliver in Melbourne on September 12 at the Peter Doherty Institute for Infection and Immunity, to mark its fifth anniversary.

Authors: Alan Finkel, Australia’s Chief Scientist, Office of the Chief Scientist

Read more http://theconversation.com/there-is-a-problem-australias-top-scientist-alan-finkel-pushes-to-eradicate-bad-science-123374

The Hannah Clarke inquest reveals, yet again, significant system failures. Here's what's urgently needed for women's safety

arrow_forward

So this is how it feels when the robots come for your job: what GitHub's Copilot 'AI assistant' means for coders

arrow_forward

The Conversation

Business News

How to Provide the Best Service Your Industrial Business Can

In the current business world, you may very well find that the difference between your business and that of your closest competitor is that of customer service. This is good as it is an area where...

NewsServices.com - avatar NewsServices.com

WhiteBIT continues its transcontinental expansion and enters the Australian market

Europe's largest crypto-exchange WhiteBIT continues to grow its business globally. The company has now joined the massive adoption of blockchain technology in Australia. By providing a secure and ea...

Daily Bulletin - avatar Daily Bulletin

How Business Permit Processing Works in the Philippines

Are you an aspiring new business owner? A business permit is a must for any establishment here in the Philippines. Without it, you have no legal right to run one. But do you know how the permit pr...

Angelo Castelda - avatar Angelo Castelda

WebBusters - Break into local search

WebBusters.com.au