Daily Bulletin

The Conversation

  • Written by Dr Jessica Borger, Postdoctoral Research Fellow in Immunology, Monash University
Peer review has some problems – but the science community is working on it

Peer review is the central foundation of science. It’s a process where scientific results are vetted by academic peers, with publication in a reputable journal qualifying the merits of the work and informing readers of the latest scientific discoveries.

But peer review sometimes gets a bad rap – criticised for a purported lack of transparency, low accountability and even poor scientific rigour.

There’s now considerable movement towards tweaking or even remodelling the peer review system. Key areas of focus include making journal editors more directive in the process, rewarding reviewers, and improving accountability of editors, reviewers and authors.

Read more: Science isn't broken, but we can do better: here's how

Peer review relies on volunteers

The peers in the peer review system are volunteer academics with expertise relevant to the paper being considered. But it’s hard to find suitable volunteers.

Reviewing is more complex and onerous than just rejecting or accepting a manuscript. More often than not, a reviewer suggests additional experiments that authors have overlooked, or challenges the interpretation of some of the data. This initiates a dialogue between author and reviewer aimed at improving the integrity and scientific merit of the paper.

It takes time – at least seven to eight hours per paper done properly, with no remuneration or recognition for the reviewer and hence rarely regarded as a priority in a busy academic schedule. As a result, scientific rigour can be lost when reviews become fast-tracked.

Read more: Not just available, but also useful: we must keep pushing to improve open access to research

On the other extreme, sometimes novice reviewers (perhaps trying to impress the editor) can turn small discrepancies into significant flaws. This presents a breakdown in the fairness of the review process.

Overall, these issues create a limited number of peer reviewers in practise, an outcome that can lead to cronyism.

Delving further into this remaining inadequate pool of reviewers, a significant gender gap is also apparent. Nature reports that less than 20% of its reviewers in 2017 were women.

More accountability from editors

We should demand more of our journal editors.

Editors can become more proactive by rejecting articles that are not at publication standard upon submission, rather than placing the arduous task on a reviewer to be both scientist and copy checker.

To retain and train novice reviewers, clearer evaluation criteria from editors would vastly improve the reliability and quality of submitted papers.

Editors could also engage better in a dynamic dialogue between author and reviewer – digital communication technologies enable real-time global discussions to facilitate streamlined review processes for all involved.

Read more: Bored reading science? Let's change how scientists write

Recognition of reviewers

Traditionally, editors are held up to be a revered part of the peer review process, and reviewers are simply not acknowledged for their contributions. But this is changing.

To promote increased transparency, greater accountability and fairness, open peer review processes list reviewers and editors in addition to authors in each publication.

This is happening now in newly established online journals such as eLife. Independent platform Publons rewards reviewers by listing all peer reviewing and editorial activity to provide evidence of a reviewer’s expert contributions in their field. Publons also runs a reviewer awards program.

Read more: Demasi cleared, but images in science continue to attract intense scrutiny

Similarly, Elsevier has started a “reviewer recognition programme”, extending various rewards and publishing a yearly list acknowledging the contributions of all the reviewers.

This process has been met with criticism by some who insist anonymity guarantees unbiased opinions.

Post- vs pre-publication peer review

It’s now becoming clear that scientific dialogue does not need to stop at the endpoint of publication, and that not all problems within a manuscript may be identified at the time of peer review.

Post-publication peer review in its most validated form, involves a journal such as Frontiers asking academics to perform a published interactive dialogue with authors during the review process, giving a level of accountability and responsibility.

Other journals such as Faculty of 1000 Research, Copernicus and PLOS ONE publish papers with minimal evaluation. This shifts the focus towards post-publication peer review – authors, reviewers and readers critique and comment on the paper to judge its scientific merit in the public domain.

Alternative post-publication peer review platforms such as ScienceOpen invite all scientists registered with digital identifier ORCID to write a review or comment on DOI-linked papers. This facilitates engagement of a large cross-section of the scientific community for dynamic appraisal of a publication’s scientific merit.

Forums such as Pubpeer invite anonymous commentary from anyone in the scientific or general community. This occurs without moderation, openly facilitating the possibility of trolling and abusive behaviour at times culminating in legal action.

Time to try something new?

Peer review is not ready to be retired – but it is primed to change.

A recent trial by eLife intends to radically transform the roles of editor, reviewer and author. According to this model, if a senior editor deems a publication worthy of going to review, this paper immediately qualifies for publication.

Read more: Weakened code risks Australia’s reputation for research integrity

Once under review, an open dialogue between author and reviewers takes place. Upon receipt of reviewers’ recommendations, the authors can decide to continue experiments if advised, retract the paper or publish it. This leaves the author’s decision to the scrutiny of the general scientific community.

This innovation may greatly improve the transparency of open peer review, increase accountability on behalf of all participants and reduce burden on the peer review system. It addresses the three major strategies required for improvement of the peer review system. But is it a step too far, too soon? Time will tell.

The overall goal of debates around peer review and appearance of new publication platforms and approaches is to create a united front of authors, reviewers and editors to uphold scientific integrity.

This is vital not just within academic circles, but also to maintain the reputation of science in the broader community.

Authors: Dr Jessica Borger, Postdoctoral Research Fellow in Immunology, Monash University

Read more http://theconversation.com/peer-review-has-some-problems-but-the-science-community-is-working-on-it-99596

Writers Wanted

The ebb and flow of COVID-19 vaccine support: what social media tells us about Australians and the jab


Cyclone Seroja just demolished parts of WA – and our warming world will bring more of the same


New Zealand’s new housing policy is really just a new tax package — and it’s a shambles


The Conversation


Prime Minister Scott Morrison delivered Keynote Address at AFR Business Summit

Well, thank you all for the opportunity to come and be with you here today. Can I also acknowledge the Gadigal people, the Eora Nation, the elders past and present and future. Can I also acknowled...

Scott Morrison - avatar Scott Morrison

Morrison Government commits record $9B to social security safety net

The Morrison Government is enhancing our social security safety net by increasing support for unemployed Australians while strengthening their obligations to search for work.   From March the ...

Scott Morrison - avatar Scott Morrison

Ray Hadley's interview with Scott Morrison

RAY HADLEY: Prime Minister, good morning.    PRIME MINISTER: G’day Ray.   HADLEY: I was just referring to this story from the Courier Mail, which you’ve probably caught up with today about t...

Ray Hadley & Scott Morrison - avatar Ray Hadley & Scott Morrison

Business News

Step by Step Moving Guide for Moving an Office

Compromising office hours, carrying bulk equipment, moving sensitive files — office relocation is no cakewalk. There is so much to do when you want to relocate your office. Aside from the major tas...

News Co - avatar News Co

5 Effective Employee Retention Strategies You Can Apply to Your Organization

Your employees are the lifeblood of your organization. No matter how great you think your company is, without the right talents to keep your business up and running, all your efforts to rise above...

News Co Media - avatar News Co Media

5 Reasons Why Organizational Culture is Crucial to Your Business

You have probably heard a lot about organizational culture and companies finding ways to create a positive culture in the workplace, but not so much about the reasons why it even matters to busine...

News Co Media - avatar News Co Media