Could Authors Be Saying “Goodbye” to the Accept/Reject Decision?
Combining the speed and transparency of preprints with the expert scrutiny and evaluation of peer review
Complete the form below to unlock access to ALL audio articles.
Most scientists in the course of their careers have experienced crushing rejections of journal papers that may represent the culmination of hundreds or even thousands of hours of hard work, blood, sweat and tears, sometimes literally. Many also have painful tales to tell about reviewers who missed the point and rejected their work without taking the time to understand it, superficial or biased feedback or seemingly endless requests for more and more experiments when the budget has all been spent and the postdoc has left the country for pastures new.
The whole process of trying to publish scientific research can not only be slow, it can be demoralizing and put barriers in the way of further funding or job opportunities and can ultimately prevent the dissemination of valuable findings to the scientific community. While I think we all understand the vital role peer review plays in scientific publishing and ensuring the quality and validity of the research presented to readers, the traditional system is far from perfect.
But what if there was another way? One in which authors could retake more control over the process after they hit the “submit” button and readers could be more informed about how their peers evaluate the study? eLife, an independent nonprofit that publishes a journal covering all areas of the life sciences and medicine, has been shaking things up. At the start of 2023, they launched a groundbreaking new publishing model aiming to combine the speed and transparency of preprints with the expert scrutiny and evaluation of peer review.
We spoke to Dr. Damian Pattinson, executive director at eLife, about how the process works, what it means for submitting authors and readers and how it’s been received by the scientific community.
Karen Steward (KS): For those who may be unfamiliar with eLife’s new publishing and review process, can you tell us what’s changed and how the new system works?
Damian Pattinson (DP): In eLife’s new publishing model, preprints that are sent out for review go through a consultative peer-review process that results in a Reviewed Preprint and then a Version of Record (VOR). Many aspects of the review process remain the same, but there are some key differences. Firstly, it eliminates the accept/reject decision after peer review. Instead, all papers that are invited for review are published on the eLife website as a Reviewed Preprint, along with the public reviews and an eLife assessment of the work. Secondly, the authors control what happens next: they can choose to submit a revised preprint, or they can declare their Reviewed Preprint as the final VOR, which is sent to indexers such as PubMed and can be listed in funding, job applications and more.
KS: What motivated eLife to make these changes?
DP: The traditional peer-review and publishing process has a number of challenges that all fed into our decision to change things up at eLife. The process can be slow and frustrating for authors, with seemingly endless rounds of review, resubmission and rejection before an editor chooses to accept their paper. It’s also wasteful, as the rich evaluations provided as part of the review process are lost to readers when a paper gets rejected. On top of this, the reliance on journal titles as a measure of research quality means that where authors publish is often valued more highly than what they publish. We therefore chose to do something different at eLife by combining the immediacy and openness of preprints with the scrutiny of peer review by experts.
KS: How do you think these changes benefit both authors and readers? Can you see any downsides?
DP: Our new model provides a number of benefits for authors and readers. For authors, it allows them to communicate their findings immediately and widely and receive feedback quickly. It also provides them with certainty of outcome: once their paper has been sent for review, they are in control of the process and can decide what revisions to make and when to declare their paper as the final VOR. For readers, the process restores the value of peer review by giving context through public reviews, so they can see what the editors and reviewers think are the strengths or weaknesses of a piece of research. This in turn helps them to assess the work for themselves based on its own merits, rather than where it is published.
On the flip side of this final point, our model aims to shift the emphasis away from journal titles and impact factors as indicators of research quality and focus instead on public reviews and assessments. However, journal titles have long been used as a proxy for quality for assessment purposes and publishing in high-impact journals is a requirement for many. These flawed practices will continue to pose a challenge to the full adoption of our new model until we see more positive changes in how research and researchers are assessed.
KS: For papers that would previously have been rejected or designated as needing major revisions, can an author now effectively decline to make those changes/do further work and still have that version printed as the Version of Record? If so, do you think there’s a danger readers might skip over the review comments and miss shortcomings in studies that would previously not have made it to print?
DP: This is not something we have seen. As part of our process, there is greater author control, but the vast majority of authors go through a round of revisions (around 94% of VOR requests are after at least one round of revisions) and engage thoughtfully with the comments of the reviewers. If authors decide not to undertake certain revisions, they can explain why in the published author response. Also, there’s an incentive to revise as authors can receive an improved eLife assessment after substantive revisions.
KS: If some fundamental flaws are identified in a paper during peer review, how would this be handled? Would the paper be removed from your site altogether?
DP: To clarify, preprints that are submitted through eLife’s new model are not published on our website until they have gone through peer review, at which point they are published in eLife as a Reviewed Preprint.
If a paper is sent for review and the reviewers identify weaknesses, they will express their concerns in both the public reviews and eLife assessment. These will then be published together with the paper (Reviewed Preprint), highlighting the weaknesses to readers. It will then be up to the authors as to whether or not to revise their work.
In rare cases, if the findings of the preprint are found to be unreliable during the review process, the authors can withdraw their preprint in accordance with the rules of the server where their preprint was posted. In such cases, we will not proceed with publication of the Reviewed Preprint unless there are exceptional reasons to publish the public reviews.
If the findings are found to be unreliable after publication of the Reviewed Preprint (or VOR), the authors should contact the journal office to discuss a correction or retraction in the usual way.
KS: Have you found making reviewers comments publicly viewable to all has impacted the feedback reviewers provide and if so in what way has it changed?
DP: At this point we can only comment anecdotally, but we’re really pleased with the reviews we’re receiving, which are usually substantive and constructive. We have been publishing review materials since we launched in 2012, albeit in a different form, so reviewers are used to the expectations. If anything, publishing the reviews leads to higher levels of scrutiny and engagement and helps to discourage superficial comments.
KS: How has the new publishing and review process been received by the scientific community?
DP: We are pleased with the scientific community’s response to our new model so far. We have issued updates on the model’s progress at three months, six months and one year since launch in January 2023, with some recurring and positive themes around the community response. These include the fact that submissions have remained steady (and are in fact now growing); Reviewed Preprints allow authors to communicate their findings and the related reviews more quickly; and authors and editors have reported largely favorable experiences with the model, with some concerns about quality and suggestions regarding the process being voiced. Also, when we first launched the model, we saw many conversations among researchers on social media saying that they would be watching its progress closely and these conversations are still ongoing. Again, we are pleased with this response and the interest in how the model has been working and what we have learned, and continue to learn, as we continue working on it behind the scenes.
KS: Do you have plans for further changes to the system in the future?
DP: We continue to develop the new model based on feedback and experience. We will soon be making changes to the display of articles and listings so that readers can more easily identify the terms our editors have used to describe the significance and strength of evidence. This will allow readers to, for example, more easily identify papers our editors have deemed to be “landmark” discoveries. Our model presents an exciting opportunity to improve the current research communication system in so many ways. We’ll therefore keep using what we learn to enhance the sharing, discovery and use of new research for the benefit of our authors and readers, as well as other organizations that wish to explore this new approach for themselves.
Dr. Damian Pattinson was speaking to Dr. Karen Steward, Senior Scientific Specialist for Technology Networks.
About the interviewee
Dr. Damian Pattinson is the executive director at eLife, an independent nonprofit committed to improving the way research is reviewed and communicated. Damian started his publishing career at the British Medical Journal (BMJ), where he worked as an editor on BMJ Clinical Evidence and Best Practice. He joined PLOS ONE as executive editor and oversaw the dramatic expansion of the journal to become the largest scientific journal in the world. He then moved to Research Square as vice president of publishing innovation, where he launched the Research Square preprint platform, before joining eLife in 2020. He holds a PhD in neuroscience from University College London.
