Open peer review

Open peer review describes a scholarly/scientific literature concept and process, central to which is the various transparency and disclosure of the identities of those reviewing scientific publications. The concept thus represents a departure from, and an alternative to, the incumbent anonymous peer review process, in which non-disclosure of these identities toward the public – and toward the authors of the work under review – is default practice. The open peer review concept appears to constitute a response to modern criticisms of the incumbent system and therefore its emergence may be partially attributed to these phenomena.

Rationale

The traditional anonymous peer review has been criticized for its lack of accountability, the possibility of abuse by reviewers or by those who manage the peer review process (that is, journal editors),^[1] its possible bias, and its inconsistency,^[2] alongside other flaws.^[3][4] Both processes are intended to subject scholarly publications to the scrutiny of others who are experts in the same field.

The evidence of the effect of open peer review upon the quality of reviews, the tone and the time spent on reviewing is mixed, although it does seem that under open peer review, more of those who are invited to review decline to do so.^[5][6]

History

A number of reputable medical publishers have trialed the Open Peer Review concept. The first open peer review trial was conducted by The Medical Journal of Australia (MJA) in cooperation with the University of Sydney Library, from March 1996 to June 1997. In that study 56 research articles accepted for publication in the MJA were published online together with the peer reviewers' comments; readers could email their comments and the authors could amend their articles further before print publication of the article.^[7] The investigators concluded that the process had modest benefits for authors, editors and readers.

Early era: 1996–2000

In 1996, the Journal of Interactive Media in Education^[8] launched using open peer review.^[9] Reviewers' names are made public and they are therefore accountable for their review, but they also have their contribution acknowledged. Authors have the right of reply, and other researchers have the chance to comment prior to publication. As of February 2013, the "Journal of Interactive Media in Education" no longer uses open peer review.^[10]

In 1997, the Electronic Transactions on Artificial Intelligence, , was launched as an open access journal by the European Coordinating Committee for Artificial Intelligence. This journal used a two-stage review process. In the first stage, papers that passed a quick screen by the editors were immediately published on the Transaction's discussion website for the purpose of on-line public discussion during a period of at least three months, where the contributors' names were made public except in exceptional cases. At the end of the discussion period, the authors were invited to submit a revised version of the article, and anonymous referees decided whether the revised manuscript would be accepted to the journal or not, but without any option for the referees to propose further changes. The last issue of this journal appeared in 2001.

In 1999, the open access journal Journal of Medical Internet Research^[11] was launched, which from its inception decided to publish the names of the reviewers at the bottom of each published article. Also in 1999, the British Medical Journal^[12] moved to an open peer review system, revealing reviewers' identities to the authors (but not the readers),^[13] and in 2000, the medical journals in the open access BMC series^[14] published by BioMed Central, launched using open peer review. As with the BMJ, the reviewers' names are included on the peer review reports. In addition, if the article is published the reports are made available online as part of the 'pre-publication history'.

Several of the other journals published by the BMJ Group allow optional open peer review, ^[15] as do PLoS Medicine, published by the Public Library of Science.^[16][17] The BMJ's Rapid Responses^[18] allow ongoing debate and criticism following publication.^[19]

Recent era: 2001–present

Atmospheric Chemistry and Physics, an open access journal launched in 2001 by the European Geosciences Union, has a two-stage publication process. In the first stage, papers that pass a quick screen by the editors are immediately published on the Atmospheric Chemistry and Physics Discussions website. They are then subject to interactive public discussion alongside formal peer review. Referees' comments (either anonymous or attributed), additional short comments by other members of the scientific community (which must be attributed) and the authors' replies are also published in ACPD. In the second stage, the peer-review process is completed and, if the article is formally accepted by the editors, the final revised papers are published in ACP. The success of this approach is shown by the ranking by Thomson Reuters of ACP as the top journal in the field of Meteorology & Atmospheric Sciences^[20]

In June 2006, Nature launched an experiment in parallel open peer review – some articles that had been submitted to the regular anonymous process were also available online for open, identified public comment.^[21] The results were less than encouraging – only 5% of authors agreed to participate in the experiment, and only 54% of those articles received comments.^[22][23] The editors have suggested that researchers may have been too busy to take part and were reluctant to make their names public. The knowledge that articles were simultaneously being subjected to anonymous peer review may also have affected the uptake.

In 2006, a group of UK academics launched the online journal Philica, which tries to redress many of the problems of traditional peer review. Unlike in a normal journal, all articles submitted to Philica are published immediately and the review process takes place afterwards. Reviews are still anonymous, but instead of reviewers being chosen by an editor, any researcher who wishes to review an article can do so. Reviews are displayed at the end of each article, and so are used to give the reader criticism or guidance about the work, rather than to decide whether it is published or not. This means that reviewers cannot suppress ideas if they disagree with them. Readers use reviews to guide what they read, and particularly popular or unpopular work is easy to identify.

Another approach that is similar in spirit to Philica is that of a dynamical peer review site, Naboj.^[24] Unlike Philica, Naboj is not a full-fledged online journal, but rather it provides an opportunity for users to write peer reviews of preprints at ArXiv. The review system is modeled on Amazon and users have an opportunity to evaluate the reviews as well as the articles. That way, with a sufficient number of users and reviewers, there should be a convergence towards a higher quality review process.

In February 2006, the journal Biology Direct was launched by BioMed Central, providing another alternative to the traditional model of peer review. If authors can find three members of the Editorial Board who will each return a report or will themselves solicit an external review, then the article will be published. As with Philica, reviewers cannot suppress publication, but in contrast to Philica, no reviews are anonymous and no article is published without being reviewed. Authors have the opportunity to withdraw their article, to revise it in response to the reviews, or to publish it without revision. If the authors proceed with publication of their article despite critical comments, readers can clearly see any negative comments along with the names of the reviewers.^[25]

An extension of peer review beyond the date of publication is Open Peer Commentary, whereby expert commentaries are solicited on published articles, and the authors are encouraged to respond. In the summer of 2009, Kathleen Fitzpatrick explored open peer review and commentary in her book, Planned Obsolescence, which was published by MediaCommons using "Commentpress", a Wordpress plugin that enables readers to comment on and annotate book-length texts.

Another form of "open peer review" is community-based pre-publication peer-review, where the review process is open for everybody to join.

See also

• Open research

References