Open peer review is the practice of making peer review reports, signed or unsigned by the reviewer, publicly available online. The practice is still relatively new but becoming increasingly common in some fields. Open peer reviews can accompany a manuscript in several stages, including a preprint (Review Commons, PREreview, Peer Community In), a submitted version (the STI 2023 conference submission system), or the final published “version of record” (the journals Quantitative Science Studies or Journal of Open Educational Resources in Higher Education, among others).

I did not become aware of open peer review as a concept until about two years ago, but in that short time, I have found it has quickly become part of my standard process when reading literature in my field of bibliometrics.

This short post will outline the benefits I have found in regularly reading open peer review reports. It is my hope that the practice will continue to become more widespread, both in terms of reviewers posting their reports, but also readers making use of this new source of information in the publications they choose to read.

Concerns

Concerns do exist about publicly posting review reports. Early career researchers may not submit fully candid thoughts if their names will be attached. Even when reviews are unsigned, disagreeing with an established researcher in the field may invite unwanted attention if the posted comments can be used to identify the reviewer. Open peer reviews may be seen as airing internal or private discussions, and authors may not want criticisms of their work to be so widely seen.

Many reviews of published articles are accessed through Publons, which was bought by Clarivate in 2017 and folded into Web of Science. It is not clear to me who can access these reports and who cannot. We subscribe to Web of Science here at Iowa State University so I am able to see such reports (though after following several confusing redirects). If non-subscribing institutions or unauthenticated researchers are prevented from seeing reports on Publons, it defeats the entire purpose of posting the reviews.

Additionally, I (and others) have had the unfortunate experience of journals not allowing reviewers to publicly post their reviews. This policy, in my opinion, is simply wrong. As a reviewer, I have not signed anything that transfers the copyright to the results of my volunteer labor. The review is not the journal’s to control. I continue to maintain the right to my review, and I am free to post it anywhere I like. I have declined a review opportunity because of this and will continue to do so in the future, only reviewing for journals that allow me to openly post my peer review.

Benefits

There are several benefits to posting and reading open peer reviews, applicable to various interest groups. First, simply by making review reports more visible, we demonstrate that this traditionally under-valued practice is indeed valuable and should be taken into account as a time-consuming job duty. Open reports are typically assigned a DOI. I have completed reviews that I’m very proud of, and I include identifiers in my CV and yearly evaluation.

As a peer reviewer

Put simply, reading other peer reviews makes my reviews better. It is a common complaint that researchers are not given formal training in how to conduct or write a peer review. Even though toolkits do exist that attempt to formalize the review process, it is my perception that most are expected to just figure it out.

From reading review reports, I can learn how others structure their reviews. I learn by seeing patterns from others in my field, and I have adopted templates, examples, and tactful turns of phrase. For example, my reviews now typically start with a general summary of the paper’s main idea, restated in my own words. I then give a bottom-line conclusion and transition to my comments by saying, “however, I did have some questions.”

I start with major concerns, then address more minor suggestions. I reference line numbers in the manuscript when possible, and try to balance things I merely suggest vs. things I deem more critical.

As an author

Most helpful for me, reading reviews is proof that others get corrective feedback too! So much of writing is thinking that everyone knows more than me (this still may be true). When I only ever see the final, polished version, I start to forget that everyone produces a first draft that needs improvement. Normalizing revisions helps me with my own writing.

When I receive comments back, I like knowing who has reviewed my paper. In some cases, I have sent the reviewer a short note of thanks for their comments. Opinions and comfort levels vary, but I do generally sign my name to the reviews I publicly submit.

It is also instructive to see other authors’ responses. How do they deal with reviewer comments? Am I expected to address all criticisms and make every suggested change, or can I disagree with reviewers? If so, how can I do that gracefully?

As a contributor

As a prospective author to a journal, I get to see what a handling editor says, does, and recommends. What are the stages or degrees of decision? Major revision, minor revision, reject, accept? Note: this applies only to submitted and accepted articles, as preprints will not include editor comments.

As a reader

Finally, as a reader, I enjoy seeing the background and development of the paper. Did reviews from the first draft change the paper drastically? Was it basically fully formed and comments very minor? What outstanding questions still remain? How did reviewers handle a controversial topic, and what did the author address (or not)?

Conclusion

While I have outlined some of the reasons I personally find open peer review reports valuable, there are several more benefits outlined in ASAPBio’s Publish Your Reviews initiative. I am joining ASAPBio’s Crowd Preprint Review project which begins later this summer, and I look forward to having more chances to practice submitting open comments.

I plan to continue writing and posting my reviews openly, and seeking out and reading those that are available. I think you could benefit from the practice, too.

Disclaimer: I serve as peer reviewer for the two journals mentioned in paragraph one (QSS and JOERHE) and contributed a preprint submission to the STI 2023 conference.