Making peer reviews more open

Chris Rowan at Highly Allochthonous discusses the peer review system, and gives some suggestions as to how  it can be improved:

Personally, I don’t really mind anonymous reviews if we can break the correlation between anonymity and hatchet-job reviews which are not motivated purely by scientific concerns. And I think the only way to do this is to open up the process somewhat. In my ideal system, papers should only be rejected outright if the reviewers identify methodological or analytical errors with the data. If a reviewer has serious problems with the interpretation, their comments should be published together with the paper, alongside any rebuttal or response from the authors.

While papers get rejected by anonymous reviewers because of their hatchet-job reviews, sometimes they also get accepted for the same reason; in fact, I have seen rejection reports, and the appearance of papers in the same journal, at the same time frame, with the same “ostensible” flaws for which the paper has been rejected. Thus, making the referee reports of all papers available in a blog so that comments can be made on the acceptance/rejection by anybody who knows enough on the subject and cares enough to make a comment is the most ideal scenario I can think of–in some sense, PLOS one review system is pretty close to my ideal peer review mechanism.

Of course, as Rowan concludes in his post, it is the classic problem of who is going to bell the cat?

Of course, this is all very interesting, but we can’t ignore the elephant in the room: unless the big boys like Elsevier adopt such systems, or we all put our money where our mouths are and opt for the more innovative journals, nothing much is going to change.

Take a look!


One Response to “Making peer reviews more open”

  1. futureofscipub Says:

    i think the PLoS ONE system is only a small step in the right direction.

    what we need is a general open post-publication peer review system with
    – any scientist allowed to review any paper
    – public availability of all reviews along with each published paper
    – reviews as open letters to the community
    – signed and unsigned reviews
    – digital author authentication
    – numerical judgments on multiple scales allowing for automatic paper selection by means of arbitrary ranking functions

    i’m exploring these ideas here:

    it would be great to hear your thoughts.

    –nikolaus kriegeskorte

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: