Training for peer reviewing

Virginia Walbot comments on how grad students and post-docs should be trained for peer reviewing in a piece in Journal of Biology — Are we training pit bulls to review our manuscripts?:

Good early training of graduate students and postdocs is needed to prevent them turning into future generations of manuscript-savaging reviewers. How can we intercalate typical papers into our training?

There is a good discussion on one of the possible reasons why students become pit bull reviewers (and, of course, followed by some suggestions as to how a better training can be imparted):

The majority of our collective publications, and hence scientific progress, comes from incremental insights in which the context is provided by the ongoing struggle to resolve a number of outstanding questions in a field. A series of papers, often from different labs over a span of several years, will add up to the solution to one or several questions. Each publication was timely when published, but may be wrong in some of the details of interpretation – the focus in the discussion may have dealt primarily with the most popular model, missing the chance to ‘redesign’ that model to better fit all of the data. None of these papers is a complete answer: the new insights will eventually be summarized in a short review article weaving the incremental threads of data into one story that becomes the new paradigm, at least for a while.

Taking a phrase from the current US political scene, these experimentally solid papers are “timely, targeted, and temporary”. That is, they address unanswered issues that are on the minds of those in the field, they target specific issues amenable to experimental or theoretical resolution, and in some ways their impact is temporary, because subsequent papers using the emerging insights and new methodologies will supersede these solid papers. Yet these solid papers are the foundation for progress most of the time.

Students are trained to be pit bulls in finding even the tiniest faults in great papers. Nearly all the truly remarkable papers we teach contain a few ‘typographical’ errors such as reference to the incorrect panel of a figure or a small mistake in a large table or the wrong initials for an author in the reference list. These errors do not detract from the impact of the work, but instruct students to be vigilant in that even the deservedly famous can make mistakes. This insight may even inspire some students to use spell-checker and other automated tools to eliminate such errors. Similarly, the papers with fatal flaws, particularly those in which a critical control is simply missing, are highly instructive. These papers highlight the dangerous ‘snow globe world’ of belief in a particular theory – a world circumscribed to consider only those things within view – and even then only when obscured by snow. It’s instructive to point out that the meaning of ‘belief’ is to accept as true in the absence of facts. The papers with fatal flaws help students appreciate that maintaining skepticism about current interpretations is essential for progress.

How then can we teach students to appreciate the bulk of our own contributions to the literature? Great manuscripts with minute flaws and bad papers with fatal flaws will represent a tiny minority of the manuscripts that our fledgling reviewer will actually encounter. The majority of manuscripts will be sound in conception and fair in data presentation, and contain some new information. How do we teach judgment of where in the pantheon of journal quality a particular study belongs? How do we teach what constitutes a timely ‘publishable unit’ – not complete proof of a major concept but a defined step in that direction?

A very interesting article; take a look!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: