Tyler Cowen at Marginal Revolution:
I don’t envision the free access system as the status quo but free. Papers would be ranked directly in terms of status and popularity rather than ranked through the journals they are published in. Ultimately there wouldn’t be journals and this would make a big difference as journals are the current carrier of selective incentives and status rewards. It would be easy to refuse to referee, since you wouldn’t fear being shut out of publication of that journal; I suspect refereeing might die. And if status were attached to the individual paper rather than the journal, who would bother to become an editor? It would be a very different world and in some ways more like (academic) blogging than its proponents may wish to think.
In other words, the partial monopolization of for-fee journals makes it possible to produce status returns to motivate both editors and referees. Returning to the free setting, refereeing will survive insofar as writing detailed referee comments on other people’s work helps with your own research; it is interesting to ponder in which fields this might hold.
The interesting bit for me here is Tyler’s suggestion about the implicit incentives for reviewing; that people referee papers for fear of not being able to get published in the journal in question. My personal take on it (as is the take of a number of other people, if this discussion is anything to go by), is a little different. I review not so much because I feel that if I don’t review a paper for journal x that the editors of that journal will look unkindly on me in future, but because of a broad sense that I send papers out that others ought to review, and hence there’s a diffuse obligation on me to review other people’s papers in turn. In other words, I think that the motivating factor is general reciprocity rather than specific reciprocity. Not only that: when I have been on search committees where we are considering people who have been in the field for a few years, I usually check their resumes to see whether they have been reviewers for a few journals. This isn’t so much to figure out what the editors think of them (very often, editors are happy with whoever they can get as a reviewer), as because it seems to me to be the best publicly available proxy for whether the candidate is the kind of person who is likely to take on their share of the unofficial responsibilities that any school or department has.
This isn’t to say that Tyler may not be right when he suggests that an open publication world might not support the kinds of detailed and thoughtful review that we hope for, and sometimes get, in the current system. But I suspect (perhaps wrongly) that the mechanism that would undermine reviewing would primarily be a sociological one rather than an economic one. That is, it would have more to do with the disappearance of the social role of reviewer, and the set of perceived general responsibilities that go with it, than with the opportunities for specific quid-for-quo interactions between reviewer and editor that the current review system lends it to.
Trevino suggests, correctly I think, that journals serve an important function independent from housing articles. Journals signal which papers are worth reading. Sure, you could imagine that other signals might operate in a free access system, but they would likely be tethered with additional sorting problems: information cascades, author prestige would matter more, etc. How would a new scholar, especially one who is at a lower-ranked school, get discovered in such a system? It seems like the costs for searching for high quality papers and for breaking into the system as a new scholar would be much higher in a free access system.
Trevino also addresses the issue of reviewer incentives. She notes that one of the main problems confronting any journal editor is finding good reviewers. A surprising number of scholars simply refuse to review. Like most of us, she thinks reviewing is a professional responsibility. Ethically, you should feel compelled to review papers at journals that you read and to which you submit papers. Unfortunately, a number of people refuse to review (keep in mind that she’s talking about seeking reviewers for AMR, one of the top journals in organizational scholarship; the headaches that editors at lower-ranked journals have are probably much greater).
She notes that there are many good reasons for choosing not to review a paper. You may not read the journal or ever seek to publish in the journal, you may be too closely associated with one of the authors, or the paper may lie outside your area of expertise. These are all good reasons for not reviewing. But some scholars simply don’t review, period. The evidence, then, suggests that, as Tyler thinks, there is at least a minimal reviewer incentive problem. Her solution?
As ethics ombudsperson for the Academy (another Academy role I play), it occurred to me that perhaps we should consider adding a statement of this responsibility to our ethics code and enforce it. It’s fine for someone to “opt out” of reviewing. But, if so, shouldn’t that individual “opt out” of the submission process too? The Academy journals now have a very efficient web-based tracking system, so these individuals are easily identifiable. What do you think? Should we consider enforcing the quid pro quo expectation in some way?
If scholars are going to treat the journal in an instrumental way, refusing to review because they don’t want to bear any of the collective costs of producing a quality journal, shouldn’t the journal have the right to refuse their article submissions as well? Trevino’s suggestion is an interesting one. Notably, she’s taking Tyler’s logic to the extreme. Normative constraints might not be enough; professional responsibilities/ethics may have to be actively enforced.
Open Access Open Review journals where there is an option for scholars to review papers anonymously, pseudonymously or under their own names is the best option, since, the reviewer, if he/she so wishes, can get credit for his/her contributions in such a model, while, those who are afraid of a backlash for the kind of opinions they give in their review can still review without having to give away their identities.
Tags: Peer review
March 14, 2008 at 8:51 am |
I have seen another type of problems even in the traditional reviewing process in mathematics. Papers have been getting increasingly long and difficult depending on previous long and difficult papers. Very few are prepared to spend so much time on difficult and sometimes incomplete papers (there are exceptions like Poincare conjecture series). One such example is the papers on Tarski problems in group theory. Even though the papers are now published and preliminary results interesting, I do not know anybody who can confirm the final results. It is not clear whether one can use the intermediate results with any certainity. I myself have colloborated on a 170 page paper which we published after keeping it as a preprint for about 3 years. And then wrote a correction of 20 pages and we use all these in the next papers. This is not an unusual situation in mathematics but it leaves considerable parts of the literature in some uncertainity.
March 14, 2008 at 12:36 pm |
Dear Swarup,
It indeed is a very serious problem when huge sections of literature are not understandable for most of the people in the field; it becomes worse when more and more of material is built on such foundations which most of the people do not understand.
I think, conferences, workshops, and tutorials are the ideal places to disseminate such information — not to mention textbooks — if somebody can go through all the literature and bring some clarity and organisation to it — like, for example, Knuth has been doing successfully through his Art of Computer Programming books, it would be of immense value to the field.
If people get recognised for their (largely) pedagogical contributions (through conferences, workshops, tutorials, and text-books), more and more people might be willing to do it — which I think would be hugely beneficial to the community.
Consider Halmos’ Finite dimensional vector spaces book for example–though almost all of it based on the lectures of von Neumann, all of us are better off today for the efforts that Halmos put in, isn’t it? I understand S Chandrashekhar also did similar stuff to several areas of applied mathematics and physics. So, I think we need more people who will take Chandrashekhar, Halmos, and Knuth as their role models (and, preferably, publish their books online, for free).