Nature of Peer Review Undergoing Review at Nature
The science cops are on trial. Peer review, the process that many are led to believe enforces objectivity and validity in scientific papers, has come under fire lately (see 03/17/2006, 02/05/2006, 01/31/2006, 01/09/2006). In response to scandals and criticisms that the peer review as practiced is no guarantee against fraud, Nature is stepping out into uncharted waters.
In an Editorial entitled “Peer review on trial,”1 Nature announced two initiatives that seem revolutionary if they become the new standard. One is to attach blogs to every item posted on News @ Nature, such as this one on the fish-o-pod, Tiktaalik. On a blog, readers can vociferate for or against a claim, rather than take the author’s word for it in a published paper that is assumed factual since it passed peer review.
Another change opens the door to the hitherto secretive process of reviewing papers. The trial run of the new process is accompanied by an online debate on peer review at Nature.
During the trial, which will last several months, Nature’s traditional approach to peer review will continue: typically, we send selected submissions to two or three experts whose identities are kept confidential. We believe that this approach works well. Meanwhile, over the next few weeks, the web debate will explore other approaches, as well as the potential for online techniques to unpack the various functions of conventional journals, the ethics of peer review, and more
Our online trial opens up a parallel track of peer review for submitted papers for authors willing to go down that route. The traditional process will still be applied to all submissions selected for peer review. But we will also offer to post the submitted manuscript onto an open website. Anyone can then respond to it by posting online comments, provided they are willing to sign them.
After a period of time, the website will close, and the author(s) will have the opportunity to modify the paper; Nature will also consider the comments posted before publishing it in the journal.
The editors realize this opens research up to critique by journalists and the public as well. Nature is calling this a trial, not an experiment; they only want explore, expand, and test these ideas to see if they take hold. But, they end, “our core goal remains as always: to bring our readers the most stimulating content that our editorial skills can deliver.”
1Editorial, “Peer review on trial,” Nature 441, 668 (8 June 2006) | doi:10.1038/441668a; Published online 7 June 2006.
Notice they said stimulating content, not reliable content. Science fiction is stimulating, too, and so is alcohol.
It’s about time peer review got some review itself, and lost some of its mystique. It’s a relatively recent tradition that many great scientific discoveries got by without just fine (e.g., Newton’s Principia). While scrutiny by others is wise for any claim (even in theology, the Apostle Paul said that “the spirits of the prophets are subject to the prophets”), peer review as practiced in science has created new problems while trying to solve others.
In the information age, there are newer and better ways to screen out fraud and guarantee validity than to publish papers in high-profile journals that people are expected to trust simply because they have been peer reviewed. (Science reporters, among the worst of the believers, have tended to treat each issue as a message from the gods.) We want to see the day when Darwinian storytelling is no longer immune from scrutiny in the public arena, when scientists will be praised for the quality of their observations rather than their imaginations, and when claims outside the bounds of evidence are shamed out of science.
Public review will not solve everything. Scientists risk their life work being stolen by rivals. But an author knowing his or her ideas will be exposed to knowledgeable critics willing to tie their names to their criticisms may be forced to act more reserved in their conclusions. That can be a good thing. No initiative or process is a substitute, however, for old fashioned honesty, an absolute requirement for credibility in any line of research.