Walking to work this morning listening to BBC Radio 4 I tuned in to two pieces about Nature. They were triggered by the journal publishing the results of an experiment we undertook where we invited scientists to submit their papers for public (as opposed to the more traditional system of confidential reviewing) peer review. You can hear the debate at 7.25am here between the editor of Nature and the editor of an online scientific journal from the charity-supported Public Library of Science. There was also an earlier journalistic piece at around 640am. I've also pasted in below an article from the Wall Street Journal on the same subject.
Journal Nature Drops Open-Editing Experiment
Few Scientists Accept Offer to Critique Work Or Be Critiqued Online
By NICHOLAS ZAMISKA December 20, 2006 1:07 p.m.
The journal Nature is abandoning an experiment aimed at bringing Wikipedia-like group editing into the world of scientific publishing.
For several months beginning this past summer, Nature has invited scientists whose articles were shortlisted for publication in the journal to first post their work online for public review. Normally, a handful of scientists review such submissions anonymously.
Nature's experiment was reminiscent of Wikipedia, an online encyclopedia that is written and edited collaboratively by many of its readers. The scientific magazine's move was intended in part to see if a more-open review process could expose low-quality or fraudulent papers that critics of the current system say too often slip into print.
But Nature, which is published by a unit of Macmillan Publishers Ltd., said in an editorial in Thursday's issue that it was ending the experiment due to lack of participation. The journal found that in the competitive world of scientific publishing, the vast majority of authors were unwilling to post their papers and few scientists were willing to criticize their peers' work publicly by posting comments on Nature's Web site.
Last January, the journal Science retracted two papers by South Korean scientist Hwang Woo-suk after evidence emerged that some of his experiments in cloning were fraudulent.
"It is an interesting question whether a more-open peer-review process might have led to the detection of Hwang's fraud," read the Nature editorial. "At present, however, the level of interest in open peer review is too small to hope for such an outcome."
Of the 1,369 shortlisted papers submitted during the trial, which ran for around four months, only the authors of 71 were willing to post their work online, Nature said. The papers that were put online only received 92 technical comments, according to the journal, which said that scientists seemed unwilling to comment candidly on others' papers, given that comments weren't anonymous.
The journal concluded that "most of them are too busy, and lack sufficient career incentive, to venture onto a venue such as Nature's Web site and post public, critical assessments of their peers' work."
Meanwhile, another experiment with collaborative editing got under way this week. A new online scientific journal called PloS ONE invites readers to post comments or questions about articles once they are published. PLoS ONE is published by the Public Library of Science, a nonprofit scientific publishing project aimed at creating a library of scientific literature that is accessible to the public.
It's great that a subject so apparently arcane as scientific peer review should be considered important enough to warrant two slots on the most important radio programme in the UK and a feature in the world's leading financial newspaper. What is not so great is that the discussions manage to confuse open reviewing with free access, comment with criticism, freedom of information with free information, an excellent system which catches nearly all attempted scientific fraud with a flawed system which allows fraud to happen, the desire to speak confidentially and openly as opposed to the apparently open but necessarily guarded alternative. In other words and as usual, a tricky and important debate has been reduced to a few soundbites of little value and significant distortion.
On a more immediately important subject, the disgraceful death sentence imposed on the health workers in Libya. Declan Butler has written a professional state of play piece. Do read it and do, if you can, support the resistance to this terrible injustice.