Last week, John Bohannon published an article, “Who’s Afraid of Peer Review?” in Science, a AAAS journal. The article described how he had submitted various versions of a spoof article (with many flaws) to lots of different Open Access journals. The spoof papers were accepted by many of them, and the author concluded that he had shown that their peer-review system was poor.
Here is the story as reported in The Economist, and in The Guardian.
However, this was not the end of the matter. Here’s a selection of some of the criticisms.
Kevin Smith, Scholarly Communications Officer at Duke University:
Different versions of a “fake” article, which the authors tell us could easily be determined to be poor science, were sent to a lot of different OA journals, and it was accepted by a large number of them. This has set off lots of smug satisfaction amongst those who fear open access — I have to suspect that the editors of Science fall into that category — and quite a bit of hand-wring amongst those, like myself, who support open access and see it as a way forward out of the impasse that is the current scholarly communications system. In short, everyone is playing their assigned parts.
Smith identified two main errors in Bohannon’s method: that his study lacked any control, and that it used a pool of journals which had already been identified as having poor procedures for peer review. Furthermore, it only included OA journals which charged article processing charges, and did not include any non-OA journals.
Michael Eisen, biologist at University of California at Berkeley pointed out that poor peer-review is not just a problem in Open Access publishing:
[the] sting exposed the seedy underside of “subscription-based” scholarly publishing, where some journals routinely lower their standards – in this case by sending the paper to reviewers they knew would be sympathetic – in order to pump up their impact factor and increase subscription revenue.
He concludes: “the lesson people should take home from this story not that open access is bad, but that peer review is a joke” and “There are deep problems with science publishing. But the way to fix this is not to curtain open access publishing. It is to fix peer review.”
Heather Joseph, Executive Director at SPARC, concludes:
While the Science “sting” does not lend itself to drawing any truly valid conclusions about the overall quality Open Access journals, it does raise significant questions about potential flaws in the peer review and editorial processes used throughout the scholarly publishing marketplace. Perhaps an unintended outcome of this “sting” operation will be to spark a closer scrutiny of this aspect of journal publishing.
I think this episode gives a good example of how the current battle/discussion over the future of scholarly publishing: who will pay for it, and how much it cost; is not guided or overseen by any academic committee. In a commercial marketplace, organisations which currently profit from scholarly publishing want to continue to do so, and will employ various forms of pressure and propaganda to ensure the future of their business.
It’s Open Access Week soon – a great time to get involved in discussions and education about these issues. Alma Swan’s UNESCO report, Policy Guidelines for the Development and Promotion of Open Access, is a good place to start reading about OA. Learn something new, then go and teach it to someone else…