The Internet and Me and Complete Harmony

Yesterday, Bossman asked me the following question: "Do you think that the ACS will eventually host a reddit-type scoring of it's publications?  Do you think it's a good idea?  Nature's experiment with open peer review failed, but this approach may have more appeal." As he put it, despite my inexperience with peer review for major publications, he wanted the opinion of someone "embedded in the internet." And embedded I am. Now I'd be remiss if I didn't say that a lot of my opinions were shaped by reading Paul's blag and the rather prolific commenters, especially on this post. Below lies my response.

I constantly waver back and forth on this issue.

I'm not familiar with Nature's open peer review experiment, and since I've never been a part of the peer review process for any publication,my opinions are based simply on what I think seems the most reasonable approach. I don't see ACS adopting a reddit-style scoring system anytime soon, and I think one of the biggest obstacles facing a more open peer review process is clout. ACS journals are respected, in part, because of the way they go about their peer review process. It's a tried-and-true technique. However, we've seen the limits of the success of this peer review process lately, in particular with the serial retractions of Dalibor Sames' papers. Some feel that the retraction process was handled smoothly, but others believe that if things were a little different, then the issues could have been handled much more quickly. It took nearly four years for some of the papers to be retracted by Sames. If the peer review process was more open (more "democratic," I suppose), so say the open review advocates, this scandal could have been taken care of in a much more quick and efficient manner, and it would prevent these things from happening as often. Right?

Well, yes and no. Although I'm of the "internet generation" and pretty much everything I do revolves around it in some way, I think that exposure has made me even more suspicious of the advocacy of open review. On places like reddit and digg (especially digg), one senses the tyrrany of the majority implicit in these communities. Anyone whose opinions differ from the majority are quickly slienced, it seems. To be fair, those who are silenced probably had it coming (poor arguments, foul language, etc.), and regardless of what you think of such a democratic method (I abhor it, personally), places like digg and reddit are talking about trivial matters. We are talking about professional science journals. And is that what the future of scientific publishing should be? Tyrrany of the majority? Absolutely not. In order for a reddit-type system to work, there would have to be restrictions put on who is allowed to review and comment on papers. For example, JACS papers could only be commented upon by ACS members, and no anonymity would be allowed- you make a comment, people will know who you are and what you said. I think, if the open review process is going to work, it has to work within a community that respects its peers- transparency is essential. The internet is full of assholes who ruin communities, and that REALLY needs to be taken into account.

Let me present one example that addresses another issue I have with an open review process. Let's say Prof. X publishes a paper on Topic A. It gets accepted into JACS and the Topic A paper online becomes a forum on Topic A, as all of these types of paper-based forums will become. The relevance of Topic A will be addressed, Prof. X will answer questions, defend his work, argue with Prof. Y who is also working on Topic A, etc. That would certainly be a paradigm shift- Prof. X and company will now have to actively defend their work on the internet. Of course, you could have Grad Student Z just do it for you, but get a bunch of young professors who incorporate such work into their daily lives, and the people will expect more author AND prof dialogue. I think older professors, weaned on the older methods, may be alienated from such a method. It would be a major shift in how things are done, and I don't expect a lot of the older crowd to be able to accomodate. There is also the issue of reddit-style popularity and how notoriety may affect neutrality: popular articles may be popular only because of who wrote them, and it might be difficult for new PIs to get the kind of new notoriety that they need to establish themselves. As it stands, a paper by Marty Burke can stand next to one by EJ Corey as potential equals. A reddit-style "popularity" system may keep such an equality of oppourtunity from occuring.

Sorry, that was long, but in conclusion: I don't expect a HUGE shift in how ACS journals are reviewed any time soon, but I think it would help transparency in review, which in turn can aid in scientific dialogue- the internet has made everything faster, so why not dialogue in research as well? However, such an implementation, if a journal is to stay respectable, will have to impose limits on how dialogue is performed. I can see great things happening from it, but with great power comes great responsibility.