Peer Review and Science2.0 Talk

On March 15, 2010 I spoke on "Peer Review and Science2.0: blogs, wikis and social networking sites" as a guest lecturer for the “Peer Review Culture in Scholarly Publication and Grantmaking” course at Drexel University. The main thrust of the presentation was that peer review alone is not capable of coping with the increasing flood of scientific information being generated and shared. I make arguments to show that providing sufficient proof for scientific findings does scale and weakens the tragedy of the trusted source cascade.

The students were mainly in a technical writing program. Lawrence Souder runs the course and has set up an impressive list of guest speakers this term. I think that these topics are at the core of what it is going to be like to write about science in the next few years. Communication channels and information sources are only going to multiply even more and learning how to navigate this evolving system will require effort and skill.

If they took anything away from my talk, I hope they question all their information sources - even those labeled by a particular group as a "trusted source".

Peer Review and Science2.0

View more presentations from Jean-Claude Bradley.

From ONS to Peer Review: our JoVE Article is Published

Our article "Optimization of the Ugi Reaction Using Parallel Synthesis and Automated Liquid Handling" is now published on the Journal of Visualized Experiments (JoVE). I am very pleased with this because it showcases some interesting approaches to communicate science that were not possible not so long ago.

First, and foremost, this demonstrates that lab notebook pages and blog posts can be used to support claims made in a peer reviewed article. In a way this isn't drastically new since it has been possible for a while now to cite web pages in the peer reviewed literature. The key question is whether the reference is appropriate, regardless of its format. When providing a reference for a melting point or spectrum, nothing is more relevant that the lab notebook page where the specific batch of product was obtained and characterized.

Second, we have demonstrated that it is possible carry out research under Open Notebook Science conditions, write an article openly on a wiki, post it on a pre-print server (like Nature Precedings) and finally publish it in an peer reviewed journal. No, this won't work with every publisher. But if communicating science openly (beyond the confines of the regular Open Access model) is important to you, there are options out there that don't take anything away from the traditional system of academic validation.

Third, this is a good example of the use of video to enhance the communication of a protocol for a chemical reaction. But this is not a shortcut by any means. The process of writing a script and preparing for the shoot was very time-consuming because we were describing a whole workflow. When using video as raw data to record details of a specific experiment, it can actually save time that would otherwise be required to describe using text.

Finally, JoVE is an example of an Open Access journal with some Web2.0 capabilities, like the ability to leave comments and label them as agreeing or disagreeing with the authors. The final article can now also serve as a location for continuing the scientific conversation.

Happy Accidents: A Must-Read for Open Scientists

I usually limit my book reviews to Goodreads or Shelfari but this one deserves much more attention.

In Happy Accidents: Serendipity in Modern Medical Breakthroughs; When Scientists Find What They're NOT Looking for, Morton Meyers reviews examples of the unpredictability of scientific progress.

This could just be a collection of interesting anecdotes - and some of the stories are truly fascinating. My favorite is probably the discovery of platinum compounds for the treatment of cancer. It came about from the accidental electro-dissolution of a platinum electrode during an experiment studying the effect of electricity on cell cultures!

But Meyers goes further and uses these examples to make larger observations about the way science operates today in both academia and industry. A quote from the preface foreshadows the tone of the book:

The dominant convention of all scientific writing is to present discoveries as rationally driven and to let the facts speak for themselves. This humble ideal has succeeded in making scientists look as if they never make errors, that they straightforwardly answer every question they investigate. It banishes any hint of blunders and surprises along the way. Consequently, not only the general public but the scientific community itself is unaware of the vast role of serendipity in medical research. Typically, a discoverer may finally admit this only towards the end of his or her career, after the awards have been received.

And starting on page 304:

An applicant for a research grant is expected to have a clearly defined program for a period of three to five years. Implicit is the assumption that nothing unforeseen will be discovered during that time and, even if something were, it would not cause distraction from the approved line of research. Yet the reality is that many medical discoveries were made by researchers working on the basis of a fallacious hypothesis that led them down an unexpected fortuitous path.
....
The peer review system forces investigators to work on problems others think are important and to describe the work in a way that convinces the reviewers that results will be obtained. This is precisely what prevents funded work from being highly preliminary, speculative or radical. How can a venture into the unknown offer predictability of results?(my emphasis)
....
Indeed the basic process of peer review demands conformity of thinking and disdains a maverick's approach.
....
What it comes down to is this: Who on a review committee is the peer of a maverick? (my emphasis)

The fact that some of us in the Open Science community are discussing this does not mean that we are advocating for the abolition of peer review or the NIH. We are not that naive. We still submit proposals and manuscripts for publication in peer-reviewed journals (although given a choice we probably would pick an Open Access journal over one running on a paid subscription model).

The point is what we do in addition to all those traditional processes.

We can share our failed experiments. We can share our research plans. We can discuss science freely admitting what we don't know. We can record our talks at closed meetings and make them public. We can initiate and participate in serious scientific conversations going on in the blogosphere without worrying about everyone's title and rank.

Basically, we can collaborate in ways that are most conducive to serendipitous discoveries. The free social software, databases and other infrastructure now available make this information exchange easier than ever.

The key question for a researcher today: to hoard or not to hoard?

To me, it seems likely that data hoarders will find it more and more difficult to claim priority for a contribution when competing against loose associations of open collaborators motivated by insatiable curiosity.

Some of the folks from the funding side are getting it. Take a look at SubMeta.