Heinz Pampel of the Helmholtz Gemeinschaft’s Open Access team pointed me to this interesting project on Google Code.
A few things should probably be noted in conjunction with GPeerReview. Firstly, it is not associated with Google in any immediate way, but merely lives on Google Code. Some blog posts suggest that it is an ‘official’ Google project, which as far as I can see it isn’t – at least not yet.
Secondly, the process envisioned by Mike Gashler, the main GPeerReview developer, departs from the traditional review process (as for example most journals use it) in several important (and positive) ways.
Here is the publishing process according to Gashler:
So, you’ve got a good idea. You’ve done some experiments, gathered some results, and written a paper. Now what?
- The first thing you should do is pre-publish your work. Put your paper, your datasets, scripts, results, etc. on a public server. This does two things: 1- It ensures that science can move rapidly (without waiting for a response from a slow journal), and 2- It protects you from dishonest reviewers who might steal your ideas. [..] If some journal doesn’t permit works that were pre-published, you should not support that journal with your ideas anyway. Such journals will try to lock up your ideas rather than promote them. This is not good for you. Pre-publishing is good for you.)
- Try to publish in a top-tier journal. A few publications with really good journals will benefit your resume/career much more than a lot of publications with so-so journals. It is well worth the extra effort required to get the endorsement of a respected journal. (Notice that up to this point nothing is different. Now, here comes the new stuff…)
- Also submit your paper to several big-name endorsement organizations (EO’s). An EO is similar to a journal, but it doesn’t care whether or not a paper has already been published, it only cares how good the paper is. An EO doesn’t publish your paper, it just reviews and (hopefully) endorses your pre-publication copy. The “editor” of the EO will solicit the help of qualified reviewers to review your paper (just like a journal). He/she will coordinate double-blind reviews to ensure fairness. If the EO decides to reject your paper, they send you a private email with suggestions for improvement. If they accept it, they will send you a digitally-signed endorsement. (See more about EOs below.) Your c.v. (resume) should list all the endorsements that you obtain for each of your works. The idea that only the publisher can endorse a work is becoming antiquated. It is perfectly reasonable for many organizations to endorse a paper.
The concept of the endorsement organization is obviously the main innovation here (and I’m not saying that other aspects of GPeerReview are not innovative).
Why is it such a persuasive idea? Because it detaches evaluation from publication, two processes which have only been conflated in yesterday’s/today’s journal publishing system for historical and technological reasons. The infrastructure that publishers provide for disseminating your work is no longer needed – it’s a relic of the paper age. What’s still needed is peer review and peer endorsement in some form, but there is no practical reason why two separate processes should remain conflated into one once the technical requirements change. Authors can (and should) take care of making their ideas accessible and their institutions should support them with doing so. But evaluation is what that the community does – not the author herself (obviously), not her institution and not publishers.
From the website:
GPeerReview attempts to makes it easy for authors to seek post-publication endorsements of their works. We provide the following tools:
- A command-line tool to digitally sign endorsements (done and available).
- A web-based version of the signing tool (about 70% done).
- Client tools for analyzing endorsement graphs to establish credibility (in planning stages).
- Additional tools to facilitate the running of endorsement organizations (in the brain-storming stages).
- Tools for analyzing citation graphs (in the brain-storming stages).
On the onset, GPeerReview has two central components: a) a facilty to endorse a publication and digitally sign your endorsement and b) a tool to evaluate the endorsements you receive.
The second component is not entirely dissimilar to PageRank in that it makes ranking via dynamic social networks possible, and that the weight of my opinion as a reviewer is dependent on how others rate me. I won’t pretend to fully understand the mathematics behind it, but it seems plausible that the combination of by-name endorsements and numerical data provided by the peer reviewing network will provide a valuable indicator of quality – more valuable than what we have at the moment, at least.
From the Q & A:
If an endorsement comes from another scholar, then the scholar’s name determines the significance of the endorsement. Of course, there are too many scholars out there for anyone to recognize them all by name, but there are graph analysis techniques that can arguably provide valuable information. When researchers review and sign each others’ works, a decentralized social network is naturally formed. This network will eventually mirror the structure of the research community. If, for example, you wanted to determine how influential a particular scholar is with his research community, you could use an analysis technique that gives the information you want. We think the following algorithm might be a good choice:
- Use a max-flow/min-cut algorithm.
- Represent the individual being analyzed as the “source” node.
- Select a number of well-known scholars of high reputation in your field. Represent each of these as a “sink” node.
- Perform graph-cut to identify the sum strength of endorsements that would need to be hypothetically severed in order to separate the source from the sink.
- Compare this value with those of other reputable researchers in your field.
As Gashler points out, another problem of the conflation of publishing with reviewing is that only one set of reviewers evaluates a publication, whethas a large and open community of people can review and endorse a publication with GPeerReview or a system built on similar principles.
So, how could this work from a practical point of view?
I think there’s another name for ‘Endorsement Organization’ and it’s Scholarly Society.
What makes it problematic for societies to publish journals is that they lack the infrastructure to act as a publisher and that maintaining such an infrastrucure is inefficient and costly. But with Gashler’s model they don’t need to actually store or archive anything. They let researchers typeset, proofread and upload their own material (or forego all of these things and risk being penalized) and instead act purely as EOs that leverage their social network qualities to provide something they are excellent at providing: a seal of quality.
At the same time this leaves room for disciplinary and institutional repositories in the system. The question of where something is stored is rather boring from a researcher’s perspective anyway – put it into your institutional repository if you want to make your librarian happy, just upload it to your website if you want to annoy him (with extra bonus points if it’s your private website), or go for something disciplinary if that feels like the best place.
What we really need are new ways of discussing our research with each other more rapidly and openly.
I think GPeerReview may be a big step into that direction.
Edit: Coverage of GPeerReview by Dean Giustini; more about open access and open peer review in Peter Suber’s blog.
