dreamingspires posterous

GPeerReview are bringing out a peer review tool in a social networking environment.  From a user point of view it looks very good and could have major implications for academic houses and institutions who have business models based on peer review. 

The aim:
“We intend for the peer-review web to do for scientific publishing what the world wide web has done for media publishing”

This is what GPeerReview say about their product and they are none too gentle with traditional academic publishing houses:

“GPeerReview is a command-line tool that makes it simple to write a review of someone's work and digitally sign them together.

How does it work?

1. First, you read someone's paper.
2. Next, write a review. (The review is just a simple text file that contains a few scores and your opinions about the paper.)
3. Use GPeerReview to sign the review. (It will add a hash of the paper to your review, then it will use GPG to digitally sign the review.)
4. Send the signed review to the author. If the author likes the review, he/she will include it with his/her list of published works.
5. Prospective employers or other persons can easily verify that the reviews are valid.

Why?

• Peer reviews give credibility to an author's work.
• Journals and conferences can use this tool to indicate acceptance of a paper.
• Researchers can also give credibility to each other by reviewing others' works.
• This enables researchers to publish first, and review later.
• It meshes seamlessly with existing publication venues. Even the credibility of works that have already been published can be enhanced by obtaining additional peer reviews.
• A decentralized social-network of reviewers and papers is naturally formed by this process. The structure of this network reflects that of the research community.

Graph Analysis
Analysis of the network/graph of reviewers and papers can provide an automated technique for evaluating how well-connected a particular author is with the research community. For example, one analysis algorithm might work as follows:

1. Seed the analysis with a set of well-established researchers in the field. (These might include the chaired professors at your university and a few of their peers that they select.)
2. Use a max-flow/min-cut algorithm to determine the smallest set of reviews that must hypothetically disappear in order to sever the individual being analyzed from the research community.
3. The sum strength of these hypothetically severed reviews now serves as an indication of how well-connected that individual is with the research community.

Of course, this is not the only graph-analysis technique available. There are numerous algorithms and metrics for measuring the connectedness, centrality, or other aspects of an individual's relation to a community. Further, the reviews themselves contain evaluations of significance, novelty, quality, and other values that may assist an evaluation. You can design your analysis to emphasize your own preferences. Thus, there is no single magic formula that can be gamed!

Ultimate Goal
We intend for the peer-review web to do for scientific publishing what the world wide web has done for media publishing. As it becomes increasingly practical to evaluate researchers based on the reviews of their peers, the need for centralized big-name journals begins to diminish. The power is returned to those most qualified to give meaningful reviews: the peers.

As long as big journals provide a useful service, this tool will only enhance their effectiveness. But the more they take months to review our publications, and the more they give unqualified reviews, and the more they force us to clear irrelevant hurdles prior to publication, and the more they lock up our works behind fees and copyright transfers, the more this tool will provide an alternative to their services.

With GPeerReview, you can:

• Publish immediately (and get reviews later),
• Seek an unlimited number of reviews,
• Verify the integrity of the reviews,
• Verify the credibility of the reviewers,
• Publish without limitation on format, style, or number of pages,
• Maintain complete copyright ownership of your works, and
• Enhance the acclaim of your already-published works.”

Looking at this objectively I can't help but think that this is could be a good thing for the dissemination of research and content in a social networking environment from a users perspective. Especially as it can be done alongside traditional publishing review processes in the interim. In the long term I suspect these traditional peer review processes will disappear with major implications for academic publishing companies.

We could of course just bury our head in the sand -- with regard to book sales strategy I heard one Publisher say in a recent Sales Conference:

"We must be doing it really well because it's still working"  Nothing could be further from the truth.

Loading mentions Retweet