Q and A

Why are you trying to fix peer review?

I had an idea. Thus far, the idea has withstood the scrutiny of quite a few scholars. Until someone finds a flaw with my idea, it seems like I should continue to pursue it, just because I can. If you find a flaw, please spare me from wasting my time on this project. I have other things I'd like to do.

Who are you?

http://gashler.com/mike. A lot of other people have joined the project since it first started, though, so I'm not alone here.

How will an endorsement organization make money?

An endorsement organizations doesn't provide as many services as a journal. Consequently, it should be able to operate on a shoe-string budget. I would hope that scholars and Universities might operate them just because they are needed, or perhaps for the associated name-recognition that would come from operating a good one. For the most part, I don't anticipate that running an EO would be a particularly lucrative undertaking.

Is GPeerReview a form of open peer commentary?

No. Here's a table of differences.

What good is peer review if authors can simply discard all the negative reviews?

All ideas are assumed to be poor and are consigned to obscurity unless they come well-endorsed. This is one thing the existing system does right, and we don't seek to change it. You don't mention on your c.v. how many journals rejected your paper before it was accepted. If you are really passionate about an idea, you should be able to keep trying until you obtain a sufficient level of endorsement. If we throw out every idea that isn't received well on average, we'll throw out all the truly innovative game-changing ideas with them.

If people can publish first and seek endorsements later, won't the world be flooded with ideas of poor quality?

The world has already been flooded with ideas of poor quality. The purpose of peer review is to identify the good ideas, not to prevent or delay publication.

How do you evaluate the quality of a researcher?

We don't. It is absurd to think that some automated tool could use a magic formula to calculate the quality of a person. Our tools just provides a mechanism for signing endorsements. The endorsements are what give credibility to an idea, and by association, to the people who produce good ideas.

How do you evaluate the quality of an endorsement?

If the endorsement comes from a journal or EO, then the name of the journal or EO should speak for itself. (This is no different than the existing system.) If an endorsement comes from another scholar, then the scholar's name determines the significance of the endorsement. Of course, there are too many scholars out there for anyone to recognize them all by name, but there are graph analysis techniques that can arguably provide valuable information. When researchers review and sign each others' works, a decentralized social network is naturally formed. This network will eventually mirror the structure of the research community. If, for example, you wanted to determine how influential a particular scholar is with his research community, you could use an analysis technique that gives the information you want. We think the following algorithm might be a good choice:

1. Use a max-flow/min-cut algorithm.
2. Represent the individual being analyzed as the "source" node.
3. Select a number of well-known scholars of high reputation in your field. Represent each of these as a "sink" node.
4. Perform graph-cut to identify the sum strength of endorsements that would need to be hypothetically severed in order to separate the source from the sink.
5. Compare this value with those of other reputable researchers in your field.

Doesn't Arrow's Impossibility Theorem imply that no evaluation technique is immune from being gamed?

Yes. We don't provide any magic formulas for evaluation. We think every institution should evaluates the graph structure in whatever way suits its needs the best.

What's wrong with establishing a magic formula to evaluate a person based on review scores?

1. The whole community would never settle on a single formula.
2. If they did, no one would believe it.
3. If the community did settle on one and people trusted it, smart people would find a way to boost their evaluation score without deserving it. (This is called gaming the system.) All magic evaluation formulas can be gamed. No Slashdot-like moderation system can possibly suffice for scientific publications.
4. Quality is not a one-dimensional concept. Any magic assessment formula would have to make far too many compromises in order to collapse various attributes down to a single number. By the definition compromise, this would result in something that no one believes is very good.

How do digital signatures work?

If you want to know details, read about asymmetric-key cryptography. Basically, every researcher gets two entangled keys. He keeps one private (perhaps on a flash drive hidden under his bed), and makes the other key public (on his web page). He can generate a signature using the private key. This signature can be validated with the public key.

What incentive do credible researchers have to review my works?

If your idea is really good, reviewers will want to have their names associated with it. Also, performing reviews makes a scholar appear to be an active participant in the community. Beyond these reasons, this isn't really our problem, it's yours. We'd love to fix all of the world's problems, but we don't know how. Our tools don't make that problem any worse, they just doesn't fully solve it either.

How will all the papers people self-publish be organized?

That is also a problem we are not trying to solve. This problem is better solved by someone else. Someone will probably create a site that aims to index these publications. If not, there's always search engines.

How do I get an asymmetric key-pair?

gpeerreview gen-key

How do I review and sign someone's work?

First, evaluate the work and write a review. You can do "gpeerreview template" to generate a review template. Fill out the review, then do "gpeerreview sign endorsement.txt ThePaper.pdf". This will generate a signed-review, which you can then send to the author.

What does an author do with a signed review?

Put all the reviews for the same work into one text file. Put this text file next to your work. For example, if you have a web page that lists your published works, you can add a link for "reviews" next to each work.

What happens if an author tries to change an endorsement?

Then the signature won't match and he will expose himself as a fraud. The author can place his own comments or rebuttals before or after the endorsement (it's his resume), but he can't change the endorsement itself or the signature will no longer validate.

What if my private key is compromised?

The encryption stuff is really just a convenience to enable automated verification. It's not strictly necessary for this system to work. You could always send someone an email or call him on the phone and ask him if an endorsement is legitimate. It's not like you're transmitting money to an untrusted person. If people actually do try to falsify endorsements, they will certainly be caught. It's not like they can grab their tenure and flee the country with it. But if you want to avoid having to make a lot of phone calls, just don't lose your private key.

Why don't you use a certificate authority?

Decentralization is extremely important to the notion of peer review. Anyone that becomes "the authority" will certainly corrupt the system. There is no need for a certificate authority. What problem would they solve? If you keep resumes on file, falsified claims will eventually come out by themselves. Digital signatures only make it easier to catch liars. If someone creates a cloud of phony signatures, there will be no path connecting those signatures to reputable researchers. Using a certificate authority would be a very costly solution to a mostly non-existent problem.

How should citations be done?

Because citations belong in a paper, they is beyond the scope of GPeerReview. We do not dictate the style or form of an author's work. The author should know best how to present his/her idea. Ideally, we would like people to provide a hyperlink to the author's c.v. in each citation. This would properly give the author credit. It would also enable the author to update the list of endorsements for his/her paper. It would also enable the author to specify meta-data regarding the paper (Bibtex, retractions, acknowledgments, how-to-cite-me, brief plugs for newer versions, etc.) And, of course, the author will provide a link to the actual paper, because he/she wants you to read it more than anyone else. It is nonsensical to expect the citer to include meta-data in the citation. Nevertheless, GPeerReview cannot and should not try to enforce this.