Tuesday, January 6, 2009

Responding to Peer Review

So far in this series, I've discussed my approaches to reviewing a paper. Aside from writing a quality paper and recommending potential reviewers, the process is out of the author's hands. Reviewers do their thing, and then editors do their thing. After a few weeks or months, you (the author) get an email in your inbox with the dreaded(?) results.

Inevitably, this email never has the verdict in the subject line. So, you'll have to click and open the email. . .and see one of several possible overall responses:
  1. Reject without review. This is the most common sort of response you'll see if you're uppity enough to aim for Science or Nature. You might also get this if the editor deems the paper unacceptable for grounds of style (you didn't follow the instructions to authors), scientific quality (often reserved for the wackiest of paleo conspiracy theories), or scope (your focus is too narrow for the intended audience--this is what you'll see most frequently). Usually, the editor won't provide any detailed comments beyond a "thanks, but no thanks."
  2. Rejection after review. This may be done for reasons of scientific quality or scope. Perhaps your arguments or methodology are flawed. Sometimes, the reviewers will think it's a good paper but just not of sufficiently broad scope or importance for the readership of the journal.
  3. Accept with major revision. The paper has some good stuff, but needs a lot of work in one area or another. Maybe additional data are needed, or expansion of a description.
  4. Accept with minor revision. I love seeing this one! The paper is generally good, except for a few minor points. Change these, and you're as good as published (barring any last minute antics from the editors).
  5. Acceptable as is. This has never happened to me, and I don't know that it even happens that often in the most lax of journals (and almost never in the most rigorous!).
One other thing. . .
Don't be afraid to bug the editors if you think the review is taking an abnormally long time to get back to you. More than once, I've discovered that a reviewer had forgotten about my paper they promised to review, and my little push was enough to get things moving again. Use your judgement, and ask around to friends or colleagues who have published in the journal previously about their waiting periods. In these days of electronic manuscript submission, three months is not an unusual turn-around time, and six months usually is an acceptable time to start rattling cages.

What if you're rejected?

This happens to everyone--more than once. When you get that rejection letter, take a good, hard look at it. Usually, there are some good reasons for rejection (and editors and reviewers who are doing their jobs will give you a good summary of these reasons), and if you fix up the paper you might have better luck in another journal. If it's a question of scope, resubmit to a more specialized journal (more on journal selection in an upcoming post). If it's a question of scientific content, try to improve your research and resubmit elsewhere. If you think you were unfairly scorned, you might consider a response to the editors - but only if your case is very, very good. There have been one or two times when I've been close to doing so, but decided against it after "cooling off" for a day or two. It stinks, but the editors and reviewers are often correct in their decision to reject your treasured manuscript (at least in my own personal experience)!

What if you need to do revisions?
If I get a request to revise and resubmit, I'll usually set the reviews and manuscript aside for a day or two, and then return. This often saves some silly mistakes or misinterpretations in the heat of having one's pet project marked up by an anonymous reviewer. More than once, I've returned to a review only to find out that the reviewer didn't actually say what I thought they said. This saved me a ton of work in the end!

Then, do the revisions! I'll often do the easy stuff (grammar, references, and stylistic tweaks), and then move on to the more grinding aspects of the revision. This often takes time, but it's time well spent. As I make the revisions, I'll also craft my "response to the reviews" letter. As Dave Hone mentioned elsewhere, it's not necessary to note every little change--my letter usually has a line to the effect of, "All stylistic and grammatical suggestions were incorporated into the manuscript." This letter is important--it's your chance to really highlight how you've incorporated the editors' and reviewers' comments. It never hurts to thank those involved, either.

What if they're full of it?
Sometimes, you'll get a comment or request that's completely unreasonable, or just flat-out wrong. In this case, first make sure you're absolutely certain that you're in the right--did you mis-state a point, or perhaps your phrasing was confusing to the reviewer? If this is the case, fix it and move on. If the reviewer is full of it, be proactive in a positive way. In your response to the reviews, state why you disagree with the reviewer and provide evidence to back this up. In most cases, editors will accept this. Be polite and thorough, and you're in the clear.

The last step
Let your manuscript sit for a day or two. Review all of the comments, and make sure you addressed them. Now, you're ready to resubmit. The manuscript is revised, figures are finalized, and your resubmission letter is complete. Good luck!

One final thing
Remember--the editors and reviewers are not your adversaries (well, 99 percent of the time). They are colleagues and scientists who want to help you publish the best research possible. I've had a variety of experiences with peer review--some agonizing, some ridiculously drawn-out, some finished in a breathtakingly short amount of time. In every single case, my work was improved by the process.

I'll close with an anecdote. Awhile back, I poured my efforts into what I thought was a great paper. I submitted to a high-profile paleo journal, with high hopes. It was rejected without review on grounds of being too narrow in scope. Despite this disappointment, I was absolutely ecstatic by the fact that the editor wrote two pages of comments on how I could improve the manuscript and resubmit elsewhere. Even though he didn't send it out for review, he read the paper, understood it, and took the time to provide constructive feedback. My research is better because of this one editor. I find inspiration in his efforts, and hope that you will too.

For more on responding to reviews, see Dave Hone's post on "How to write a paper." Also, Jeff Martz also offers some excellent perspectives on the importance of peer review over at his blog. I agree 100 percent.


Anonymous said...

Wow. I'll keep that in mind for the future. I'm aiming to be a Vertebrate Paelontologist myself.

Mike Taylor said...

I had a strange experience in peer review, and I wondered if any of this blog's readers have seen the same thing. I sent a manuscript to a respected journal, and the handling editor sent it off to two reviewers. As it happens, both reviewers liked it a lot, and the handling editor sent the reviews back to me with a verdict of "accept with minor revision". So far, so good.

Then, six days later, as I was making the reviewers' recommended revisions, I got a THIRD review out of the blue, not submitted via the journal's usual automated system but forwarded by hand by the editor. That third review had not been solicited by the editor, and was dated five days AFTER the editorial decision had been made. Coincidentally, or not, that review was from someone who is on the editorial board of the journal in question, though not involved in the handling of my manuscript; also perhaps coincidentally, the additional review was very, very negative -- almost comically so in light of the two very positive reviews from that actual reviewers.

I don't want to say too much about this for fear of accidentally divulging details (though I will say we have a happy ending, as the paper is now in press), but I did want to ask: is this normal? Is it even ethical? Has it happened to anyone else?

Anonymous said...

Mike, that is *NOT* normal. The ethical-ness of it seems questionable. I guess the editor can show the MS to anyone he wants to (he'll often hav to show it oth other editors and it might go through several referees before it finds a home for real review), but once the decision is made I don't see how they can change it. I am not so surprised that another editor spotted it and asked to provide a review, but to do so after the others were doen, and to be submitted by the handling editor seems both strange, unnecessary and unpleaseant to the point of being unethical.

On a happier note, responding to Andy's main text (though related to Mike's tale) I did actually once get a pair of reviews that said "ready for publciation". As a result the editor then got two further reviews for the manuscript full of nonsense comments I then had to rebut. So having got a 'perfect' review I still had to rebut comments anyway. I should note that all 4 reviews arrived togehter, but the editor explicitly said in his covering letter than the first 2 were so good he asked for the other 2!

Andy said...

Yeah, that's totally not normal as compared to any review I've ever experienced. I've been involved in one or two cases where I was asked to be a third reviewer to help resolve "split" review decisions, but nothing where a completely random editor threw extra comments in. As for ethics. . .it does sound questionable. I seem to recall that all of the editors do have access to all manuscripts for a journal, so I guess it's not like he "stole" the MS. . .but, this might fall in the category of "barely ethical but really jerkish actions." So what Dave said, basically.

Glad the story has a happy ending, though! And I'm totally jealous of Dave getting a "ready for publication" review!

Anonymous said...

Great post. WRT Mike's story, my experience as a journal editor would indicate that, yes, the mysterious third reviewer had abused his/her power as a member of the editorial team. It almost looks as if they muscled in after seeing Mike's ms in the system, somehow convincing the handling ed that their views were needed because it was 'their' subject. That's pretty deplorable.

As it happens, I do know of one other case of a rather similar - though even more serious - thing happening in the world of journal editing. In that case the relevant individual was (so I understand) removed from the editorial team as a result of their actions.

Incidentally, palaeontology/palaeozoology is definitely not exceptional in this regard. Unfortunately, poor and/or absent ethics abound. Of course, this does NOT make it ok.