Publishing research can be a frustrating, exciting, and anxious experience all wrapped up into one for junior scholars. There is a lot of excitement surrounding the submission of a manuscript, receiving an acceptance letter or favorable review of a manuscript. However, the critical reviews and rejections that are also part of the academic game are disheartening - in some cases they may make a doctoral student or new PhD rethink an academic career.
As a seasoned academic with an active publication record, I'm here to tell you that the bad reviews happen to all of us! Sure, our writing gets better over time. We develop acumen in our writing style or tailoring manuscript content to the audience of specific journals. However, the blinded process to peer review is a double-edged sword - it reduced biases among reviewers who may (unknowingly) form judgement about the article based on their perceptions of the author(s), but the anonymity allows some reviewers to be nonconstructive, unnecessarily critical, or even downright mean.
The goal of this week's post is to give junior scholars reassurance that a bad manuscript review or rejection does not necessarily mean that they are not cut out for academia. I consulted with a number of my mid-career and senior colleagues to ask them about terrible reviewers they have encountered. Rather than retreating from academia, these scholars used what they could from terrible reviews to improve upon their work and keep marching on until they reached success in their careers! From their stories - along with my own - I hope to demystify some of the blinded peer review process by categorizing some of the terrible reviewers you may encounter.
1. The snarky reviewer.
Sometimes reviewers ARE JUST MEAN. We get it - they didn't like the manuscript. But is it really necessary to go out of their way to make comments that are potentially hurtful and do not contribute to our understanding about how best to improve upon our work? Some examples:
I once had a reviewer claim that I "obviously knew nothing about qualitative methodologies" because I referred to my interview-based study as "a qualitative study" and "there are more methods out there that can be classified as qualitative." I am still confused by the comment - do we need to use ALL qualitative methods for something to be qualitative? Despite the reviewer's lack of confidence in my knowledge, I was two months away from receiving tenure and had been teaching a PhD-level course in qualitative methodology for several semesters.
A reviewer very recently provided a wildly condescending review of a paper I'd written. S/he commended me for trying to get a publication as a doctoral student and acknowledged how hard it is to get the hang of publishing in academia. I have been a professor for 8 years and have 67 publications.
I once had a reviewer who actually said that they found the abstract to be so terrible that it drew him/her in like an American Idol preview with untalented performers. That was several years ago at a time I was thinking about getting a job outside of academia. However, we have since worked on the manuscript and have turned it into several successful projects.
Sometimes the "mean reviewer" seams out of place, because other reviewers have positive comments about the same manuscript.
I had a reviewer who just blasted my piece, and made me wonder if the reviewer even bothered to read it, especially when the first reviewer looked it through and talked about how it was a great contribution.
My experience was when one reviewer commented very briefly and overall liked the the study, while the other one trashed the statistical analysis and made several suggestions for higher level statistics to be conducted. The irony was that this is a little pilot study and had a very small sample size. It was beyond the scope of our little study to do such extensive analysis of such a small dataset. The editor in my situation was very understanding and as long as I responded to both reviewers addressing each comment, the article was accepted without any additional statistical analysis being conducted.
2. The reviewer who wishes s/he wrote your manuscript.
There are also reviewers who seem to wish that they thought of writing your manuscript first. This is the reviewer who makes EXTENSIVE suggestions for edits, especially proposed changes to your writing style that do not necessarily improve the quality of the research, but it is how s/he would have written it.
I once had a reviewer make several pages in suggested edits. At first I was overwhelmed at the number of comments made and worried that it would be too difficult to revise the paper. However, as I sorted through I realized that most of the comments were small changes and/or synonyms to phrases and terms I used. While I adequately supported my arguments, the reviewer gave me extensive references that they suggested I add or replace my existing reference with. This was a co-authored paper with other experts in this field, so it seems that many of the comments reflected the reviewer's preferences, rather than the quality of the manuscript.
The good news about this type of reviewer? They are often easy to please - they will tell you EXACTLY how they want the article revised, giving you exact phrases, terms, and references they would like for you to use. Hopefully, their suggestions do not cause any ethical and/or rigor issues for you (although your ego may be a little hurt).
3. The reviewer who didn't read your manuscript.
Sometimes reviewers' comments are confusing because they do not reflect the manuscript that you submitted. This can lead to confusion about how to address their comments (or at least how to respond to their comments without being snarky yourself!).
I once had a reviewer make a comment about the focus groups I conducted in the study, though there were no focus groups involved with data collection.
Reviewers sometimes are given numerous manuscripts to read and comment on - especially if they are on an editorial board. Ideally, they should keep their comments organized, but there may be confusion or mix-ups from time to time. Note, that this has also happened to me for federal grant reviews.
4. The reviewers who want opposing edits.
I did not receive any comments about this - most colleagues discussed the comments of a single reviewer. However, another common problem is when reviewers do not agree on how to revise a manuscript. One reviewer wants you to add an estimated two additional pages of content while the other thinks that the article needs to be shorter. One wants more details in the findings while the other wants less. This can be more frustrating than the reviewers mentioned above, since we may be unclear on how to edit our manuscript.
It could be worse! Be thankful this didn't happen to you...
In my research on peer review, I came across two stories that were exceptionally horrifying. One was from a tenured colleague who recounted an early experience in publishing:
I got a paper accepted. It was the first one. I asked the editor what I should do next and he said, sit back and wait...So I did... Then he died without telling anyone he had accepted the paper. I kept waiting for next steps and was trying to be patient because that is what he said I should do...about a year later, I checked into and found out what had happened. The new editor honored his commitment (I still had all the emails.)
The other story was not from a colleague, but from a Letter to the Editor published in the Annals of Internal Medicine. It provided detail about a case where researchers submitted a manuscript for peer review, only to have one of the reviewers take the manuscript and submit it to another journal as their own. Thankfully, the original researchers caught the culprit and the egregious instance of plagiarism was revealed to all.
The take away.
Sometimes, it is difficult when we are reminded that as academics we chose careers that involve constant and ongoing peer review - our work is reviewed when we submit manuscripts, apply for grant funding, and even want to be promoted or tenured. Sometimes the feedback isn't pleasant. Though bad reviews may be frustrating, we can thicken our skin and try to find the deeper meaning of these terrible comments and use them to move forward and become successful. At a minimum, we can learn from bad reviewers about how to be better, more constructive reviewers ourselves! Need a good laugh after a bad review or rejection? There is a Twitter Account dedicated to snarky review comments. Read them, and laugh knowing that you are not alone.
Though peer reviewers may often hide behind the cloak of the blinded review process, It is good to know that not all terrible reviews go unpunished. A previous article in Science highlighted the response to sexist remarks by a peer reviewer for Plos One, which ultimately removed the reviewer from their peer review database. A small victory for those of us burned!