In 1974 I had my first experience with submitting to a major academic peer reviewed journal in my field. It was my initial study of what would come to be known as counterproductive work behavior (CWB). Many more on the topic would follow that would go through an increasingly more difficult review process. What was striking on this first paper is that the feedback from the editor, Edwin Fleishman, was short and to the point. Considering the scope, Fleishman wanted me to reduce the paper to a short note. I gladly cut the paper down, and he accepted the shortened version. From a contemporary perspective this was remarkable. The reviewers did not provide long lists of issues to be addressed. They did not impose their own personal views on my paper. There was no need for a point-by-point response. The editor was clear in what he expected. I did my best to do what he asked. He accepted the paper. This was a time when the ethics of academic peer review was clear. The reviewer’s job was to evaluate the suitability of a paper for publication, not to become as Art Bedeian would later write, ghost writers.
The Good Old Days
It was never easy to publish. Most submissions were rejected. In my field the top journal in the 80s rejected well over 90% of submissions. What has evolved over time is the process. Before the mid 80s you submitted your paper and reviewers would provide an overall evaluation. Was the basic question important enough to justify publication? Was your approach sound? Do conclusions make sense? These were yes/no questions. Editors asked for revisions, but they were mainly clarifications and corrections. Reviewers did not impose their views on authors. They commented on what was there and not what they believed should be there. Yes or no. Accept or reject. When I received my first Revise and Resubmit, I immediately showed it to one of my faculty mentors who told me that this was almost a guarantee of publication. Rarely did an editor ask for a revision that was rejected. It only happened if you did not do what they asked.
The Bad New Days
In the fields of industrial-organizational psychology and management, the review process has evolved into something that is unnecessarily difficult. I feel bad for young academics who have to spend huge amounts of time fighting with editors and reviewers round after round trying to get their work published. Reviews have gotten longer and longer, pickier and pickier. Reviewers impose their personal opinions and favored ways of doing things. Editors invite resubmission after revision (R&R) and routinely then reject them. Sometimes that rejection doesn’t occur until multiple rounds. Even if the paper is eventually accepted, days and weeks of hard work must be invested—time that could be more productively utilized. The field would be better off if we all could spend more time doing the research and less time fighting to publish that research. If the review process resulted in a substantially better paper or at least in corrected errors, it could be justified. But neither is generally the case. Major overhauls are imposed on papers that really just need minor tune-ups.
The Ethics of Academic Peer Review
Ethics boils down to three words, “Do no harm”. The academic peer review system, at least in my field, is hardly avoiding harm, especially to young academics who will perish if they do not publish. What we collectively do to one another creates harm in several ways.
- Emotional distress over receiving hyper-critical reviews that can undermine the confidence of young (and even some established) academics.
- Frustration over wasting time revising and resubmitting papers only to have them rejected.
- Lost time fighting a cumbersome review system. That time could be better spent in doing more research, working with students, or spending time with family.
- Practitioners sit on valuable data because it is not worth their time to fight for publication.
- Reviewers feel overloaded by having to re-review the same paper multiple times, and so they limit the number of invitations they accept. Several editors have told me recently how difficult it has become to find reviewers.
- Talented individuals with a lot to contribute give up their academic careers in part because they no longer wish to endure this system.
There Is a Better Way
Publishing has never been easy, but during my career I have seen the system evolve from something reasonable to something that is anything but. New faculty have been socialized into this broken system. They might feel the sting when they are on the receiving end, but they have been taught that it is their duty to do to others what has been done to them. Like the weather, everyone complains, but no one seems motivated to address the problem.
One of my former students said it best when we published her master’s thesis in a top journal. We went through a few rounds where reviewers insisted we do certain things. We complied and when we finally received the acceptance letter she told me that we spent all this time doing lists of things, but the bottom line of the paper was unchanged. It seemed to her that it was all a waste of time, and I agreed. This difficult peer-review system is perhaps no more than scientific rigor theater. It gives the illusion that our work is more than it actually is.
A better way begins with editors. An R&R should be a commitment to publish if certain specified things are done. Those things should be clearly explained, and they should be obviously possible. Editors should not send revisions back to reviewers but make the decision themselves after reading the revision. I have done this as an editor myself many times as I considered the ethics of academic peer review. I can attest that it is easier than dealing with a new round of reviews. This means that the threshold for R&R invitation is high. Only papers that make an obvious contribution are invited, and all the issues noted need to be fixable by rewriting. I have been fortunate to have experienced such a system as an author and as an editor.
Photo by Deepak Digwal from Pexels
SUBSCRIBE TO PAUL’S BLOG: Enter your e-mail and click SUBSCRIBE
Dr. Spector:
We DBA students at the University of South Florida have the utmost respect for you, your teaching style, and the messages that you deliver.
When I began to learn about publishing in top journals and the length of time that it takes to reach publication, I said to myself, “Why would you do that unless you are looking to be a tenured professor or want to have some “bragging” rights among your peers?”
I would also speculate that it also depends on the topic of your research. If I were to publish on a topic that other researchers may learn from and utilize, being in a top journal may make sense.
However, someone such as myself that is trying to impact millions of people through an informational movement would only stand by and watch thousands of additional construction workers take their own lives or be injured on a job site while suffering through mental illness.
Let me begin this statement by saying that I am not the brightest crayon in the box. However, I truly believe that my research, which would never come close to qualifying for a top journal, will have more impact on society than the majority of the papers that are published.
In the end, we all have to ask ourselves what is the purpose behind our research. If I knew I could submit to a top journal, and have a reasonable review process, I would. However, based on what I know, I’ll go attempt to save a few thousand more pople rather than spend countless hours in a review process having someone push their agenda and writing style on me.
Thank you for what you have done in your career, what you are doing, and for your future endeavors.
Thanks for the kind words Vince.
Publishing in an academic journal reaches a narrow audience of specialists on the topic. Their impact on the broader society is indirect and perhaps incremental as a body of research, rather than single paper, can inform policy and practice. For practitioners, it often makes most sense to partner with academics who have to publish. Many practitioners have the skills to publish, but it can be hard to justify the time. Academics are being paid by their universities to put in that time.
Thank you, Paul, for this. Yes, the process has become hardly bearable, and I do know that it discourages many students (and some faculty members) from pursuing publishing their research. I agree, although what was a tough system, has gotten even tougher (I recall some outrageous rejections after several R & Rs in the 1980s). Rather than treating others as we’ve been treated, as reviewers and editors, we should treat them as we would want to be treated.
Editors accept based on relevance to journal. Paper is published ‘as-is’. All reviewer comments are published alongside paper. A score from 1 to 10 is assigned by each reviewer on key factors (e.g.: method) as well as overall score and included with review. Median review scores are presented at the end of paper. All reviewers are identified by name, affiliation, highest degree obtained, length of time working in the field and number of relevant publications.
The READER then can evaluate submissions, reviewer comments and scores to decide for themselves how the CONTENT of the research can be used to further their own work.
This blog was interesting as always. I can see how it touched a nerve in others who responded to it. It touched a nerve in me, too. The R&R solution Paul advocates makes sense. Why send a paper back to the submitters, have them correct any mistakes or omissions, then give them a second R&R. The editor needs to make a decision regarding whether the submitter fulfilled his or her obligations as set forth in the first set of reviews. Of course, there should be freedom for submitters to contest a reviewer’s suggestion that is problematic.
I experienced a situation in which I got an R&R but the passive-aggressive editor took one year to get back to me after I submitted the R&R. Then he wanted a second R&R, which I delivered (although I was fuming over the delay and the request for a second R&R). I got the paper published when the editor’s term was over and a new, more sensible editor took the helm. The new editor understood that the former editor was playing games.
The Frontiers journals have a policy that reviewers try to be helpful and not hurtful and try to help the paper along if that help will lead to a satisfactory paper. Reviewers, if they want, can have their names published along with the paper they reviewed. I have reviewed for Frontiers and received some bad papers that I rejected–but in a nice way– because I didn’t want to embarrass the authors or the particular Frontiers journal if the paper would somehow get published. Frontiers journals can have a larger capacity than traditional journals (which are a print and online combination) because Frontiers journals are totally online and not limited in their capacity. Of course, that great capacity increases the chances that many weak studies will be published. If a journal contains too many weakly organized studies, readers will abandon it and miss the good studies.
–Irvin