Not long ago, a friend shared with me an editor’s decision letter from one of our top IO psychology/management journals. The paper had failed to support some theory-based hypotheses, and was rejected for failing to make a sufficient contribution. So far, so good. But then the editor provided advice, that I would only characterize as unhelpful help from a research integrity perspective. The editor suggested rewriting the introduction by revising the theory and hypotheses to be more in line with the findings. It is bad enough when authors engage in questionable research practices, but shouldn’t we demand that our editors must practice research integrity?
HARKing As a Questionable Research Practice
Hypothesizing after results are known or HARKing is a questionable research practice that undermines the integrity of our research. It is a dishonest practice because it gives readers the idea that the hypotheses put forward were based on prior theory and analysis, not on the results of the current study. The way the scientific process is supposed to work is that you raise a question that can be answered with a study. Many times you use prior research and theory to derive hypotheses that can be put to empirical test. Finding what you expect provides evidence that the theory upon which the hypotheses were based is correct. Failing to find support suggests that the theory is incorrect and that we need to do more work on it. What makes science scientific is that we design studies so that both outcomes–support and lack of support, are possible. HARKing guarantees that you will only find support because the results drove the hypothesis. You cannot fail to find support, so the scientific enterprise has been undermined.
Our Editors Must Practice Research Integrity
Journal editors are the first line of defense against questionable research practices. They should evaluate research on the importance/relevance of the questions addressed and on the methodological rigor. Failure to support hypotheses should not in and of itself be the reason for rejection. Editors also need to be on the lookout for reviewers who suggest that authors change hypotheses or try additional analyses to see if they can squeeze statistical significance out of the data (p-hacking). The editor’s letter should make clear that doing so would be inappropriate, and it would be good for editors to provide feedback to reviewers about it.
Editors should certainly never be the ones suggesting questionable research practices. As I wrote about a few weeks ago HARKing and p-hacking are very widespread, especially in papers at elite journals from top research universities. Researchers in my field certainly don’t need encouragement by editors to engage in even more of it. Our editors must practice research integrity to help combat questionable research practices that undermine our science.
Image generated with Nano-Banana. Prompts “Image of a police officer scientist with a laboratory. Give her a police officer hat. The original picture with a white hat.
SUBSCRIBE TO PAUL’S BLOG: Enter your e-mail and click SUBSCRIBE