I just finished reviewing a paper for a top journal, and it occurs to me that too many organizational researchers are applying a formula for writing papers rather than the methods of good science. Instead of thinking through theory and using it to derive hypotheses, and then designing the best studies to test those hypotheses, academic researchers are writing formulaic research papers. They look at the literature and analyze the approaches, argumentation and positioning of papers that made it through a draconian peer-review system. Given that careers depend on publishing in good outlets, and for those at Research 1 business schools, in a narrow list of journals like the FT50, you can’t blame people for adopting this approach. Unfortunately, the science suffers and too many formulaic papers are marvelous pieces of writing, with limited contribution to our understanding of people at work.
Writing Formulaic Research Papers
If you look at recent papers in my field, there are a number of elements that are part of the formula. Keep in mind that publishing is as much a marketing as a scientific enterprise. You have to sell your paper to the peer reviewers. You have to convince them that what you did is important and that you are making a significant contribution to the field. Appearance can be more important than substance. The formula looks something like this:
- Lead with exaggerated and unsupported claims. In the first paragraph, state several assumptions and claims that if true would have impact on people and organizations. Finding such claims that others have made is even better, even if the basis of that claim is suspect. An appeal to authority, e.g., a paper in a top journal, is more important than actual data.
- Use citations to support claims. All you need to do is stick cites, (names; years), at the end of a claim. It doesn’t matter if the cited work provided no evidence but the authors just speculated about what you claim.
- Draw on theory. Mention one or more theories and claim that your paper “draws upon” or was “informed by” a theory. You don’t need to explain what the theory is, or how it influenced the design of your study. You don’t even need to explain how it leads to the hypotheses–you only need to say that the theory informed them.
- Include lots of methodological details with citations. It helps to check the methodological literature so you can have a laundry list of cites to justify every procedural detail. If you conduct a t-test, it is best to cite something that says it is a good idea. It isn’t important that what you do is actually what the cited paper recommends. Mike Brannick and I once wrote a paper arguing that you should not use demographic variables as control variables. I can’t tell you how many times I have seen people use demographic variables and then cite us as if we recommended the practice. At best we are a source that mentioned the issue, although in the direction opposite to what is implied by citing us.
- Use complex statistics. Complex statistical modeling has become almost a requirement for publication in top journals in the organizational sciences. Never mind that luminaries in the field of statistical methodology have noted that one should use the simplest methods necessary to address the problem at hand. Our field has adopted the opposite approach. Use a complex structural model to test a causal process, and then all you need is a disclaimer at the end of the paper that your cross-sectional data can’t be used to draw a causal conclusion. Then why test the model if you can’t draw a conclusion about it?
A Better Path Forward
Our field needs to do better by putting the science first and the positioning second. Work the problem and not the targeted journal outlet. If research addresses an important and relevant problem and uses rigorous methods, there will be no need to use a formulaic approach. It bears repeating–science first and positioning second. This means taking a systematic and pre-planned approach. First answer the following questions, and do so thoroughly. Don’t rush things at this early stage. Take care of the science, and the journal success will follow.
- What is the problem? Too little attention is taken in identifying a specific problem that serves as the foundation of the research. Vague or overly general problems can lead to uninterruptible results and questionable contributions. If the issue is stress, it is better to define the problem precisely “does weekly work hours relate to burnout?” than generally “does workload relate to burnout?” The latter raises concern about what is meant by workload as there are so many ways to operationalize it. It is best to confront those conceptual and definitional issues up front.
- What is known about the problem? A reasonably comprehensive and balanced literature review should be conducted. The current state of knowledge includes studies that both support and refute the researcher’s theory and personal position. Mixed results should lead the researcher to explore the reasons and that can feed back to the purpose.
- Is there a theory that explains the phenomenon? This means defining precisely what the theory says, and then explaining how the theory leads to the hypothesis. In science, we do not need theory-generated hypotheses in every research report, and in fact the biggest advancements come from exploratory research.
- What is the most rigorous study method that answers my question? If a good foundation has been provided, it is almost a mechanical process to sift through potential research designs to choose the one best suited. If the question is whether two variables are related, a cross-sectional design is appropriate, especially at early stages. If the question concerns drivers of a phenomenon, the best choice is a randomized experiment. If not feasible the next best is a quasi-experiment. Unfortunately, experimental approaches outside of the laboratory are rare in the organizational sciences. When I went looking for experimental field study examples for my business methods class, I had to use examples from non-business fields.
- What are the simplest statistics that shed light on the question? This should be a no-brainer. If your question is whether two variables are related, if they are both continuous, a correlation is the statistic of choice. For an experiment, a simple t-test can compare an experimental with control group.
- What conclusions can I reasonable reach and what new questions does the research raise? This is where researchers can first summarize their findings and provide an answer to the question, and then go beyond the study. Often little time and attention is given to the Discussion section of papers, but it is here where the major theoretical contributions should typically occur. This is where new ideas are born to explain phenomena. Old theories can be critiqued as new theories emerge. Here is the place to wonder why results came out as they did and to provide alternative explanations. Here is where the next steps in the research program are laid out, and where new questions are generated.
A few weeks ago I wrote about how AI is flipping where we spend our time on projects. As AI takes on more of the routine work, people are spending more time specifying the problem and validating the results. We can only hope that this will occur with research in the organizational realm with a greater focus on defining the problem and interpreting results. If so writing formulaic research papers should be a thing of the past.
Image generated by ChatGPT
SUBSCRIBE TO PAUL’S BLOG: Enter your e-mail and click SUBSCRIBE