Nothing in the fields of industrial organizational psychology and organizational behavior has been more neglected than the nature of the underlying populations that we study. There is an unspoken assumption that employees are employees and so we treat all samples as if they come from the same population. I recently did a quick check of a top journal, and found that the most used source of data was from online panels followed by students. Far less common were studies from a particular occupation or organization. Even more rare (none in the issue I reviewed) is treating population as a variable to be studied as population is neglected in organizational research. When I think about where population matters, occupation comes first to mind, and here we have some evidence that it matters.
Does Occupation Really Matter?
The short answer is yes. Unfortunately, little research has addressed the issue, but there is some. For example, Kate Sparks and Cary Cooper compared 13 occupations on health and stress, finding differences among them. In the development of my Job Satisfaction Survey 2, I purposely collected data from a variety of occupations, and there were considerable differences in means. For example, engineers were high in pay satisfaction while state government workers were quite low. On the other hand, state government workers were high on communication satisfaction, while licensed practical nurses were low. Although these two studies are compelling, we know relatively little about differences among occupations, and certainly not enough to draw conclusions or start to build theories.
Population Is Neglected in Organizational Research
The basic purpose of meta-analysis is to quantitatively combine results across studies because the results of multiple studies is more conclusive than a single study. 0ne of the original uses was to determine effect sizes for a particular phenomenon, such as the correlation between job satisfaction and job performance. An underlying assumption, however, is that the samples come from the same population. This is highly unlikely as in almost all cases the variation among samples is more than we would expect if they all came from the same population. Clearly there are reasons relationships vary across samples, but those reasons are not typically the focus of research.
With this in mind my collaborators and I set out to do a meta-analysis to study differences among occupations. The idea would be to collect samples of people who held different jobs: accountant, nurse, police officer, and so on. What we discovered quite quickly is that with the exception of nurses, there aren’t enough studies that have samples of the same occupation, so we abandoned the project.
Closing the Gap
Clearly we need more primary studies of occupation and other population differences such as industry. These would likely start off as exploratory studies, collecting data from samples of occupations on the same phenomenon to check for differences to answer the following questions.
- Are the means the same? Just as I did with the JSS-2, we can see where differences occur and whether we can identify clusters of occupations that are similar versus different. From there we can investigate what is common and different among those occupations.
- Are correlations the same? Do occupations vary in the magnitude and direction of correlations. Are potential antecedents and consequences the same? Again clustering based on correlation might provide insights into underlying reasons.
- Are factor structures the same? This is the measurement equivalence/invariance ME/I issue. Does a particular multi-item scale behave the same across occupations. Perhaps how accountants interpret items is different from electricians.
- Can we develop explanations for differences? Once we start accumulation data, we can begin to work on theoretical explanations. Such explanations might go beyond occupation to provide a deeper understanding of people’s experience of work.
Of course, studying occupation is not necessarily easy. There is a reason that online panels are so popular today, despite all the hand wringing about bots impersonating humans and uncertain generalizability. But in the long term, the will be a payoff in knowledge and understanding if at some point we can no longer say that population is neglected in organizational research.
Image generated by Boodle.Bot from BoodleBox
SUBSCRIBE TO PAUL’S BLOG: Enter your e-mail and click SUBSCRIBE