When we collect data on people, those people are members of groups. Employees, for example, are members of teams and members of organizations. People more broadly are members of families and members of communities. Because of this dual-level nature of the data on people, care must be taken in drawing appropriate inferences. When someone extends conclusions from one level to another, they have committed the ecological fallacy. But what is the ecological fallacy definition?
What Is the Ecological Fallacy Definition?
The ecological fallacy is simply drawing conclusions about one level from another level. For example, suppose you collect data on 250 employees, each of whom is a member of a 5-person team. You can analyze data on all 250 and draw conclusions about those individuals. You might collect data from a survey study in which you ask about job satisfaction and how much effort they exert at work. You can compute a correlation coefficient for the 250 individuals. Your results might show that individuals who are satisfied exert more effort. You also might take the average for each of the 50 teams for both variables, and correlate those averages. This might show that teams with higher average satisfaction have members with higher average effort. What you cannot do without committing the ecological fallacy is conclude that because there is a correlation at the individual level there is one at the team level and vice versa. Sometimes the same relationships exist and sometimes they do not. The only way to know is to do both analyses.
How Can Individuals and Teams Differ?
There are times when data line up so that results are different when you look at individuals versus teams (or other groupings). With the satisfaction-effort example, it is possible that the satisfaction and effort spread within each group is the same. Each group has a similar mix of low, medium, and high satisfaction and effort. It might be that the low satisfaction goes with low effort and high satisfaction with high effort for individuals, but the means for each team are the same. Each has say, one low, three average, and one high member. Teams all have similar means on both variables, and thus there is little correlation at that level.
It is also possible that at the individual level there is no correlation between satisfaction and effort. Some people are high on both, some low on both, and some high on one and low on the other. The mixed pattern where low satisfaction is associated with high effort for some people and low effort for others cancels out the correlation at the person level. It could wind up, though, that for some reason some teams have all members who are high-high, some all members who are low-low, and some a mix. The result is that the means for some teams are high for both variables, the means for some teams are low on both variables, where others teams are in the middle on both variables. This produced a correlation pattern at the team level, even though there is none at the individual level.
Where We See the Ecological Fallacy?
We see the ecological fallacy in research reports and in the media. Some examples include
- Research Reports Extending Results Across Levels: Sometimes researchers are not careful in the conclusions they draw from studies. They might study individuals and extend conclusions to organizations. Merely finding that satisfaction is correlated with effort does not tell us that if we design an intervention that increases satisfaction, effort will improve as well. It might if job satisfaction is the cause of effort, but what if it is not? What if the reverse is true. People who are motivated and exert effort perform better, are rewarded for that performance, and wind up more satisfied. Trying to increase effort by raising satisfaction is not going to be helpful.
- Meta-Analysis Moderators: Meta-analysis is the quantitative summary of results across studies. For example, we might find 80 studies reporting correlations between job satisfaction and effort, each of which was at the employee level. In most cases, the individual correlations are very different–far more varied than you would expect by chance alone. Researchers want to understand why the studies differ, so they look for moderator variables that reflect differences among the studies in the characteristics of the people studied. A common moderator is gender–researchers wonder if relationships are different for men and women. As they code the individual studies to extract the needed information, they might note the percentages of men in the sample for each study–one might be 80% men, whereas another is only 40%. They might find that percentages of men accounts for differences between the correlations of interest. The satisfaction-effort correlation is larger for samples that are mainly men than for samples that are mainly women. The researchers commit the ecological fallacy if from that finding, they conclude that men have a stronger correlation than women. They can only conclude that samples with more men have a larger correlation than samples with fewer men. It is possible that at the individual employee level there is no difference between men and women even though at the sample level there is. The reason for the sample level difference might be due to occupational differences–samples with mostly men (e.g., trades) are from different occupations than those with mostly women (e.g., nurses).
- Meta-Analysis Mixing Levels: Sometimes people doing meta-analysis will mix levels in their analysis. They might include studies that analyzed at the individual level with studies that analyzed at the group level. It is fine to include both types of studies, but they need to be analyzed separately to show if results are similar or different at the two levels.
- Assuming All Members of a Demographic Group Are the Same: Politicians and much of the media like to classify people into groups based on a variety of characteristics, such as ethnicity, gender, generational cohort, and political party. They then draw inferences about individuals based on average characteristics for the group. They might, for example, talk about the Black vote or female issues as if all Blacks and all women are uniform in their political concerns and leanings. They might assume they know about someone’s political positions based on their party. There is usually more differences among people within groups than there are between the groups.
- Salary and Pay Satisfaction. We know that at the individual level people’s pay satisfaction relates to their salary. The fallacy occurs when people assume that therefore higher paying jobs have higher satisfaction than lower paying jobs. People’s satisfaction has more to do with how they are paid relative to others in their occupation than how they are paid relative to other occupations. Thus, physicians in general aren’t necessarily more satisfied with pay than nurses in general even though they are paid a lot more.
The ecological fallacy is easy to make because so often it seems logical that results from one level should apply to the other. In many cases results match, but often they do not. When results differ, additional research is needed to figure out why. People are members of many kinds of groups, each of which can be the focus of study to distinguish what happens at the level of people versus the groups they belong to.
Note: Image was generated by DALL-E 4. ChatGPT 4 was used to help generate the pay example.
SUBSCRIBE TO PAUL’S BLOG: Enter your e-mail and click SUBSCRIBE