Scientific Skepticism Is a Smart Thing

As medical science became politicized during COVID-19, a new catch phrase of “follow the science” emerged in the media. People used it to justify their particular point of view about what steps we should all take to combat the pandemic, referring to issues ranging from lock downs to vaccines. What got lost in these policy discussions was the nature of science and the difference between scientific facts or observations, and the most reasonable inferences we can make. It is easy to point to some research finding and claim it supports your position, but there is a difference between science fact and science fiction, that is, conclusions based on data that are wrong. Scientific skepticism is a smart thing because the inferences that scientists make are best guesses about what their findings mean.

Science Facts

If you break it down, science is merely the application of systematic methods to gather information about a phenomenon of interest. We might conduct a survey to predict the outcome of election, or we might conduct a randomized drug trial to investigate the effectiveness of a new medication. We design our studies to collect observations using a predetermined measurement procedure. Thus we might ask a random sample of registered voters which candidate they plan to vote for, or we assess symptoms in patients who receive our new medication. The raw observations might be considered scientific facts. Assuming there were no errors in carrying out the study and the researchers were honest in what they did, we can be reasonably confident that what was found was real. What it means and what we can conclude is another matter.

As a stress researcher, I have conducted many survey studies where employees were asked to use rating scales to indicate how stressful their jobs are and how they feel about work. For example, I might ask them about their level of workload and about burnout. Each person who completes the survey will provide a rating of each. Those ratings–the numbers provided–are the facts of my study. I can compute a correlation coefficient between workload and burnout to show whether or not the two are related. Do people who score high on workload score high on burnout? The correlation is verifiable. I can give my data to 10 researchers and ask them to compute a correlation, and show that everyone gets the same number. But these verifiable facts are given an interpretation and meaning. We assume that ratings of workload reflect the objective reality of the job. We draw an inductive inference and claim they can be generalized to other employees. When we go beyond the raw facts of the study, things can get fuzzy.

Scientific Inference

Science is not just about raw observations. In order to give those observation meaning, we have to interpret them and draw inferences. When we measure things, we make inferences about what the numbers represent. When we find relationships among variables, we make inferences about why those relationships exist. If people give us ratings, we assume that they are able and willing to accurately report on their experiences and feelings. We assume that if someone reports having a high workload, they really are busy at work with lots of things to do that take a lot of effort. But what decades of stress research has shown me is that given the same job in the same company, people vary in how they rate it. Their rating says as much about them as about the job. We have to be careful in drawing inferences that if workload correlates significantly with burnout, that it is the job and not the person that is at play.

Even if we can assume that our measures reflect exactly what we are after in a survey study, we cannot really draw inferences beyond the fact that the two variables are correlated. It is possible that having a heavy workload leads to burnout, but it is also possible that people who feel burned out are overwhelmed and view the job as having a heavy workload even if it does not. In other words we cannot know if workload leads to burnout or the reverse. Even more concerning is the possibility that there is some other factor that leads to both of them. Maybe people who are unhappy with their bosses just rate workload and burnout as being high regardless of the reality.

Scientific Skepticism Is a Smart Thing

Scientific conclusions are always tentative and based on preponderance of evidence. Like a court case, scientists weigh the facts and come up with their best conclusions. Given the same facts, scientists will often disagree and engage in vigorous debate about what findings mean. Sometimes the conclusions reached are correct, but often they are wrong and produce what might be called a science fiction. It is not irrational to question conclusions of scientists, and scientists themselves are often the biggest science skeptics.

Scientific skepticism is a smart thing when it involves critical thinking. It is about taking a hard look at what someone claims rather than taking it on faith. Were the methods used sound? Often even peer-reviewed articles report on weak studies using questionable methods. Are there alternative explanations for results? Did the inferences make logical sense based on the evidence, or is there even evidence? We shouldn’t accept someone’s conclusion just because they claim it is based on science. We should ask them to explain the evidence before we accept what they say is true, rather than blindly following the science.

Image generated by DALL-E 4.0

SUBSCRIBE TO PAUL’S BLOG: Enter your e-mail and click SUBSCRIBE