Five ways to take confirmation bias out of your experimental results

Photo of Charles Rock and Jiangwei Yao.

Confirmation bias is a pitfall for many investigators. Charles Rock and Jiangwei Yao discuss five ways to take bias out of investigational results.

“The first principle is that you must not fool yourself – and you are the easiest person to fool.”

Richard Feynman (Nobel Prize in Physics, 1965)

Scientists are plagued by confirmation bias (interpreting information in a way that confirms one's preexisting beliefs or hypotheses). The most basic requirement for scientific research is the unbiased interpretation of experimental results. Unfortunately, scientists are human, and as humans, susceptible to fooling themselves when looking at their investigational outcomes. American politics is rife with obvious examples of confirmation bias. Scientists are also particularly susceptible to confirmation bias because we create novel hypotheses and then experimentally test whether those ideas are correct.

Maintaining objectivity as time investment grows

Confirmation bias grows stronger as we invest more time and energy in our research, often making us the least objective person to interpret the results.  Scientists understand the importance of having their science reviewed by experts who did not participate in the research, but peer review usually comes after they have decided their work is worthy of publication or funding. Peer review comments come at a time when confirmation bias is most likely to be applied and underlies the disturbing amount of research misconduct that occurs in responding to peer review.  Everyone involved in the research enterprise is susceptible to confirmation bias (faculty, postdoctoral fellows, students, staff scientists and technologists).

Five tips to prevent confirmation bias

To avoid being skewered by confirmation bias, it is necessary to structure your ongoing research practices to prevent bias from creeping into the analysis of results starting at the earliest stages of a research project. Jiangwei and I came up with five tips:

  1. Structure an open and transparent research atmosphere where data and experimental design are examined and evaluated by everyone, especially those not working directly on the project.
  2. Encourage and carefully consider critical views on the working hypothesis.
  3. Ensure that all stakeholders examine the primary data. Do not rely on analysis and summary from a single individual.
  4. Design experiments to actually test the hypothesis. The potential outcomes of an experiment should include the possibility to both prove and disprove the working hypothesis.
  5. Before executing the experiment, set the standard for what results support the hypothesis, what results disprove the hypothesis, and what results fail to provide useful information. This is an excellent safeguard against bias sneaking into the interpretation of results.

For more on the challenges in experimental science, read our review of Richard Harris' Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions.

A researcher’s look at Rigor Mortis: Are motivators and incentives to find a cure hurting scientific research?
A researcher’s look at Rigor Mortis: Are motivators and incentives to find a cure hurting scientific research?
Research

A researcher’s look at Rigor Mortis: Are motivators and incentives to find a cure hurting scientific research?

Charles Rock, PhD

St. Jude researchers take a look at Rigor Mortis, Richard Harris’ exposé of how the drive to find results hampers scientific progress.

About the author

Charles Rock, PhD, was a member of the Department of Infectious Diseases and later the Department of Host-Microbe Interactions at St. Jude Children’s Research Hospital until his passing in 2023. Learn about Dr. Rock's research career.

More Articles From Charles Rock

Related Posts

two men at a whiteboard
two men at a whiteboard

Shared Resources Spotlight: Cytogenetics

Your fatty acid delivery is on the way: studying cellular logistics provides disease insights

dali-like illustration of clocks in a wasteland
dali-like illustration of clocks in a wasteland

The ticking ‘epigenetic’ clock measures true biological age

Stay ahead of the curve