Five ways to take confirmation bias out of your experimental results

Scientists are plagued by confirmation bias (interpreting information in a way that confirms one's preexisting beliefs or hypotheses). The most basic requirement for scientific research is the unbiased interpretation of experimental results. Unfortunately, scientists are human, and as humans, susceptible to fooling themselves when looking at their investigational outcomes. American politics is rife with obvious examples of confirmation bias. Scientists are also particularly susceptible to confirmation bias because we create novel hypotheses and then experimentally test whether those ideas are correct.

Maintaining objectivity as time investment grows

Confirmation bias grows stronger as we invest more time and energy in our research, often making us the least objective person to interpret the results.  Scientists understand the importance of having their science reviewed by experts who did not participate in the research, but peer review usually comes after they have decided their work is worthy of publication or funding. Peer review comments come at a time when confirmation bias is most likely to be applied and underlies the disturbing amount of research misconduct that occurs in responding to peer review.  Everyone involved in the research enterprise is susceptible to confirmation bias (faculty, postdoctoral fellows, students, staff scientists and technologists).

Five tips to prevent confirmation bias

To avoid being skewered by confirmation bias, it is necessary to structure your ongoing research practices to prevent bias from creeping into the analysis of results starting at the earliest stages of a research project. Jiangwei and I came up with five tips:

1. Structure an open and transparent research atmosphere where data and experimental design are examined and evaluated by everyone, especially those not working directly on the project.
2. Encourage and carefully consider critical views on the working hypothesis.
3. Ensure that all stakeholders examine the primary data. Do not rely on analysis and summary from a single individual.
4. Design experiments to actually test the hypothesis. The potential outcomes of an experiment should include the possibility to both prove and disprove the working hypothesis.
5. Before executing the experiment, set the standard for what results support the hypothesis, what results disprove the hypothesis, and what results fail to provide useful information. This is an excellent safeguard against bias sneaking into the interpretation of results.

For more on the challenges in experimental science, read our review of Richard Harris' Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions.