Humans are remarkably good at self-deception, and growing concern about reproducibility is driving many researchers to seek ways to fight their own instincts.  Scientists are susceptible to the same cognitive fallacies as everyone else – so how can they mitigate the risk?

The journal Nature  sheds light on four common fallacies that are found in scientific research, and several techniques to “de-bias” results.  Some of these go by different names, but they are basically sharing the same set of ideas:

Cognitive Fallacies

  1. Hypothesis Myopia:  Collecting evidence to support a hypothesis, not looking for evidence against it, and ignoring other explanations.
  2. Texas Sharpshooter:  Seizing on random patterns in the data and mistaking them for interesting findings.
  3. Asymmetric Attention:  Rigorously checking unexpected results, but giving expected results a free pass.
  4. Just-So Storytelling:  Finding stories after the fact to rationalize whatever the results turn out to be.

Debiasing Techniques

Scientists can avoid these fallacies by applying the following “debiasing” techniques:

  1. Devil’s Advocacy:  Explicitly consider alternative hypotheses – then test them out head-to-head.
  2. Pre-Commitment:  Publicly declare a data collection and analysis plan before starting the study.
  3. Team of Rivals:  Invite your academic adversaries to collaborate with you on a study.
  4. Blind Data Analysis:  Analyse data that look real but are not exactly what you collected – and then lift the blind.


Cognitive Fallacies That Fool Scientists

Source:  Nature