From: veritasium
This article explores how cognitive biases, particularly confirmation bias, can impede effective hypothesis testing, drawing insights from a number sequence puzzle and its relation to the scientific method.
The Number Sequence Puzzle
Participants were presented with an initial sequence of three numbers, 2, 4, 8, and asked to deduce a hidden rule governing sequences of numbers [00:00:01]. They could propose their own three-number sequences, to which the presenter would respond “yes” if it followed the rule or “no” if it did not [00:00:09].
Initially, participants often proposed sequences that continued the pattern of multiplying by two, such as 16, 32, 64 [00:00:32], 3, 6, 12 [00:00:55], or 10, 20, 40 [00:01:00]. All of these sequences were confirmed to follow the rule [00:00:44]. Despite repeatedly testing this “multiply by 2” hypothesis, it was revealed not to be the actual rule [00:00:50].
The Challenge of Confirmation
The common approach taken by participants was to propose numbers that they expected to fit their presumed rule [00:03:39]. For instance, after hearing that 2, 4, 8 fit, participants would try 5, 10, 20 [00:01:10] or 100, 200, 400 [00:01:13]. These all followed the rule, but did not help in identifying the actual rule [00:02:38]. This strategy, while seemingly logical, is a manifestation of confirmation bias [00:03:50].
A turning point occurred when a participant proposed 2, 4, 7 [00:01:47], which also fit the rule [00:01:51]. This indicated that the rule was not “multiply by 2” and was much broader.
The true rule was simply “numbers in increasing order” or “numbers in ascending order” [00:02:53]. Any three numbers, such as 1, 2, 3 [00:02:24], 7, 8, 9 [00:02:27], or even 1, 7, 13 [00:02:41], would follow the rule as long as they were in increasing sequence [00:03:10]. Conversely, 10, 9, 8 did not follow the rule [00:02:51], providing crucial disconfirming evidence.
The Importance of Disconfirmation
The puzzle highlights a critical aspect of hypothesis testing:
“You’re always asking something where you expect the answer to be yes… But, instead you want to get the no’s. You want to get the no because that’s much more informational for you then yes” [00:03:53].
This concept is analogous to the metaphor of the black swan, inspired by Nassim Taleb’s book The Black Swan [00:03:14]. The theory that “all swans are white” was reinforced by every observation of a white swan [00:03:22]. However, the discovery of black swans in Australia disproved the theory, demonstrating that no number of confirmations can definitively prove a theory true, but a single disconfirmation can prove it false [00:03:29].
The Scientific Method and Falsification
The experience of the puzzle participants mirrors a fundamental principle of the scientific method:
“We set out to disprove our theories and it’s when can’t disprove them that we say this must be getting at something really true about our reality” [00:04:06].
To truly understand truth and avoid self-deception, one should actively attempt to disprove their beliefs [00:04:21]. Only by rigorously testing a hypothesis by seeking disconfirming evidence can its robustness be genuinely assessed.