Confirmation bias: what Wason's 2-4-6 test really showed, and when a positive test is not the error

Confirmation bias is the tendency to search for, weigh and use information so it confirms a hypothesis you already hold. The APA Dictionary of Psychology describes it as the tendency to gather evidence that confirms preexisting expectations. It typically pursues supporting evidence while it dismisses contradictory evidence or fails to seek it.

• Confirmation bias shapes search, focus and weighting of information, not stubbornness as a character trait.
• Peter Wason showed it in 1960 with the triple 2-4-6: only 6 of 29 students found the rule without error.
• Lord, Ross and Lepper showed in 1979 with 48 students: both camps on the death penalty rated the supporting study as better and polarized further.
• A positive test strategy is not always an error: it becomes confirmation bias only when it tests a subset of true cases.
• Whether confirming is remembered better is not securely shown.
• Three questions help in study: seek a counter-example, search neutral, weigh facts apart from opinions.

What confirmation bias is

The definition is narrow. It concerns the test of a hypothesis. You hold an assumption. Then you select information so it fits. This often happens without intent. It starts at search, not only at reading after. Britannica describes the process: confirming is sought out, interpreted and recalled with preference, while disconfirming is ignored or given little weight, as an efficient but often flawed way to process the wealth of information.

Three processes work together. First comes selective search: you enter directed terms and find almost only support. Second comes focus: you read mainly the hits that fit and skip the rest. Third comes weighting: you rate the fitting study as stronger and the unfitting one as weaker. A review of the bias summarizes it as the tendency to notice, focus on and give greater credence to evidence that fits existing beliefs.

The error is thus different from stubbornness. Stubbornness keeps an opinion even when counter-proof lies clear on the table. Confirmation bias sorts before, what ever reaches the table. It is also different from the ten cognitive biases in brief. That overview names it in one line beside nine others. This article treats it alone, with test and procedure.

The 2-4-6 test of 1960: what Wason really showed

Peter Wason worked in London in 1960 with 29 psychology students. He named the triple 2-4-6. He said: it follows a simple rule about three numbers. Find the rule. Write your own triples. After each triple he said whether it fits the rule or not. The true rule was: three numbers in ascending order. That is all. No even numbers needed. No fixed gap needed.

Most thought too narrow. They guessed even numbers with gap 2. They tested triples such as 4-6-8 or 10-12-14. Each answer was yes. Each yes felt like proof. Yet it was only a subset of true cases. Who tests only ascending with gap 2 never sees whether 3-5-9 or 1-7-30 also fits. That was Wason's point.

The numbers from the original appear in a summary of the experiment. Six of 29 named the correct rule without first announcing a wrong one. A later review puts it the same way: 23 of the 29 missed it on first attempt, because they offered only examples that fit the hypothesis. Wason's point was sharp: they never tested the hypothesis. They only confirmed it.

For exams one sentence helps. A confirming instance never proves a rule. A disconfirming instance breaks it. That is the logic of falsification behind Wason. It explains why 4-6-8 tests weakly and 3-5-9 tests strongly. Plan a no-case first, before you collect more yes-cases.

The death-penalty study of 1979: why same facts divide

Charles Lord, Lee Ross and Mark Lepper worked in 1979 at Stanford. They sought students with a clear view on the death penalty. They selected 48 people, supporters and opponents of deterrence. Each person read two invented studies. One supported their own side. One opposed it. Then each person rated both studies. At the end they measured the view again. The details appear in the original paper.

Both camps judged with the same skew. Each side rated the supporting study as cleaner and more convincing. Each side found more flaws in the opposing study. Both sides were not milder after, but more polarized. Same evidence, more distance. That is the core: the error lies not only in search. It lies also in rating after.

Take two points exactly. First, the studies were invented. The people learned this after in a debriefing. That belongs to the ethics of the trial and limits transfer to real debates. Second, these were students with a strong prior. The effect shows clearest where the view is firm and the topic matters. It does not say facts never work. It says mixed findings can divide firm camps.

Three examples from study and reading

First example: directed search in study. You think a vegan diet harms health. You search for unhealthy vegan instead of effect of vegan diet. You find almost only warnings. You read almost only warnings. Two studies lie before you. One warns when poorly planned. One shows good values when well planned. You rate the first as important and the second as exception. That is the triple search, focus, weighting in small. The fix is not more reading. The fix is different search.

Second example: news and filter bubble. You follow almost only accounts that share your view. The algorithm shows you more of it. You feel well informed, because you read much. Yet you read much of the same. Studies of debates show the pattern: same debate, opposite winners in the eyes of both camps. An overview of the bias summarizes the 1984 debate poll finding in the same sense. It repeats today in feeds. Only your own counter-channel helps: read once a week with care a source you else avoid.

Third example: early hypothesis in tests. You fix early on one reading. Then you steer which tests you choose and how you read results. Fitting counts as proof. Unfitting counts as noise. Britannica notes the medical form: an early hunch can steer the next steps of assessment. The point also holds for papers: who fixes the reading too early reads sources only as proof. Put the counter-hypothesis on page one of your notes, not in a footnote.

Positive test strategy: when search is not the error

Stay exact here. Often it makes sense to test the near case first. Joshua Klayman and Young-Won Ha named it in 1987 positive test strategy, as a history of the debate summarizes. You test cases that should hold under your hypothesis. That limits the endless set of possible cases to a testable set. That is heuristic and often rational. It can also break the hypothesis.

It becomes the error only in one case. When your test cases form only a subset of true cases, each yes also confirms the false narrow rule. Then Wason repeats: 4-6-8 always fits, but proves nothing about ascending. Memory rule for exams: positive testing is allowed. Only positive testing is the error. Plan always at least one case that must say no under your narrow rule.

That also explains why smart people err. They test fast and clean, but too narrow. They collect many yes-cases in short time. They feel sure, because the count of hits is large. Yet the hits all come from the same subset. One case from outside weighs more than ten more yes-cases from inside.

Three questions against the error

Use the questions before you fix your reading. They cost little time. They fit every paper and every talk.

1. 1.Seek a counter-example that would break your rule, and test it first.
2. 2.Search neutral, with terms free of direction, and read one source that opposes your view.
3. 3.Weigh facts and opinions apart, and test the strongest opposing study as fairly as the supporting one.

If you answer no to question 1, plan a no-case now. With Wason that would be 3-5-9 or a falling triple. In literature search it would be the study that hits your thesis hardest. If you answer no to question 2, phrase the search new. Replace unhealthy vegan with effect of vegan diet. Replace proof with test and critique. If you answer no to question 3, summarize the opposing study in your own words before you rate it. Only then follows your judgment.

A fixed place helps learning. Make a list named cognitive biases in the dictionary. Collect there confirmation bias, dissonance and heuristics as own cards. Learn the list by retrieval not by re-reading. How that works is described under how to memorise psychology terms so they stick. The multiple-choice mode there needs at least four words per list, so a list with ten to twelve cards pays off. The four steps of the learning loop and what each study mode does show the path from lookup to retention.

Boundary: what the error is not

Confirmation bias steers choice and reading of information before and during the test. Cognitive dissonance after Festinger describes the discomfort after, when two beliefs clash, and the paths to lower it. The availability heuristic and representativeness heuristic describe judgments by ease of recall or by similarity. All three hang together. None replaces another. In exams always name the mechanism: search and weighting here, discomfort and reduction there, ease of recall beside.

Know one more limit. Whether confirming is remembered better is not securely shown. Reviews find no clear sign for it. Name memory in exams as uncertain. Securely shown are search and weighting. That suffices for the definition. It guards you against lost points from too strong a claim.