confirmation bias

[screwtape]

I found a cool site that gives confirmation bias tests.  In case anyone is discussing it with someone who has never heard of it, this is a good tool to use.

http://hosted.xamai.ca/confbias/index.php

[Nick]

I went thru confirmation when I was 14 or so.  But I don't remember being bias.

[jaimehlers]

I'm not very good at "guess the rule" types of things.  I always get snarled up and can't think of anything to establish a rule.

I suppose that means I should keep doing it even though I'm not good at it, huh?

[Graybeard]

The answers seem a bit arbitrary: one answer was "They are odd" - but they are also prime; another was trivial - "the numbers are decreasing"

It seemed like "What is the next number - 7,3,4, 8, ... (The answer is 0 - it's my telephone number backwards... : )

[Noman Peopled]

I'm not very good at "guess the rule" types of things.  I always get snarled up and can't think of anything to establish a rule.

There are too many patterns compatible with simple triplets of numbers to choose the correct one based on that information (especially if they're handpicked to fit multiple obvious patterns).
With the sequence 2,4,6, the answer given was that the numbers are increasing. Well, yes; if you start with that simple rule, 2,4,6 is one possible pattern. There is no way to pinpoint that rule because it does not uniquely describe the sequence, however.
Answers like "even numbers", "the third number is the sum of the other two", "decimal single-digit numbers", "numbers that end in "0" in binary", etc are just as applicable. There is no way of knowing the "rule" if you don't know (all) possible solutions - and even then you'd be most likely to come up with a combination of rules.
The difficulty then, of course, is that if you know all the answers and are finally able to postulate (a) rule(s) that uniquely fit(s) the solutions, you will not be able to find a triplet that fits a similar pattern that is not already available to you in full.

This is precisely why the site can automatically tell you you were "wrong"; it has more data to work with than you do and can just pick a rule that fits the given sequence, but not the one you guessed.
The point is of course to show that there's a number of patterns that can be easily superimposed on stuff with no greater validity than a multitude of others, and to make you question why in the hell you chose a particular one.

Ramblings such as this one are the reason why the poor woman who tested my IQ got so frustrated ...

[Tonus]

http://youarenotsosmart.com/ has some interesting articles on psychology, including how confirmation bias works.  One thing to remember is that if you know you're going to be tested for "confirmation bias" your mind will automatically make certain assumptions which will bias the results.

[Anfauglir]

SPOILERS!  DO THE TEST FIRST!!!

The point of this test is not whether you pick the same answer as the computer does, but which "certainty" button you press.  The point is to show that just because you see a pattern in the numbers, does NOT necessarily mean that you have identified the correct pattern (in this case, correct being the answer the computer chose).

The confirmation bias comes in with the certainty button.  I was shown "5, 7, 9" as my set.  I entered "2, 4, 6", because I saw the pattern "increases by 2 each time".  I was "wrong" - the computer wanted the rule "odd numbers".

The point of the test, I believe, is how much you argue with the computer about the pattern you chose.  If you insist that your answer is correct (for whatever reason) you have a high confirmation bias.  If you are more likely to say "ah - I thought I'd seen the pattern but the PC's one is just as valid" then you have low confirmation bias.

[screwtape]

wild.

My understanding was this:
the computer would randomly select a rule from a collection of rules.  It would present a triplet of numbers that would fit the rule, but would also be sufficiently inconclusive so as to fit other rules.  You, the player, would then try to detect a pattern and then, by testing, deduce the rule.  By only selecting triplets that confirm you rule, you are likely to deduce wrongly.  By selecting triplets that would falsify your rule, you are more likely to find the correct rule.

So, if the rule is "three different odd numbers" and the triplet presented is 3,5,7, then yes, they are indeed prime numbers.  And if your test triplet includes 2, you will find out your hypothesis is wrong.  But all other prime numbers are odd.  So testing triplets that would confirm your hypothesis will not yield the right answer.  But by entering triplets that would fail your hypothesis, you will discover your error sooner. And that is the point, I think.

[jaimehlers]

Yep, but what do you do if you want to try to falsify the hypothesis but can't think of a way to?

I think this can show just how insidious confirmation bias can be, when you can get nailed by it even when you're trying to avoid it.

[screwtape]

Yep, but what do you do if you want to try to falsify the hypothesis but can't think of a way to?

It may mean you need to rephrase your hypothesis.  In the case of the number game, I have a hard time imagining a rule you cannot falsify.  Just put in numbers that do not meet the rule.  If the hypothesis is "increasing odd numbers", then try an even number amid the odds, three evens, try repeating a number, try descending numbers, etc.   

there is a good chapter on that in Yudkowsky's Harry Potter fanfic:
http://hpmor.com/chapter/8