Overconfidence effect

From Wikipedia, the free encyclopedia

 This article is written like a personal reflection or essay and may require cleanup.
Please help improve it by rewriting it in an encyclopedic style. (December 2007)

The overconfidence effect is a bias in which people are correct in their judgements far less often than they think they are. For example, for certain types of question, answers that people rate as "99% certain" turn out to be wrong 40% of the time. Overconfidence is one kind of what is called the miscalibration of subjective probabilities.


Contents

[edit] Experimental demonstration

The effect has been demonstrated in a number of different ways (for surveys, see Hoffrage (2004) or Lichtenstein et al. (1982)).

  1. Get subjects to evaluate their confidence in a statement. Group together all the statements with a given level of confidence (e.g. 90%) and compare that to the actual frequency of being correct.
  2. Test students with multiple-choice questions, then elicit their level of confidence in their answer, on a scale from chance to 100% (total certainty). Compare this to the true accuracy of the answers.
  3. Give subjects a question with a numerical answer, and get them to choose a confidence interval such that they have have a particular level of confidence that the true answer is in that range. E.g. "Pick a low number and a high number such that you are 90% confident that the population of Bulgaria is between those numbers."
  4. Offer subjects the opportunity to bet on the correctness of their answers, with odds that are favourable if their judgements of accuracy are correct. They lose money if they are overconfident.[1]

If human confidence had perfect calibration, judgements with 100% confidence would be correct 100% of the time, 90% confidence correct 90% of the time, and so on for the other levels of confidence. By contrast, the key finding is that confidence exceeds accuracy so long as the subject is answering hard questions about an unfamiliar topic. For example:

  1. In a spelling task, subjects were correct about 80% of the time when they were "100% certain".[2] Put another way, the error rate was 20% when subjects expected it to be 0%.
  2. In a confidence-intervals task, where subjects had to judge quantities such as the total egg production of the U.S. or the total number of physicians and surgeons in the Boston Yellow Pages, they expected an error rate of 2% when their real error rate was 46%. Once subjects had been thoroughly warned about the bias, they still showed a high degree of overconfidence.[3]

Different classes of experts have been tested for overconfidence within their area of expertise, with varying results. A test of 25,000 predictions by weather forecasters found no overconfidence, but an experiment with clinical psychologists found a high margin of overconfidence in their conclusions about a specific case (see below).

When the accuracy is close to 100%, confidence is usually somewhat less, in other words underconfidence. This is interpretable as a ceiling effect (i.e. at the top of the scale, errors can only be made in one direction).

[edit] Overconfidence in specific areas

Stuart Oskamp tested groups of clinical psychologists and psychology students on a multiple-choice task in which they drew conclusions from a case study. Along with their answers, subjects gave a confidence rating in the form of a percentage likelihood of being correct. This allowed confidence to be compared against accuracy. As the subjects were given more information about the case study, their confidence increased from 33% to 53%. However their accuracy did not significantly improve, staying under 30%. Hence this experiment demonstrated overconfidence which increased as the subjects had more information to base their judgement on.[4]

[edit] Overconfidence and optimism

Main article: Optimism bias

[edit] Example : "Winner takes all" - tournaments for promotion

Many "real-world" situations can be characterized as a rank order tournament. In other words, prizes are not proportional to outcomes, but accrue to the top performers. In many employment situations, only the best performers are promoted - for example tenure in academia, or promotion to partner in consulting or law firms. In such situations, overconfidence bias may:

  1. Cause employees to overestimate their chances of promotion, this causes them to overly value the possible promotion as an incentive, because they deem it to be more likely to occur.
  2. Cause employees to work harder than they normally would for a chance at a promotion.
  3. Prefer tournaments and other compensation schemes where most of the rewards are concentrated at the top, and where the costs of failure are extreme (for example an "up or out" promotion system). The overconfidence bias causes these employees to consider the chance that they will fail to be very slim and to overestimate the chance that they will succeed.

[edit] Overconfidence and overvaluing the likelihood of being in the top 1% of incomes

Many people tend to overestimate the likelihood that they will be in the top 1% of incomes. They also underestimate the likelihood that their incomes will fall substantially. This can bias individuals in favor of policies and political views that are quite generous to those at the top of the income scale, and against policies that aid the poor.

[edit] Overconfidence and other biases

[edit] Overconfidence and escalating commitment

Overconfidence bias often serves to increase the effects of escalating commitment - causing decision makers to refuse to withdraw from a losing situation, or to continue to throw good money, effort, time and other resources after bad investments.

[edit] Overconfidence and ignoring of base rates

People often tend to ignore base rates (cf. base rate neglect) or undervalue their effect. For example, if you are competing against individuals who are already winners of previous competitions, one's odds of winning should be adjusted downward considerably. People tend to fail to do so sufficiently.

[edit] See also

[edit] Notes

  1. ^ Fischoff, Baruch; Paul Slovic, Sarah Lichtenstein (1977). "Knowing with certainty: The appropriateness of extreme confidence". Journal of Experimental Psychology: Human Perception and Performance 3: 552-564. 
  2. ^ Adams, P. A. & Adams, J. K . (1960). Confidence in the recognition and reproduction of words difficult to spell. American Journal of Psychology, 73 pp. 544-552
  3. ^ Alpert, Marc; Howard Raiffa. "A progress report on the training of probability assessors" in Kahneman, Daniel; Paul Slovic, Amos Tversky (1982). Judgment under uncertainty: Heuristics and biases. Cambridge University Press, 294-305. ISBN 978-0-521-28414-1. 
  4. ^ Oskamp, Stuart (1965). "Overconfidence in case-study judgements". The Journal of Consulting Psychology 2: 261-265. American Psychological Association.  reprinted in Kahneman, Daniel; Paul Slovic, Amos Tversky (1982). Judgment under uncertainty: Heuristics and biases. Cambridge University Press, 287-293. ISBN 978-0-521-28414-1. .

[edit] References

  • Baron, Johnathan (1994). Thinking and Deciding. Cambridge University Press, 219-224. ISBN 0-521-43732-6. 
  • Gilovich, Thomas; Dale Griffin, Daniel Kahneman (Eds.). (2002). Heuristics and biases: The psychology of intuitive judgment. Cambridge, UK: Cambridge University Press. ISBN 0-521-79679-2
  • Hoffrage, Ulrich. "Overconfidence" in Pohl, Rüdiger (2004). Cognitive Illusions: a handbook on fallacies and biases in thinking, judgement and memory. Psychology Press. ISBN 978-1-84169-351-4. 
  • Lichtenstein, Sarah; Baruch Fischoff, Lawrence D. Phillips "Calibration of probabilities: The state of the art to 1980" in Kahneman, Daniel; Paul Slovic, Amos Tversky (1982). Judgment under uncertainty: Heuristics and biases. Cambridge University Press, 306-334. ISBN 978-0-521-28414-1. 
  • Shefrin, H. (2002). Beyond Greed and Fear: Understanding Behavioral Finance and the Psychology of Investing. Oxford University Press.
  • Sutherland, Stuart (2007). Irrationality. Pinter & Martin, 172-178. ISBN 978-1-905177-07-3. 

[edit] External links