Overconfidence is responsible for a lot of mistakes, here's how to avoid it

People are notoriously overconfident. Regardless of the context - sports, finance, politics - people believe that their judgements and decisions are better than they really are. The shock comes later after Steven Bradbury wins a Winter Olympic gold medal, Brexit destabilises financial markets, and Donald Trump wins the Republican nomination.

Overconfidence has been blamed for everything from the sinking of the Titanic to the Great Recession. Research into overconfidence implicates it in impairing judgements across a range of situations including investors’ over-trading behaviour, managers' poor forecasting, their tendency to introduce risky products, and their tendency to engage in value-destroying mergers.

Overconfidence is one of the most powerful cognitive biases because it is so ubiquitous, and causes us to make important judgements and decisions without a sensible degree of consideration. Fortunately, there are some strategies you can use to reduce overconfidence.

How do you know when you’re being overconfident?

Overconfidence is typically measured in terms of judgement accuracy when estimating a range of plausible outcomes. For example, when making a judgement about BHP Billiton’s future share price you could probably imagine a range of plausible prices within which you would be fairly confident the future price would fall. Scientists call this a “confidence interval”.

A confidence interval comprises of two numbers – a lower bound and an upper bound – that together create a range that you are, typically, 80% sure will include the true answer. For example, you might guess that BHP shares one year from today will be $25 and produce an 80% confidence interval with a lower bound of $15 and upper bound of $35.

In this example, you would be asserting 80% confidence that BHP shares in one year will be somewhere between $15 and $30. If asked to create a number different 80% confidence intervals for several different questions then 80% of these confidence intervals should turn out to be accurate and contain the true outcome.

Typically, however, accuracy rates are much lower than they should be. For example, in one comprehensive study, peoples’ 80% intervals contained the correct answer just 48% of the time. Therefore, people’s judgements are overconfident because the range of outcomes they consider plausible often misses the truth.

Why are people overconfident?

Although several theories have been proposed to explain why people are so overconfident, none of them explain all of the observations that scientists have made and so currently there is no overarching theory of overconfidence.

According to one theory, when making a judgement, people make an initial best guess that serves as the starting point and then estimate the range of plausible outcomes by expanding outward from that anchor. For example, if asked to give a plausible range for BHP’s future share price you might use the current share price, which is around $20, as the starting point, and then expand outward from that based on other factors.

According to this anchoring theory, people’s final range of plausible outcomes remains too close to the starting point and, as a result, they appear overconfident because their expected range often does not include the truth.

This theory predicts that setting an explicit anchor by having people first stating their starting point should increase overconfidence and yet research has found the opposite.

A second theory states that, when communicating with others, people prefer being informative to being accurate. For example, most people would prefer to guess BHP’s future share price to be between $15 and $25 than between $1 and $100. The latter is certainly more accurate but is relatively uninformative and not practically useful.

However, when people judge only a narrow set of outcomes to be plausible, they appear overconfident because their expected range often does not include the truth. This theory predicts the degree of overconfidence to change depending on the context (for example, how important accuracy is). However, there’s no evidence that such changes in context affect the degree of overconfidence.

A third theory states that overconfidence actually reflects extremely poor starting point guesses. For example, if you did not know the current BHP share price then your starting point might be way off, say $2,000. In this case, no matter how wide you expanded your range of plausible outcomes from this starting point, you will appear overconfident because your expected range would not include the truth.

This theory has support in laboratory contexts where judgements are made about chance events where the researchers can work out the correct range of plausible outcomes. However, this theory is impossible to test in most typical circumstances when the correct range cannot be calculated.

How can overconfidence be avoided?

Although overconfidence is one of the most powerful cognitive biases, there are some strategies that can be used to reduce it. The most effective strategies encourage consideration of more information and possible alternatives.

One strategy is to conduct a “pre-mortem”. To do this you make a best guess, then assume that guess is inaccurate, and then generate plausible reasons for why the guess was inaccurate. Research has found that overconfidence is reduced after asking people to list arguments that contradict the reasoning that led to the guess.

Alternatively, you can assume that your first guess is wrong and then think of a second guess that is based on different reasoning. Research reveals that averaging these two guesses tends to produce starting points that are more accurate than the first guess alone.

Another strategy uses the “wisdom of the crowd”. The strategy involves collecting the best guesses from others and then using the average of these guesses as your own starting point. Research shows that often the more estimates that are averaged the better, so long as the underlying reasoning (and hence, the errors) are different.

So, when making judgements, be humble, seek out new perspectives, and expect to make mistakes.