Do investors suffer from behavioral biases? New research demonstrates that they do: They think that a crash is far more likely than it actually is. After you read the newspaper, you might well overreact to bad news about the market — and lose money as a result.

The best explanation is that investors suffer from what behavioral scientists call the “availability heuristic,” which distorts people’s decisions in many domains. In their pathbreaking work on human behavior, Amos Tversky and Daniel Kahneman found that people make judgments about probability by asking about which events come most readily to mind and hence are cognitively “available.”

For example, if you know someone who was recently in a car crash, you will probably overestimate the likelihood that the average person will get into a car crash. Hence the availability heuristic can produce “availability bias,” in the form of excessive fear (when it is easy to recall bad outcomes) and excessive complacency (when bad outcomes do not readily come to mind).

Back in 1973, Tversky and Kahneman used a series of simple tests to demonstrate the pervasive use of the availability heuristic. In a random English word, do you think that it is more likely that “k” will be the first letter, or instead the third? People tend to say that it will be the first, because it’s easier to think of words that start with k — even though a typical text will have twice as many words with k in the third position. (Awkward!)

Kahneman and Tversky speculated that the availability heuristic is used in important areas, as when “the fortuitous availability of incidents,” such as a recent airplane crash, ends up distorting people’s judgments. If people are preoccupied about an outcome, and so talk about it a lot, its availability in their minds will increase — which means that people will see it as far more likely than it actually is. This was an early suggestion that people can fall prey to “availability cascades,” which arise when a report of an apparent or potential event spread rapidly from one person to another, leading people to believe something that is not true. 

Which brings us to the new research, conducted by William Goetzmann and Robert Shiller of Yale, and Dasol Kim of Case Western Reserve. For 26 years, Shiller has asked investors to estimate the likelihood that a severe crash will occur over the next six months. A severe crash is defined to mean a stock market decline of at least 12.82 percent, as on Oct. 28, 1932.

The mean estimate over those 26 years has been 19 percent, and the average 10 percent. That’s pretty wild: If we look at the history of the Dow Jones Industrial Average, the actual probability has been about 1 percent.

Over any lengthy period, an estimate of 10 percent or higher is greatly inflated. It seems clear that American investors have been overestimating the likelihood of a crash.

What explains that persistent mistake? Goetzmann and his coauthors hypothesize that because of availability bias, investors give too much weight to recent information in assessing crash probabilities. They speculated that whenever the media reports on bad news about the stock market, investors will overreact.

To test that hypothesis, they searched through the Wall Street Journal for the 26-year period, looking for both positive and negative words, such as “stock market advance,” “market drop,” “market rise,” “stock market boom” and “stock market crash.” They predicted that investors would give excessive weight to the most recent information in deciding on the probability of a crash. On the basis of prior research, they also predicted that the availability bias would be “asymmetric,” in the sense that investors would be more influenced by negative events and terms than by positive ones.

The data confirmed that prediction. After the media speak in negative terms, investors end up giving a significantly higher estimate of the crash risk. The front page is what drives this result — further testimony to the power of the availability heuristic. Negative articles that are not on the front page have no effect.

At the same time, media uses of positive words, such as “advance” or “boom,” had no impact on people’s estimates. And institutional investors were largely immune to negative media stories, which didn’t have a statistically significant impact on their estimate of the likelihood of a crash.

A tempting objection to this research is that whenever words like “crash” appear on the front page of a newspaper, it’s pretty rational to increase your estimate of a crash. Where’s the bias there?

But behavioral scientists do not contend that use of the availability heuristic is irrational. Sensible people should and do update their beliefs on the basis of recent events — they just tend to overdo it. When people base their probability judgments on such events, those judgments often reflect a bias, in the form of an exaggeration of the statistical reality.

In this regard, the researchers’ best finding — their smoking gun — is that the estimates of institutional investors are impervious to negative news stories. Individual investors beware: If you’re constantly worried about a crash, you’re probably making some big mistakes, and losing a lot of money in the process.

By Cass R. Sunstein

Cass R. Sunstein, the former administrator of the White House Office of Information and Regulatory Affairs, is the Robert Walmsley university professor at Harvard Law School and a Bloomberg View columnist. — Ed.