NEW HAVEN ― We are in the midst of a boom in popular economics: books, articles, blogs, public lectures, all followed closely by the general public.

I recently participated in a panel discussion of this phenomenon at the American Economic Association annual meeting in Denver. An apparent paradox emerged from the discussion: the boom in popular economics comes at a time when the general public seems to have lost faith in professional economists, because almost all of us failed to predict, or even warn of, the current economic crisis, the biggest since the Great Depression.

So, why is the public buying more books by professional economists?

The most interesting explanation I heard was that economics has become more interesting, because it no longer seems to be a finished and closed discipline. It is no fun to read a book or article that says that economic forecasting is best left to computer models that you, the general reader, would need a Ph.D. to understand.

And, in truth, the public is right: while there is a somewhat scientific basis for these models, they can go spectacularly wrong. Sometimes we need to turn off autopilot and think for ourselves, and when a crisis occurs, use our best human intellect.

The panelists all said, in one way or another, that popular economics facilitates an exchange between specialized economists and the broader public ― a dialogue that has never been more important. After all, most economists did not see this crisis coming in part because they had removed themselves from what real-world people were doing and thinking.

Successful popular economics involves the reader or listener, in some sense, as a collaborator. That, of course, means that economists must be willing to include new and original theories that are not yet received doctrine among professional specialists.

Until recently, many professional economists would be reluctant to write a popular book. Certainly, it would not be viewed favorably in considering a candidate for tenure or a promotion. Since it does not include equations or statistical tables, they would argue, it is not serious work that is worthy of scholarly attention.

Worse than that, at least until recently, a committee evaluating an economist would likely think that writing a popular economics book that does not repeat the received wisdom of the discipline might even be professionally unethical.

Imagine how the medical profession would view one of its members who recommended to the general public some therapy that had not yet passed scrutiny from the appropriate authorities. Medical professionals know how often seemingly promising new therapies turn out, after careful study, not to work, or even to be harmful. There is a rigorous process of scholarly review of proposed new therapies, associated with professional journals that uphold high research standards. Circumventing that process and promoting new, untested ideas to the general public is unprofessional.

In the decades prior to the current financial crisis, economists gradually came to view themselves and their profession in the same way, encouraged by research trends. For example, after 1960, when the University of Chicago started creating a Univac computer tape that contained systematic information about millions of stock prices, a great deal of scientific research on the properties of stock prices was taken as confirming the “efficient markets hypothesis.” The competitive forces that underlie stock exchanges were seen to force all securities prices to their true fundamental values. All trading schemes not based on this hypothesis were labeled as either misguided or outright frauds. Science had triumphed over stock-market punditry ― or so it seemed.

The financial crisis delivered a fatal blow to that overconfidence in scientific economics. It is not just that the profession didn’t forecast the crisis. Their models, taken literally, sometimes suggested that a crisis of this magnitude couldn’t happen.

One way to interpret this is that the economics profession was not fully accounting for the economy’s human element, an element that can’t be reduced to mathematical analysis.

The relatively few professional economists who warned of the current crisis were people, it seems, who not only read the scholarly economics literature, but also brought into play more personal judgment: intuitive comparisons with past historical episodes; conclusions about speculative trading, price bubbles, and the stability of confidence; evaluations of the moral purposes of economic actors; and impressions that complacency had set in, lulling watchdogs to sleep.

These were judgments made by economists who were familiar with our business leadership ― their inspirations, beliefs, subterfuges, and rationalizations. Their views could never be submitted to a scholarly journal and evaluated the way a new medical procedure is. There is no established scientific procedure that could prove their validity.

Of course, economics is in many ways a science, and the work of our scholars and their computer models really does matter. But, as the economist Edwin R. A. Seligman put it in 1889, “Economics is a social science, i.e., it is an ethical and therefore an historical science….It is not a natural science, and therefore not an exact or purely abstract science.”

To me, and no doubt to the other panelists, part of the process of pursuing the inexact aspects of economics is speaking honestly to the broader public, looking them in the eye, learning from them, reading the emails they send, and then searching one’s soul to decide whether one’s favored theory is really close to the truth.

By Robert J. Shiller

Robert Shiller, a professor of economics at Yale University and chief economist at MacroMarkets LLC, is co-author, with George Akerlof, of “Animal Spirits: How Human Psychology Drives the Economy and Why It Matters for Global Capitalism.” ― Ed.

(Project Syndicate)