Models. Behaving. Badly.: Why Confusing Illusion with Reality Can Lead to Disaster, on Wall Street and in Life by Emanuel Derman. Free Press, 2011. 240pp.
We no longer use crystal balls to peer into the future. It seems that they are less efficient in figuring out what’s around the corner than silicon-supported algorithms and computer-generated data, at least in recent history. But the upside of crystal balls is their simplicity: they don’t require batteries and, with only a single component, you likely won’t need tech support (though I learned they can start fires if left uncovered in the sun). But if Emmanuel Derman, the author of Models. Behaving. Badly. is correct, computationally dependent models are neither simple nor efficacious.
Computational models take information and then make it interact according to rules and variables that economists, mathematicians, physicists, or other specialists establish. It is, in short, a more sophisticated version of the childhood “what happens if we put a spider and an ant in a jar and let them fight” scenario. Models, whether computer based or set up physically in a lab, make use of a very much simplified version of some part of the world that is allowed to interact under our control so that we can see what happens. Responsible model builders recognize how artificial this is and ensure that humility attends related predictions. But recent history reveals that the financial services sector has not had to trifle with humility.
At its heart, Models. Behaving. Badly. is a reflective lecture about the way our very complex world works and the various means we devise to make sense of it—in particular, models. Derman uses the categories of theories, models, and intuitions to frame his argument. We use theories to describe the essence of something and expect that our theories can be confirmed at some point. We use models to compare one thing we understand with another thing that we don’t in an effort to extract meaning from the puzzling. Models are not so much confirmed as defended: Does the model reflect the thing modelled and if so, in what ways? Intuition is knowledge that arises out of a long investment in a given domain.
The Efficient Market Model is a featured example and is intended to offer insight about future economic value, so you can make informed decisions about what to invest in or sell today. In order to work, this established model has to greatly simplify the world by creating an artificial one, and it does so by allowing only a few select variables out of an innumerable number of possibilities. It also has a built-in assumption that value will increase more quickly than risk. Derman notes that a core problem with this model is that “Risk grows more slowly [than value] in the model, though not necessarily in life itself,” introducing a deep cleavage between the real world and the financial model.
Derman wants readers to understand why models can’t predict the future, and why believing they can is dangerous. A word of advice to financially eager readers: Despite the subtitle, you won’t actually get to Wall Street until you are more than halfway through the book and have trammelled through a fair bit of physics. If you’re looking for a quick, direct assessment of how well physicists and mathematicians cum financial shamans have done, you’re in the wrong place. You would be better served by the March 2009 issue of WIRED in which Felix Salmon neatly eviscerates the systems that made use of David X. Li’s Gaussian copula function formula to simplify risk assessments and make wealthy people much more so, just before it crashed the financial system.
If, however, you are patient enough for a pensive reading of Spinoza coupled with Derman’s personal history as a companion case study, then you won’t be disappointed. Derman’s experience as a Wall Street quantitative analyst with Goldman Sachs has sharpened a number of convictions about what works and what doesn’t, and he’s keen to make sure readers have adequate grounding in the differences between theory, model, and intuition before the punch line emerges. Models can help to establish how different dynamics could interact and what changing one variable does to the other variables. This can help us improve our guessing about future stock prices, what the value of a given apartment is likely to be given the other apartments on the market, and so on. The trick is to remember that the guessing part is a permanent feature.
Committed readers will get some excellent analysis about the financial system in the last two chapters of the book. Having reached that point, you will have completed a short course on the philosophy of Spinoza and been given a tour of quantum electrodynamics (QED), “the best theory in the world,” including how Newton, Ampere, Faraday, Maxwell, and others played their part in getting it right. You’ll also have a sense of how badly the Efficient Market Model of finance fares in comparison. The former can predict properties at an atomic level “to an accuracy of more than 10 significant figures; the best models in finance are not accurate to even one.”
Having made it through these preliminaries, Derman believes readers are ready to hear how these ideas have a bearing on the habits of the Wall Street crowd. In keeping with the spirit of Joel Bakan’s pathology of corporations argument (see The Corporation), he argues that establishing legal entities with limited liability insulated individuals from prosecution for corporate misdeeds, and that this has made corporate exploitation easier, including the misuse of financial models. The problem is this: if investments can be amplified, so too can failures. Finance is far more uncertain than physics, so the downside exposure turns out to be significant. If models are hinged to this fragile structure through irresponsible forecasting, their contribution becomes sinister.
We use models in finance to try to predict the future so that we can gain an advantage for ourselves or the clients we serve by buying low today and selling high tomorrow. Life is messy, and the promise of technology has expanded to include improvements not only in how we wash our clothes, but also in how we take stock of the future. As far as possible, knowing what’s coming is important. In the classic book The Long Winter, Laura Ingalls Wilder tells of how Pa knew the coming winter would be a bad one, even in July, because the muskrats built enormous mounds to insulate themselves. This proved to be critical advance notice in preserving themselves.
Derman encourages us to foster the same kind of wisdom—preparation for future possibilities, rather than over-reliance on calculations. He argues that our faith in models is misplaced. They are useful but inherently crude, and we should be careful when depending on them. We can’t predict earthquakes—and financial markets are more complex than tectonic plates crashing together.
Pulling back the curtain on economic wizardry, Berman reminds readers that long equations don’t equal future certainty, however clever or broadly adopted they may be. The most important insight that Derman offers is not related to modelling but to the human drama on which the latest financial crisis has cast its spotlight:
At the end of the cold war we imagined a future with no more history, a smooth stroll into the sunrise accompanied by democracy, privatization, and free markets. It hasn’t worked out that way. Authoritarian versions of capitalism have spread. Privatization has become oligarchy. The gaps between rich and poor, managers and workers, and owners and employees have widened. Economic models have misfired and financial models have proved enormously inaccurate . . . Lessons have not been learned.
The book could be strengthened through a shift of the some of the last third of the material forward. The four brief paragraphs on money where he outlines that those who are without greater purpose turn to accumulating money are a pithy set of ideas that we shouldn’t have to wait 88 pages for. I didn’t find the Spinoza diagrams very helpful, due in part to the painful font selections and diagram shapes that looked like an unfortunate graphic design collision between a computer logic class and the annual meeting of the Clairvoyants of America. The physics diagrams are less ambiguous and therefore more effective.
Despite these shortcomings, the deep message of the book—that we need to pay much more attention to what we don’t know and set aside the hubris of our neo-tech crystal balls—remains a scarce but vital commodity in the financial sector, and in other areas of life. We need models. But if a careful humility does not accompany their application, history tells us that there are many other blind alleys we can yet drive down: “The perfect model doesn’t exist, so we have to use imperfect ones intelligently.” Hopefully Derman’s warnings will contribute to more robust policies and practices in the financial industry and a shift in where we place our trust as a society.
