Why I Distrust Models

People like simple answers. People like to feel that they understand complex reality. Science shows us that we can master some things. But while elegant physics formulas point at the heart of reality, people forget that physics formulas apply to very "pure" events. Muddled reality is poorly represented by the mathematical models of physics.

The complex global circulation models of the Global Warming crowd point to unambiguous futures, but are misleading.

The wonderfully complex financial engineering models from the Black-Scholes model for pricing options to the latest gee whiz model in the Gaussian copula function are all models that are more fragile than their proponents recognize. Here's an article by Felix Salmon in Wired Magazine that recounts the horrors:

A year ago, it was hardly unthinkable that a math wizard like David X. Li might someday earn a Nobel Prize. After all, financial economists—even Wall Street quants—have received the Nobel in economics before, and Li's work on measuring risk has had more impact, more quickly, than previous Nobel Prize-winning contributions to the field. Today, though, as dazed bankers, politicians, regulators, and investors survey the wreckage of the biggest financial meltdown since the Great Depression, Li is probably thankful he still has a job in finance at all. Not that his achievement should be dismissed. He took a notoriously tough nut—determining correlation, or how seemingly disparate events are related—and cracked it wide open with a simple and elegant mathematical formula, one that would become ubiquitous in finance worldwide.

For five years, Li's formula, known as a Gaussian copula function, looked like an unambiguously positive breakthrough, a piece of financial technology that allowed hugely complex risks to be modeled with more ease and accuracy than ever before. With his brilliant spark of mathematical legerdemain, Li made it possible for traders to sell vast quantities of new securities, expanding financial markets to unimaginable levels.

His method was adopted by everybody from bond investors and Wall Street banks to ratings agencies and regulators. And it became so deeply entrenched—and was making people so much money—that warnings about its limitations were largely ignored.

Then the model fell apart. Cracks started appearing early on, when financial markets began behaving in ways that users of Li's formula hadn't expected. The cracks became full-fledged canyons in 2008—when ruptures in the financial system's foundation swallowed up trillions of dollars and put the survival of the global banking system in serious peril.

David X. Li, it's safe to say, won't be getting that Nobel anytime soon. One result of the collapse has been the end of financial economics as something to be celebrated rather than feared. And Li's Gaussian copula formula will go down in history as instrumental in causing the unfathomable losses that brought the world financial system to its knees.

When someone claims to "know" how something works... be very fearful, demand extensive testing, require independent audit and verification, and above all: avoid joining in a bandwagon effect of mindless endorsement and blind adherence.