By Daniel Jepson
Nate Silver is a figure who needs no introduction – thanks to the spectacular success of his election forecasting system, he has become a household name in recent years. In late 2012 he released a book, The Signal and The Noise, which quickly became a bestseller. In it, he discusses the art of using data intelligently in order to make predictions, with illustrative chapters showing how the ideas can be applied to fields ranging from climate science to poker. It is, overall, a superb book, meticulously researched and lucidly written, and Silver’s versatility in discussing such a wide variety of real-world applications is particularly impressive.
However, Silver conspicuously fails to ask one very important question: how do we know which disciplines are amenable to this type of empirical reasoning in the first place? Nowhere in the book does he question the assumption – so common in modern discourse – that the road to understanding always lies in data; that if a field of inquiry can conceivably be approached via study of quantitative variables, there is no question that it should. As such, it will come as no surprise to an Austrian reader that when Silver turns his attention to economics, the results are far from convincing, even when taken on their own terms. Interestingly, the economics chapter does contain a healthy dose of Silver’s typically incisive reasoning; despite the inauspicious choice of Paul Krugman as one of his primary sources, he nonetheless manages to form a perceptively critical evaluation of the profession’s status quo, astutely highlighting many of the difficulties that economists currently face. But because he stops short of questioning the central premise of modern economics – that an economy can be evaluated through statistical measurements, and that forecasting these measurements is therefore what economics is ultimately all about – he is unable to resolve these difficulties satisfactorily, and his conclusions end up ringing decidedly hollow. A later chapter in the book is devoted to the Efficient Market Hypothesis, and the particular phenomenon of bubbles; here Silver’s approach gets him into even deeper trouble, and his explanation ends up being entirely unconvincing. It may seem unfair and unproductive to criticize an economics discussion in what is primarily a book about other topics. We nonetheless find it worthwhile to do so, because The Signal and The Noise serves as a particularly vivid illustration of the importance of methodology in the ongoing debate between Austrians and mainstream economists.
Silver’s chapter on economics gets off to a promising start, as he takes the profession to task for their woeful record of predictive success. He notes that economists are much too confident in their GDP-forecasting ability – leading them to make predictions that have proven to be “poor in a real-world sense” – and points out that in contrast with fields such as meteorology, economic forecasts have shown little improvement over the past few decades. Guided by his discussion with Goldman Sachs chief economist Jan Hatzius, Silver then offers several very insightful explanations for why this is so. To begin with, he notes that there is little stability in the cause-and-effect relationships that emerge from the study of economic variables – for example, five of the seven “leading indicators” of the 1990 and 2001 recessions failed to indicate a problem in 2007. Furthermore, economic forecasting involves not only anticipating changes in policy decisions, but correctly gauging how these changes might impact the forecasting model itself. He cites Goodhart’s Law, which tells us that the targeting of variables by policy-makers causes them to lose their predictive value – housing prices, for instance, cease to function as a bellwether of increasing prosperity when they are deliberately manipulated by government policy. As Silver explains: “Most statistical models are built on the notion that there are […] inputs and outputs, and they can be kept pretty much separate from one another. When it comes to the economy, they are all lumped together in one hot mess.” A related problem is that the economy is a complex, ever-changing entity; as such, a seemingly well-established empirical relationship can cease to hold, seemingly without warning. This phenomenon was well illustrated in 2009, when the venerable Okun’s Law suggested that 2 million jobs should have been gained – instead, 3.5 million were lost. And as if this were not enough, we are also faced with the difficulty that much economic data is simply not very good, subject as it is to official revision months or years later. As an extreme case, the initial 4.2 percent growth estimate for the fourth quarter of 1977 was eventually rewritten as a contraction of 0.1 percent.
At this juncture, the reader is surely entitled to ask an obvious question: why are we so certain that empirical analysis is the right approach to economics in the first place? After all, we have been convincingly shown that economic data is inherently unreliable, and that even when taken at face value, it necessarily maintains only a tenuous correspondence with the real-world phenomena that it purports to explain. Surely one very reasonable conclusion would be that economics is simply a discipline that is most effectively approached through purely deductive methods – this, of course, is what Austrians have maintained all along. But amazingly, not only does Silver fail to embrace this idea, he never even seems to consider it as a possibility. While he does praise Hatzius for basing his gloomy 2007 forecast on a coherent narrative (being “right for the right reasons”) , it remains clear that he views logic merely as a guiding force toward better empirical predictions, rather than the methodological substance of good economics in its own right. The idea that one could dispense altogether with the statistical element remains an anathema.
A particularly striking microcosm of the entire chapter occurs when Silver acknowledges that the steady GDP growth of so-called Great Moderation between 1983-2006 was “fueled by large increases in government and consumer debt, along with various asset-price bubbles.” Very true: a more compelling indictment of GDP-based economic analysis would be hard to imagine! But Silver brings this up only in order to illustrate his point that changing economic conditions make statistical analysis a difficult task in economics. And so it leads him to offer only the stunningly insipid conclusion that the Fed may have erred in their 2007 GDP forecast because they failed to adequately consider data from prior to 1983. (!!)
This leaves Silver in something of a bind when he tries to offer a prescription for how the economics profession might improve its performance. He remains implicitly devoted to the idea that this performance must be in the form of statistical forecasting – yet he has just finished presenting strong evidence that this approach is merely an exercise in futility. So it is no surprise that the chapter ends on a feeble note: apart from the aforementioned suggestion that predictions should be based on a logical understanding of how the world works (with which, needless to say, we entirely agree) his only substantial suggestion is that economists need to be given stronger incentives to make good predictions. But surely this is hopelessly far-fetched. As Silver himself notes, the resulting implication is that there currently exist willing consumers of bad economic forecasts; this is a rather implausible notion, and Silver gives no real evidence of why we should accept it. Moreover, even if we grant that professional incentives are in some sense a problem, it remains unclear how a successful resolution would be sufficient to overcome the formidable obstacles to successful prediction that Silver has just outlined. Will the economy cease to be a “hot mess” of inputs and outputs, just because economists are more motivated to provide accurate forecasts? As a result, in stark contrast with the other sections of the book, the reader comes away from this chapter having been offered no convincing explanation of how signal and noise are to be separated in economics.
That Silver has failed to develop an entirely sound understanding of economics is confirmed several chapters later, when he turns his attention to financial markets, and the phenomenon of bubbles in particular. This is a prime example of an issue where the logical approach to economics favored by Austrians proves its superiority. It is quite easy to identify the underlying cause of bubbles through a simple thought experiment: where is the money coming from to support these ever-rising asset prices? Is there any evidence whatsoever that they are financed by decreased expenditure on other goods and services? When the question is considered in this manner, it immediately becomes clear that inflation of asset prices, no less than of consumer prices, is always and everywhere a monetary phenomenon. On the other hand, bubbles are inherently difficult to study empirically, so it is no surprise that commentators beholden to the positivist approach have largely come to grief in their efforts to explain them.
Unfortunately, Silver proves to be no exception. Like so many other writers in the wake of the financial crisis, he apparently believes that psychological considerations alone constitute a sufficient explanation, and his variation on this well-worn theme is no less fundamentally inadequate. He begins by blaming bubbles on the incentives of individual traders: “so long as most traders are judged on the basis of short-term performance, bubbles involving large deviations of stock prices from their long-term values are possible – and perhaps even inevitable.” But the reasoning that he offers in support of this is blatantly circular: he notes that given the empirical probability of crash in stock prices, it can take a long time for a bubble to burst, and claims (not implausibly) that anyone who prematurely calls the top during this time is likely to find himself out of a job. The problem, of course, is that the frequency of market crashes is itself the result of the aggregate actions of individual traders – it cannot be the ultimate cause as well. Silver’s attempt to ground his case in an innate psychological propensity for “herding” behavior does not fare any better. In support, he refers to a 2008 InTrade incident, where a rogue trader temporarily pumped up the market value of John McCain’s election probability, only to see the “true” price restored six hours later. But it is difficult to understand what he is getting at here. Surely this anecdote actually provides stronger evidence for the Austrian conclusion: that in the absence of sustained manipulation, the market will rapidly smooth out all misaligned prices. Why did the InTrade distortion resolve itself in a matter of hours, instead of turning into a long-term mania? (Conversely, would anyone have noticed the housing bubble if it had lasted for six hours?) Finally, as if somehow sensing that he has not quite proven his case, Silver tosses in one final attempt at an explanation near the end of the chapter: “There might be a terrific opportunity to short a bubble […] once every fifteen or twenty years when one comes along in your asset class. But it’s very hard to make a steady career out of that, doing nothing for years at a time.” This, however, is simply bizarre: why on earth should we assume that “bubble-popping” is such a specialized career niche that its practitioners are incapable of other activity during normal market conditions?
The upshot is that bubbles prove to be an unfortunate lacuna in the context of The Signal and The Noise, just as they are for mainstream economics in general. Silver’s discussion is sadly illustrative of the hopeless muddle that invariably results once commentators fall into the trap of attempting to explain economic processes through purely psychological mechanisms. It also highlights the vital importance of methodology in the study of economic phenomena. Had Silver come to the conclusion – so thoroughly implied by his discussion in the economics chapter – that it is a discipline best approached through a priori reasoning, he would surely have found the correct answer without difficulty. As it is, having initially chosen the wrong set of tools, not even his powerful intellect was able to construct anything close to a compelling explanation.
In The Signal and The Noise, Nate Silver offers Austrians a ray of hope, by revealing the degree of skepticism that many well-informed commentators possess toward mainstream economics. More importantly, however, he also gives us an invaluable (if inadvertent) reminder of where we must focus our rhetorical efforts, in order to take full advantage of this state of affairs. It is nothing short of astonishing that a thinker of Silver’s caliber, having marshaled such an impressive array of evidence against the mainstream data-crunching approach, should fail to even consider Austrian-style methodology as a possible alternative. This is a quintessential example of the unquestioning belief in positivism that underlies contemporary thought – “data uber alles” seems to be the credo of 21st-century epistemology. It is this methodological error that we must first seek to correct, if we ever hope to make substantial progress in converting intelligent, open-minded people – such as Nate Silver – to Austrian economics.
This article first appeared at: http://www.misesboston.com/2014/05/the-signal-and-the-noise-by-nate-silver-an-austrian-review/