Of all people, Nate Silver should probably not have been gloating the morning after Election Day. After all, having made his reputation as a statistical wunderkind by predicting 49 states correctly in the 2008 race, Silver called five states wrong in the 2016 election, assuming Hillary Clinton would end up with 302 electoral votes (she got 232).
In fact, the entire 2016 campaign season was been characterized by a series of spectacular Silver blunders. Not only did he notoriously give Hillary Clinton a greater than 99% chance of winning the Michigan primary (she lost), and bungle Indiana as well, but he spent much of the past 18 months emitting a series of embarrassing declarations as well as ludicrous prophecies that totally failed to materialize. Let us go through a sample:
“Donald Trump is winning the polls and losing the nomination.” — Aug. 11, 2015
“Dear media, Please stop freaking out about Donald Trump’s polls.” — Nov. 23, 2015.
“Trump is doubling down on a losing strategy.” — Aug. 18, 2016
On the whole, it’s a humiliating record. In the primaries, Silver didn’t even do as well as Carl Diggler, a fictional parody-pundit who literally just makes stuff up based on whatever his gut tells him. Presuming Silver is supposed to be something different from the rest of the jabbering punditocracy, his career should be over.
Yet bizarrely, in the days after the election, Silver was bragging about his performance. Silver insisted that after Election Night, he felt vindication, and scoffed that some major pundits had been “smugly dismissive of Trump’s chances.” Looking back on Silver’s record of statements on Trump, one wonders to which pundits he may have been referring. For over a year (July 2015 to Aug. 2016) he wrote smug “dear media” letters about Trump-hype and called Trump’s strategy “delusional,” insisting that Trump just didn’t understand the math. Having expressed regret after the primary for “acting like a pundit” and underestimating Trump, in the general election he was still acting like a pundit and underestimating Trump.
Thus Silver took a cheerful victory lap, despite having totally failed, repeatedly and embarrassingly, to provide any information of use. He bases his claim to have succeeded off his having given Trump a somewhat higher probability of a win than some other people, despite still thinking Clinton was the definite favorite. But it doesn’t take a statistical genius to be cautious in a situation of high volatility. (The main reason Silver is being praised for being wrong is that a man named Sam Wang of something called the Princeton Election Consortium was even more wrong, giving Clinton a 99% probability of a win.)
The myth of Nate Silver’s continued usefulness is based on a careful moving of goalposts. His initial claim to fame was based on number of states correctly predicted. But in 2016, if we measured by that number (especially if we subtracted the states whose outcomes were most obvious), Silver wouldn’t look good at all. So now we’re invited to focus on a different statistic, the percentage chance of an overall Trump win. Conversely, when it’s the percentage chance that goes wrong, Silver reminds us how many races he called correctly. Like a television psychic, Silver is able to carefully draw your attention to that which he gets right and ignore that which he gets wrong. If the probability percentages look good, but he screws up a large number of races, we should look at percentages. If those look terrible, as they did in Michigan, we should forget them and think about numbers of states.
Similarly, Silver will make predictions that have multiple components, so that if one part fails, the overall prediction will seem to have come true, even if its coming true had no relation to the reasons Silver originally offered. See, e.g., “It’s a tight race. Clinton’s the favorite but close enough that Trump would probably pull ahead if he ‘wins’ debate.” Silver can look back and say “I saw that Trump could pull ahead.” But what he actually predicted was that Trump could pull ahead based on debate performance. If he pulls ahead for some other reason, Silver is completely wrong (because he had excluded that other possibility), yet he seems right.
When one goes through Silver’s Twitter feed for the election cycle, one sees him predicting nearly every damn thing in the universe. Sometimes Clinton is winning, sometimes Trump is winning. Sometimes anything could happen, as in the below tweet:
Silver makes sure to hedge every statement carefully so that he can never actually be wrong. And when things don’t go his way, he lectures the public on their ignorance of statistics. After all, probability isn’t certainty, he didn’t say it would definitely happen. And of course, that’s completely true. But recognize what it means: even when Silver isn’t wrong, because he’s hedged everything carefully, he’s still not offering any information of value. Sophisticated mathematical modeling, just like punditry, can’t tell us much about the things we most need to know. It can’t predict the unpredictable, and the unpredictable is what matters most of all.
Donald Trump was trying an untested experiment. You couldn’t easily put numbers on it. Anyone who did was destined to be pulling the statistics from their ass, because there was no way for human beings to access the relevant information. The critical question was not: what do the polls, after some defensible adjustments, say about the candidates’ chances? It was “What happens when a bombastic, widely disliked male real-estate tycoon and a technocratic, widely disliked female Secretary of State go up against one another in a highly volatile race involving race, economics, the FBI, Wikileaks, and sexual assault allegations?” Since nothing like this has ever happened in human history, it was destined to be the case that the best thing you could do was be somewhat cautious.
Silver actually knows all of the limitations of his work, and states them openly: “Statistical models work well when you have a lot of data, and when the system you’re studying has a relatively low level of structural complexity. The presidential nomination process fails on both counts.”Thus the sneaky thing Silver does is this: he fills his work with caveats, but then turns around and writes articles like “The Six Stages of Donald Trump’s Doom,” in which he lays out very vivid, totally fantastical and unfounded, sets of forecasts about the future. In the primary, he foresaw a situation in which Bernie Sanders would win two states and then nowhere else, an idea that turned out to be doubly wrong (he lost one of the two, and then he won a bunch of others). None of this has any grounding beyond Silver’s gut.
This is why Silver is irresponsible and untrustworthy. It’s not, as the Huffington Post stupidly alleged, that he’s a bad or biased statistician. It’s that he mingles solid statistical observations (of highly limited usefulness) with wild prophecy and the same old know-nothing horse-race punditry. He acts as if statistics and polls can tell us to some useful degree whether Trump’s highly unorthodox political strategy will work. He offers totally worthless speculative scenarios, such as Bernie Sanders losing all but two states, even though the dynamics that would lead to such scenarios are not accessible to human observation or prediction. And over the course of the election, he used his authority and credibility as a numbers genius to tell people not to worry about Donald Trump, and to treat those who were “freaking out” as if they had were idiots.
But the central problem with Silver is that ultimately, he’s producing horse-race stuff. He doesn’t actually care about politics very much in terms of its human stakes. (In fact, according to journalist Doug Henwood, Silver once said that he “doesn’t give a shit” about politics.) He’s producing entertainment; people refresh FiveThirtyEight for the same reason that they watch actual horse races. But for anyone interested in the actual human lives affected by political questions, Silver’s analyses are of almost no help. They can tell us today that Silver thinks Trump has a 5% chance of winning. But then we might wake up tomorrow and find that Silver now thinks Trump has a 30% chance of winning. And the important question for anyone trying to affect the world, as opposed to just watching the events in it unfold, is how those chances can be made to change.
That doesn’t mean there’s anything wrong with Nate Silver, just that nobody should ever pay any attention to him. Nate Silver will probably always be the best poll data analyst. The problem is that poll data analysts are completely fucking useless in a crisis. They don’t understand anything that’s going on around them, and they’re powerless to predict what’s about to happen next. Listening to anything they have to say is very, very dangerous. If you want to change anything, you’ve got to forget Nate Silver forever. That’s because he tells you entirely about the world as it looks to him right now, rather than the world as it could suddenly be tomorrow. He has no idea what the outer boundaries of the possible are. Nobody does.
