FiveThirtyEight: The Number of Things Nate Silver Gets Wrong About Climate Change

Posted on

"FiveThirtyEight: The Number of Things Nate Silver Gets Wrong About Climate Change"

The climate science literature is vast. It merits broad and deep reading by anyone planning to write about it. The fact is the IPCC forecasts have generally underestimated key trends, including warming (see here) and greenhouse gas emissions and Arctic sea ice loss and ice sheet disintegration. I explain why here. Finally, the IPCC generally overstates uncertainty because it insists on conflating uncertainty in future emissions with uncertainty in the climate’s sensitivity to those emissions. Continuing to take no serious action on climate eliminates almost all of the uncertainty as to whether or not future impacts will be catastrophic. Even while publishing this piece by one of the country’s top climatologists debunking the climate analysis in Nate Silver’s new book, I remain a big fan of Silver’s polling analysis (as does Mann) — Joe Romm.

by Michael Mann

If you’re a science or math geek like me, you can’t help but like Nate Silver. He’s the fellow nerd who made good. His site is a must for any serious polling buff, and he regularly graces the leading talk shows with his insightful if wonky commentary. So you can imagine how excited I was a year ago when Nate’s assistant contacted me, indicating that he wanted to come to State College, PA — the “happy valley” — to interview me for his new book on “forecasting and prediction.”

Nate, I was told, was working on a chapter about global warming. He sought me out because he felt my expertise would make me an “excellent guide to the history of climate modeling”. He also expressed interest in my own upcoming (since published) book The Hockey Stick and the Climate Wars which details my experiences at the center of the climate change debate. Needless to say, I was very much looking forward to the meeting.

And so it was on a crisp early November day that Nate arrived at my office in the Walker Building of the Penn State campus. We exchanged pleasantries and proceeded to engage in a vigorous, in-depth discussion of everything from climate models and global warming to the role of scientific uncertainty, and the campaign by industry front groups to discredit climate science (something that is the focus of my own book). As I saw Nate off, I insisted he sample the Penn State Creamery’s famous ice cream before leaving town. I tweeted excitedly about my meeting with him, and by the end of the day Nate had even added me to his relatively short list of twitter followees. Certain our discussion had been productive and informative, I awaited Nate’s book with great anticipation.

And so I was rather crestfallen earlier this summer when I finally got a peek at a review copy of The Signal and the Noise: Why So Many Predictions Fail — but Some Don’t. It’s not that Nate revealed himself to be a climate change denier; He accepts that human-caused climate change is real, and that it represents a challenge and potential threat. But he falls victim to a fallacy that has become all too common among those who view the issue through the prism of economics rather than science. Nate conflates problems of prediction in the realm of human behavior — where there are no fundamental governing ‘laws’ and any “predictions” are potentially laden with subjective and untestable assumptions — with problems such as climate change, which are governed by laws of physics, like the greenhouse effect, that are true whether or not you choose to believe them.

Nate devotes far too much space to the highly questionable claims of a University of Pennsylvania marketing Professor named J. Scott Armstrong. Armstrong made a name for himself in denialist circles back in 2007 by denouncing climate models has having no predictive value at all. Armstrong’s arguments were fundamentally flawed, belied by a large body of primary scientific literature — with which Armstrong was apparently unfamiliar — demonstrating that climate model projections clearly do in fact out-perform naive predictions which ignore the effect of increasing greenhouse gas concentrations. As discussed in detail by my co-founder, NASA scientist Gavin Schmidt, Armstrong simply didn’t understand the science well enough to properly interpret, let alone, assess, the predictive skill of climate model predictions.

That Nate would parrot Armstrong’s flawed arguments is a major disappointment, especially because there are some obvious red flags that even the most cursory research should have turned up. A simple check of either SourceWatch or fossil fuel industry watchdog ExxonSecrets, reveals that Armstrong is a well-known climate change denier with close ties to fossil fuel industry front groups like the Heartland Institute, which earlier this year campaigned to compare people who accept the reality of climate change to the Unabomber, and secretly planned to infiltrate elementary schools across the country with industry-funded climate change denial propaganda.

I suspect that Nate’s failing here arises from a sort of cultural bias. There is a whole community of pundits with origins in economics and marketing who seem more than happy to dismiss the laws of physics when they conflict with their philosophy of an unregulated market. Nate may not share that philosophy, but he was educated by those who do.

Nate Silver was trained in the Chicago school of Economics, famously characterized by its philosophy of free market fundamentalism. In addition to courses from Milton Friedman, Nate might very well have taken a course from University of Chicago economist Steven Levitt, known largely for his provocative 2005 book Freakonomics and its even more audacious 2009 sequel Super Freakonomics — a book that, perhaps better than any other, serves as a cautionary tale of the dangers that lurk when academics attempt to draw sweeping conclusions in fields well outside their area of training. In Super Freakonomics as you might guess, Levitt drew questionable conclusions about climate change and related energy issues based on an extrapolation of principles of economics way, way, way, outside their domain of applicability. Even some very basic physics calculations, for example, reveal that his dismissal of solar energy as a viable alternative to fossil fuel energy in combating climate change because of possible waste heat is total nonsense. Ray Pierrehumbert, a chaired professor himself at the University of Chicago, in the Department of Geosciences, pointed this and other serious errors out to Levitt in an open letter that concluded with a campus map showing how easy it would have been for Levitt to walk over to his office to discuss his ideas and, presumably, avoid the serious pitfalls that ended up undermining much of what he ended up saying in his book about climate change and energy policy.

Unlike Levitt, Nate did talk to the scientists (I know. I’m one of them!). But he didn’t listen quite as carefully as he should have. When it came to areas like climate change well outside his own expertise, he to some extent fell into the same “one trick pony” trap that was the downfall of Levitt (and arguably others like Malcolm Gladwell in The Tipping Point). That is, he repeatedly invokes the alluring, but fundamentally unsound, principle that simple ideas about forecasting and prediction from one field, like economics, can readily be appropriated and applied to completely different fields, without a solid grounding in the principles, assumptions, and methods of those fields. It just doesn’t work that way (though Nate, to his credit, does at least allude to that in his discussion of Armstrong’s evaluation of climate forecasts).

As a result, Nate’s chapter on climate change (Chapter 12: “A Climate of Healthy Skepticism”) is marred by straw man claims that don’t stand up to scrutiny. These include the assertion that (a) climate scientist James Hansen’s famous 1988 predictions overestimated global warming (they didn’t), that (b) “the Intergovernmental Panel on Climate Change (IPCC) settles on just one forecast that is endorsed by the entire group” (pure nonsense — even the most casual reading of the IPCC reports reveal that great care taken to emphasize the non-trivial spread among models predictions, and to denote regions where there is substantial disagreement between the projections from different models) and that (c) “relatively little is understood” about the El Nino cycle (here I imagine that Nate might have misinterpreted our own discussion about the matter; I explained in our discussion that there are still open questions about how climate change will influence the El Nino phenomenon — but that hardly means that we know “relatively little” about the phenomenon itself! In fact, we know quite a bit about it). Finally, and perhaps most troubling (d) while Nate’s chapter title explicitly acknowledges the importance of distinguishing “signal” from “noise”, and Nate does gives this topic some lip service, he repeatedly falls victim to the fallacy that tracking year-to-year fluctuations in temperature (the noise) can tell us something about predictions of global warming trends (the signal). They can’t — they really can’t.

Nate’s view of uncertainty, and its implications for climate model predictions, is particularly misguided. He asserts that the projections of the IPCC forecasts have been “too aggressive”, but that is simply wrong. It neglects that in many cases, e.g. as regards the alarming rate of Arctic sea ice decline (we saw a new record low set just weeks ago), the climate models have been far too cautious; We are decades ahead of schedule relative to what the models predicted. Uncertainty cuts both ways, and in many respects — be it the rapid decline in Arctic sea ice, or the melting of the ice sheets — it is cutting against us. Uncertainty, as many economists recognize, is thus a reason for action, not inaction! I’m surprised someone as sharp as Nate just doesn’t appear to get that.

Nate also takes some unnecessary cheap shots. In what has now become a rite of passage for those looking to establish their “honest broker” bona fides in the climate change debate, Nate makes the requisite “punch the hippie” accusation that Al Gore exaggerated the science of climate change in An Inconvenient Truth (a team of climate scientists reviewed the movie for accuracy and found that by-and-large Gore got the science right). He characterizes climate scientist Gavin Schmidt as a “sarcastic” individual who is unwilling to put his money where his mouth is by betting his personal savings on his climate model predictions (this felt to me reminiscent of Mitt Romney’s widely mocked $10,000 bet challenge to Rick Perry). And while I do appreciate some of the nice things Nate says in the book about me personally (e.g. “Mann is exceptionally thoughtful about the science behind global warming”), he at the same time deeply misrepresents our discussion on several counts.

I had emphasized the importance of distinguishing the true uncertainties in climate science (and there are plenty e.g. the influence of warming on hurricanes, how the El Nino phenomenon might be affected, or how regional patterns of rainfall may change) from the manufactured uncertainties and myths typically promoted by climate change deniers and contrarians (e.g. “how come there has been no warming since 1998?” — the answer is that, of course, there has been). I stressed how important it is, when scientists communicate to the public, to make clear that while there are many details that are still uncertain, the big picture (that humans are warming the planet and changing the climate, and that far larger and potentially more dangerous changes loom in our future if we don’t act) is not.

Nate cherry-picks a single sound bite (“our statements [should not be] so laden in uncertainty that no one even listens.”) to once again reinforce the false narrative that scientists are understating uncertainty. The point I was actually making was that we cannot spend so much time talking about what we don’t know, that we don’t end up telling the public what we do know. That, as Nate correctly quotes me, “would be irresponsible”. Nate states that “the more dramatic [climate scientists’] claims, the more likely they [are] be quoted…”, seemingly implying that scientists have a motivation to overstate the science. He ignores the fact that those scientists willing to feed the false “scientists are exaggerating” narrative are the true darlings of the “balance” over “objectivity” school of news reporting — a school of thought that Nate sadly seems to have subscribed to.

Most disappointing to me of all was the false equivalence that Nate draws between the scientific community’s efforts to fight back against intentional distortions and attacks by an industry-funded attack machine, and the efforts of that attack machine itself. He characterizes this simply as a battle between “consensus” scientists and “skeptical” individuals, as if we’re talking about two worthy adversaries in a battle. This framing is flawed on multiple levels, not the least of which is that those he calls “skeptics” are in fact typically no such thing. There is a difference between honest skepticism — something that is not only valuable but necessary for the progress of science — and pseudo-skepticism, i.e. denialism posing as “skepticism” for the sake of obscuring, rather than clarifying, what is known.

Nate deeply mischaracterizes an editorial published by the prestigious and staid journal Nature (whose sentiments are echoed in my book The Hockey Stick and the Climate Wars) warning scientists that they “must acknowledge that they are in a street fight, and that their relationship with the media really matters.” Nate grossly mischaracterizes the quote, claiming that “the long-term goal of the street fight is to persuade the public and policy makers about the urgency (or lack thereof) of action to combat climate change.” Nate makes it sound like the “street fight” was of the scientists choosing, completely turning on its head what Nature was actually talking about: scientists finding a better way to defend science from cynical attacks whose sole aim is to confuse the public about what we actually do know about climate change (and therefore forestall any efforts to deal with it).

I could detail the numerous other problems with the chapter (and no — there aren’t really 538 of them; I confess to having taken some “poetic license” with the title of this commentary). But the real point is that this book was a lost opportunity when it comes to the topic of climate change. Nate could have applied his considerable acumen and insight to shed light on this important topic. But the result was instead a very mixed bag of otherwise useful commentary marred by needless misconceptions and inappropriately laundered denialist memes.

Don’t get me wrong. I’m still a FON (Fan Of Nate). I will continue to follow his thoughtful commentary on all matters of politics and polling. But when he makes claims about other topics, like climate change, I think I’ll be a lot more skeptical. Skepticism — real skepticism — is, after all — a good thing.

Michael Mann is Director of Penn State’s Earth System Science Center and author of ‘Dire Predictions’ and ‘The Hockey Stick and the Climate Wars.’ This piece was also published at Huffington Post and was reprinted with permission from the author.


« »

38 Responses to FiveThirtyEight: The Number of Things Nate Silver Gets Wrong About Climate Change

  1. Brad Johnson says:

    This is embarrassing. I had beers with Nate in Copenhagen during the 2009 climate talks when he was researching his book. It’s very frustrating to find out, based on Michael Mann’s review, that Nate gives credence to pseudoscientist polluter hacks like J. Scott Armstrong.

  2. Ken Barrows says:

    Why pay so much attention to someone who is focused on the horse race aspect of the Presidential race?

  3. In answer to Ken (#2), Nate is particularly thoughtful about the use of statistical analysis and probabilities, and is also usually careful to couch his conclusions with appropriate caveats. It sounds like he didn’t do this in his chapter on climate, but he’s not the first smart person to make a mess of things outside his field.

    The fundamental misunderstanding is to conflate lessons from economic forecasting tools with those related to physical systems. I address the differences between these two types of forecasts and why they matter to the climate issue in Chapter 4 of Koomey, Jonathan G. 2012. Cold Cash, Cool Climate: Science-Based Advice for Ecological Entrepreneurs. Burlingame, CA: Analytics Press. []

  4. dana1981 says:

    This is a damn shame. I’m a big fan of 538, but it sounds like Silver has treated climate science much as it was (mis)treated in Super Freakonomics.

    As Brad Johnson notes, giving any credence to Armstrong is a huge blunder. That’s obvious from this one quote alone: “I work with Willie Soon, who does a lot of research on this particular topic, and that’s what he tells me [that natural factors are causing global warming]”

    Armstrong also signed the latest denialist article in the Wall Street Journal.

    He’s a fossil fuel-funded hack.

  5. Dano says:

    Again, it is easy to identify folks who have not been educated in the natural sciences.



  6. Sy Baumel says:

    (a) climate scientist James Hansen’s famous 1998 predictions overestimated global warming (they didn’t)

    Should be 1988, of course.

  7. Joe Romm says:

    Good catch!

  8. I predict with a high degree of confidence that Nate will deeply regret what he wrote.

  9. dana1981 says:

    This is actually the one part of Mann’s article I disagree with. Technically Hansen ’88 did overpredict the ensuing warming. However, it did so because his model sensitivity was too high, and had he used today’s models in 1988, he would have accurately predicted the ensuing warming. It’s just more nuanced than whether or not his prediction was too high.

  10. Ormond Otvos says:

    I learned little from this disappointed diatribe, but I’d point out that human engineering, or applied sociology, is a lot more likely to solve the problem of human responses to global warming than cries of alarm punctuated with error bars.

    Ultimately, it’s about the application of money to psychology, and thence to the public. If you don’t realize that, your narrow expertise, so lauded in this article by dismissing anyone who hasn’t read “broad and deep” in the prescribed field, will soon be only applicable to descriptions of the ill effects of CO2.

    I never fail to marvel at the self-defeating insularity of the hard scientist. Try watching “The Big Bang Theory” for hints about passionate clueless geeks. It’s funny, and it’s true.

  11. Mulga Mumblebrain says:

    He sounds like a prototypical Chicago University libertarian Rightwing droog. I cannot imagine this type producing anything but denialist sullage, as their hideous and omnicidal religion of untrammeled greed means far more to them than the survival of humanity. We are talking of dead, dead, souls.

  12. Mulga Mumblebrain says:

    So, when it comes to the most crucial problem in human history, ‘Nate’ puts his hard Right, Chicago School, economic ideology before all else, and, from the examples given, could easily slither into a position at Fox News.

  13. Mulga Mumblebrain says:

    I predict that he will not, and that he will be well rewarded for them, and I further predict that he will become a feature in the denialist road-show.

  14. mjcc1987 says:

    2 things Mr. Mann; 1) Hasn’t his holiness, the most reverend ken cuccinelli arrested you for blasphemy yet, and 2) doesn’t Nate peer review his facts? If he doesn’t, they’re just opinions.

  15. Merrelyn Emery says:

    Social scientists such as myself agree that economics does not and cannot explain human behaviour and to equate economics with social science is mistaken. While some social science is rubbish, there is a body of knowledge carefully accumulated over generations that, with a high degree of certainty, explains and predicts behaviours such as competition or cooperation which are critical to our survival, ME

  16. tamino says:

    Comment #3 says “Nate is particularly thoughtful about the use of statistical analysis and probabilities.” Yet he gave credence to the Armstrong’s work. I see two possibilities: either he didn’t actually *read* Armstrong’s work, or he isn’t “particularly thoughtful about the use of statistical analysis and probabilities.”

  17. AlanInAZ says:

    Nate Silver takes exception to Michael Mann’s comments via twitter. It would be nice to see serious reply by Silver.

  18. Rob says:

    Oh yeah? Well, I happen to have Nate Silver right here.

    **pulls Nate Silver twitter link out from behind a nearby poster**

    (apologies to Woody Allen)

  19. Alex J says:

    Presumably this will be posted on RealClimate as well (and hopefully we’ll see a reference to this review on Amazon)? The broader the coverage, the better! Nipping these things in the bud as effectively as possible is one thing scientists and their messengers haven’t always been too effective at.

  20. Syd Baumel says:

    For me, the most salient thing about Hansen’s scenarios/estimates in 1988 when discussing Hansen’s less than 100% accuracy with deniers/”skeptics” is that if they (the deniers/”skeptics”) were right, and any warming prior to 1988 had been random and without cause, the null hypothesis to measure Hansen’s scenarios against would be a regression to the mean of the planet’s 20th century surface temperature. Just look at the NASA chart, for example, circa 1988 ( A regression to the mean null hypothesis would predict that the near-term surge in temperature since 1980 – or the longer term rise since the early 30s – was overdue for a major “correction.” By 2012, surely the planet’s temperature would have settled back down into the 20th century average of perhaps -0.2 anomaly. Instead – even though Hansen’s most applicable scenario (B) overshot by about 0.4 degrees C (based on eyeballing) – actual temps have continued to rise and are now about 0.4 degrees higher than they were in 1988 and, most importantly, nearly 1 degree higher than they should have been if they had regressed to the 20th century mean in line with the alternative/null hypothesis that global warming isn’t happening.

  21. Mulga Mumblebrain says:

    Oh, dear- it looks like ‘Nate’s’ true colours are beginning to shine through.

  22. Fred says:

    Yes. If he knew the answer he would have predicted it.

  23. Hal O'Brien says:

    You mean like asking that his arguments be represented faithfully and honestly?

    From one of the tweets in question:

    “Mann attributes me as endorsing a number of premises that the book actually refutes.”

    It would probably be prudent to hold off judging Mr. Silver’s arguments until we can all read them and judge for ourselves.

  24. Peter Mizla says:

    I posted at Silvers’ blog and was attacked vehemently as a ‘scientific elitist’. The attacks offered no scientific data to refute Dr. Mann- just that I was ‘arrogant’. Again we have a problem with disregarding evidence, while attacking the message of science which might prove as an inconvenience to others. Fascinating.

  25. The only thing Nate Silver denies is supporting the people and theories he’s accused of supporting, saying simply, that he doesn’t understand how Mann could have read the whole of the book and left with that view.

    “Mann attributes me as endorsing a number of premises that the book actually refutes.”

    It’s also worth noting that despite my antipathy for the very neo-liberal Dr. Friedman, he was in favour of Pigovian taxes on pollution. A more direct and radical approach than even the president takes. If I’d been president and wanted to enlist someone to design a carbon tax and give me cover on the right, it’d have been Milton Friedman, if he were still alive.

    (Of course, I’d also have used him to help justify a Guaranteed Annual Income, but whatever, this looks like another Team Blue slapfight, which, as a leftist who would like us to enjoy rising crop yields and not the kind of Deep Ecologist, Gaia-is-punishing-you, wet-dream that happens after two degrees centigrade of warming, makes me sad.)

  26. Suz says:

    A couple weeks ago I commented about the limitations of the forecasting model in terms of accounting for the impact of mega money from ‘Citizens United’ entering campaigns and the opposing trend toward early voting. These are the kind of challenges forecasting faces when potentially significant influences are embedded in the noise because they cannot be identified or measured.

    Unfortunately these so-called ‘significant influences’ are nothing more than figments in some ‘poobah’s’ head but clearly others emerge as signals that are forecast changers.
    Am I off here in terms of the discussion today and if so, how far off?

  27. The main take-away of the article for me is that Silver is like the guy in the maxim whose only tool is a hammer and so every problem is a nail. Silver is a dab hand at interpreting polling surveys, and so he treats AGW as if it were amenable to purely statistical analysis. It isn’t and it never will be.

    Or it could be that Silver wants to extend his little bailiwick beyond projecting elections into general punditry and is hoping to establish himself as an “honest broker”. To make money as an “honest broker” in climate, you have to be a stooge for industry.

  28. Joe Romm says:

    Sure. But I have read what Silver wrote and Mann’s critiques are legit.

  29. Mike Roddy says:

    Silver is not very bright.

  30. pinroot says:

    Thanks. That’s the first time I’ve ever seen “Chicago” and “libertarian” together in the same sentence.

  31. Michael Heath says:

    Care to share a permalink to the blog post with your comment post. I reviewed the last two Silver blog posts at 538 and didn’t see your name or the topic.

  32. Peter Mizla says:

    Peter S. Mizla
    Hartford CT


    you should keep your predictions about elections and economics.

    from climate progress; Dr. Micheal E. Mann
    Nate devotes far too much space to the highly questionable claims of a University of Pennsylvania marketing Professor named J. Scott Armstrong. Armstrong made a name for himself in denialist circles back in 2007 by denouncing climate models has having no predictive value at all. Armstrong’s arguments were fundamentally flawed, belied by a large body of primary scientific literature — with which Armstrong was apparently unfamiliar — demonstrating that climate model projections clearly do in fact out-perform naive predictions which ignore the effect of increasing greenhouse gas concentrations. As discussed in detail by my co-founder, NASA scientist Gavin Schmidt, Armstrong simply didn’t understand the science well enough to properly interpret, let alone, assess, the predictive skill of climate model predictions.

    lets keep science to the scientists Nate- you seem horribly misinformed.

  33. Brad Johnson says:

    FYI, I’ve now read the chapter. Nate mentions the beers (though not me). Dr. Mann’s assessment is essentially right. It’s not Superfreakonomics bad, but what could have been a good and interesting take on the difference between the work climate scientists do and that of weather forecasters or economic forecasters is buried under a mishmash of confusion, bad framing, false equivalence, and outright error.

  34. Mulga Mumblebrain says:

    How do you envisage these ‘rising crop yields’ being achieved? Not by genetic engineering, by any chance?

  35. Michael Heath says:

    Thanks Peter.

  36. RaflW says:

    If what Brad and Michael say is true, then both Nate and his editor failed rather badly in this chapter.
    That no one went back to the sources to even see if they were in the realm of correct is a disservice to the medium. I get that bloggers don’t usually do quote checks, and the NYT now rather tartly insists they don’t.
    But a book has a longevity that requires more care. And that’s an editors job. Ir it was before publishing became just another empire based on hucksterism.

  37. C. Leffington says:

    How important to your argument are the observations that Armstrong is associated with the fossil fuel industry and that Silver may have taken classes from Friedman and Levitt? They seem out of place in the piece, and might even be seen as unscientific red flags that shouldn’t be ignored. I think that it is a mistake to include them.

  38. Joe Romm says:

    Not important at all.