As a general rule for scientists, one shouldn’t hitch one’s wagon to long-debunked purveyors of disinformation. Because then you might end up circling the wagons with the wrong … tribe (see “The curious incident of Judith Curry with the fringe”).
I’m on a plane today, so I commend to you an outstanding Real Climate post, “The Montford Delusion,” by Tamino — and the stunning comments section. NASA scientist Gavin Schmidt and Tamino are in the role of Wyatt Earp, Doc Holliday and company. Judith Curry (and Peter Webster) have apparently thrown in with the Clantons. Like all analogies, this one isn’t perfect, but I’m afraid the outcome is pretty much the same.
Tamino deconstructs and eviscerates A. W. Montford’s book The Hockey Stick Illusion: Climategate and the Corruption of Science.
These days Curry, Chair of the School of Earth and Atmospheric Sciences at the Georgia Institute of Technology, spends a lot of her time demonizing the much-exonerated Michael Mann, repeating the long-discredited attacks on the much-vindicated Hockey Stick, praising the well-debunked Wegman report (repeatedly asserting the falsehood that it is a National Research Council report), and actually criticizing a blogger for failing to include WattsUpWithThat in his blogroll (see here and “Beef with Curry” and “My response to Dr. Judith Curry’s unconstructive essay”).
Judith Curry shows up on RC to make a couple of brief comments and then a long one, which RC replies to [in italics]. The discussion is a bit technical in parts (though most points/aconyms are covered in the original post), but worth repeating for the final blaze of gunfire at the very end:
Although I am very busy at the moment trying to complete a paper before leaving on travel, my original drive-by is admittedly insufficient, so I am taking a few moments to clarify the weaknesses in Tamino’s review. Note, this is off the top of my head, I don’t have the HSI book with me.
First, Montford’s book clarifies three weaknesses in the paleoreconstructions, from MBH 98/99 through Mann et al 08. These include problems with tree rings, the centered PCA analysis, and the R2 issue.
[Response: Really? This is it? The PCA analysis is completely moot as has been shown in the literature Wahl and Amman (2007) and von Storch et al (2005) and above. And you think this is a big issue in 2010? Please. The ‘R2’ issue similarly — the NAS Chapter 9 deals with the issues there very clearly. The basic point is that when you get to the relatively sparse networks further back, the reconstructions don’t have fidelity at the year-to-year variability. If that is something you care about (i.e. whether 1237 was warmer or cooler than 1238), then you are out of luck. If instead you are interested in whether the 13th Century was cooler than the 12th C, it’s not the right metric to be using. And finally, ‘tree rings’? A whole community is just dismissed in your mind? The community that actually pioneered community-wide data sharing in climate science? A community moreover in which the literature has openly dealt with the many issues that arise in dealing with the nature of trees and tree rings — they are the ‘problem’? Again, really?The points are even more bizarre when you actually look at the latest work that shows that reconstructions without tree rings or off-centre PCA give good reconstructions back centuries and that they aren’t grossly different to the ones using tree rings. What more do you want? — gavin]
The tree ring issue is admittedly murky, but unless the dendro community becomes more objective in its analysis, tree rings will become irrelevant. The centered PCA and R2 issues are much more straightforward. The centered PCA is bad statistics, and just because no single significance test is objectively the best in all circumstances does not mean that you can cherry pick significance tests until you find one you like and ignore R2.
[Response: This is simply insulting. You have absolutely no evidence that this was the case. The RE/CE statistics are perfectly fine at describing what the authors thought were relevant and have a long history in that field (Fritts, 1976) and as we have seen the PCA issue is moot. The idea that people went looking for ‘bad statistics’ to fix their results is without merit whatsoever. Please withdraw that claim.]
The key points of Montford’s book that Tamino ignores are:1. The high level of confidence ascribed to the hockey stick inferences in the IPCC TAR, based upon two very recent papers (MBH) that, while provocative and innovative, used new methods and found results that were counter to the prevailing views. Plus the iconic status that the hockey stick achieved in the TAR and Al Gore’s movie.
[Response: You are misreading the IPCC reports. The relevant claims in the SPM and Chapter 2 in TAR were that ‘the increase in temperature in the 20th century is likely to have been the largest of any century during the past 1,000 years. It is also likely that, in the Northern Hemisphere, the 1990s was the warmest decade and 1998 the warmest year’. “Likely” in TAR speak was 66%-90% chance, thus better than 2 in 3, but not as good as 9 in 10. Your characterisation of ‘prevailing views’ is simply wrong — the paleo community had long been aware that the medieval period had been very heterogenous (Hughes and Diaz 1994 for instance) and that the peaks did not line up in different records. ‘Likely’ was the appropriate distinction for the 20th C warming being greater than any century-scale warming in 1000 years, since there wasn’t (and isn’t) any evidence to the contrary and plenty in support. The only issue that one could reasonably have is the statement about 1998 or the 1990s. Those claims were based on the fact that 1998 was by far the warmest year in the warmest decade in the instrumental record, but without direct evidence that other very warm years in perhaps not quite as warm decades did not match or exceed it. Thus I would have been happier if that part of the statement had been downgraded to ‘more likely than not’.In AR4, the relevant statement was: Average Northern Hemisphere temperatures during the second half of the 20th century were very likely higher than during any other 50-year period in the last 500 years and likely the highest in at least the past 1,300 years.. Thus the statement for the last 500 years has been strengthened (which is appropriate given the increase in multiple lines of evidence for that period), and the longer term statement has been lengthened to 1300 years at the same level of confidence as before. Again a reasonable and supportable position. The differences are in the characterisation of the 20C rate of warming, and mainly the highlighting of a specific year in a millennial context. Instead, there is Eleven of the last twelve years (1995–2006) rank among the 12 warmest years in the instrumental record of global surface temperature (since 1850), indicating a move towards a (correct) realisation that the relative warmth of individual years are harder to assess. In toto, I do not see this as a significant downgrading of the conclusions — you may disagree, but this is not the stuff of conspiracy theories.In terms of ‘iconic’ status, showing the results in the SPM seems fair enough, but MBH are not to blame for how images get used or discussed in the media. At all times when the authors themselves were interviewed I have yet to see any statements that were not justified. And as for the AIT, the hockey stick only got a brief mention, and that was by mistake (he used the wrong panel from a Lonnie Thompson paper). This is irrelevant.]
2. The extreme difficulties that Steve McIntyre had in reproducing the MBH results. Any argument that defends these difficulties by saying that Steve McIntyre is incompetent or lacking in persistence is just plain counter to the evidence that Montford provides. Science needs to be reproducible. Period. And authors need to provide all of the data and metadata needed to reproduce the results, not just draft or incomplete datasets
[Response: Science is reproducible and this science was. Mann et al did not generate the underlying data themselves, they got it from public archives and from asking colleagues — and that was made public when the previously unpublished work was published. Wahl and Ammann replicated the code (as did McIntyre). There were minor errors in the data listing at Nature, but that was fixed when it was pointed out. Scientists are not obligated to hand-hold people trying to reproduce their results, especially when they have already gone public with a farrago of misstatements in non-peer-reviewed papers (try actually reading MM2003). However, you are making a big error in characterising the culture that existed in 1998. I guarantee I will not find complete public archives for every climate paper that appeared in Nature that year — are none of those papers ‘science’? Nonsense. Replication is not about repetition- it’s about finding new ways to address the same problem. Two ice cores are better than two teams measuring the same one.]
3. The NAS North et al. report found that the MBH conclusions and “likely” and “very likely” conclusions in the IPCC TAR report were unsupported at that those confidence levels. How the hockey team interpreted the North NAS report as vindicating MBH, seems strange indeed.
[Response: This is simply not true. There are no ‘very likely’ conclusions in the relevant sections of TAR (I quoted them above). The only thing they pointed out was in regards to the relative warmth of 1998 and the 1990s in the millennial context which I agree with. They did state with a ‘high level of confidence that global mean surface temperature was higher during the last few decades of the 20th century than during any comparable period during the preceding four centuries’ — this is equivalent to the strengthening of the statements made in AR4 concerning the last 500 years. They went on to say that the ‘committee finds it plausible that the Northern Hemisphere was warmer during the last few decades of the 20th century than during any comparable period over the preceding millennium’ — and in further questions, clarified that plausible was equivalent to ‘likely’ in IPCC-speak (i.e. less confidence than the statement about the last 500 years). The statement about 1998 and the 1990s was that “Even less confidence can be placed in the original conclusions by Mann et al. (1999) that “the 1990s are likely the warmest decade, and 1998 the warmest year, in at least a millennium” because the uncertainties inherent in temperature reconstructions for individual years and decades are larger than those for longer time periods and because not all of the available proxies record temperature information on such short timescales” which is true enough. Of course, now it is likely that the 2000s were the warmest decade.]
4. A direct consequence of the North NAS report is that the conclusions in the IPCC AR4 essentially retracted much of what was in the IPCC TAR regarding the paleo reconstructions. This is the only instance that I know of where the IPCC has reduced a confidence level or simply left out a conclusion that was in a previous IPCC report. This is discussed in the CRU emails.
[Response: Again, this is not true. AR4 did in no way ‘essentially retract’ much of what said in TAR — for anything substantial concerning the nature of late 20th C warmth the conclusions both in the NAS report and the AR4 report strengthened the TAR conclusions (see the statements above). Perhaps you think that the ‘essential’ thing is the position of 1998 as the single warmest year? Well, in that case I strongly disagree, this is not ‘essential’ in anything very much. Much more important in actually understanding the climate are the relations between forcings and responses both globally and spatially over this period, and none of that relies on rankings of individual years. And as for IPCC changing conclusions this has happened many times — Lindzen used to point to statements about upper tropospheric water vapour for instance that became less confident from the 1990, 1995 and 2001 reports, similarly uncertainty in aerosol indirect effects has clearly grown over time. ]
5. Even with this drawback in the AR4 conclusions and confidence level, somehow what was left was judged to hinge on the unpublished Wahl/Amman papers, one of which was having difficulty surviving peer review in GRL for a period of several years, and was finally pushed through quickly by Steve Schneider in Climate Change. IPCC deadlines were violated, and peer review in the context of the papers publication in Climate Change was a joke (all of this is described in the CRU emails). So all of these shenanigans to get these papers into the IPCC, papers that some have judged to have more methodological problems than the original MBH papers, have seriously degraded trust in the IPCC consensus, once this was illuminated in the CRU emails.
[Response: This is nonsense. The conclusions in the Wahl and Amman papers, and their published code had been public since 2005 so there was no doubt about their results. Steve Schneider was exceptional in many ways, but his journal is not the speediest in terms of turnaround of manuscripts. Weird editorial decisions with respect to the responses to the MM05 GRL paper also did not help. But the authors of the IPCC chapter knew full well that the their statement in the first draft about MM05 was not right — there weren’t any unanswered questions about the impact of PCA centering on the results of MBH98. The WA07 paper was accepted in time for this to be cited (and it was an IPCC-wide decision to decide on the cutoffs, not Keith Briffa’s) and it was (no IPCC deadlines were violated). If it hadn’t been it would not have been the end of the world and I don’t see how anything subsequently would have changed. McIntyre has had 5 years to write a comment or a new paper on the subject and he hasn’t. As for Briffa talking to Wahl during the final drafting stage, I see nothing problematic with that in the slightest. The idea put forward by McIntyre and Montford that IPCC authors are supposed to sit in purdah while writing the reports has absolutely no basis in fact or in practice. Many people were talked to and many people made suggestions where their expertise was required. The fact is that the AR4 statements in the final version were more correct than in the first draft and that is something people should be happy about.]
6. The dependence of the various proxy reconstructions used in the AR4 on essentially the same datasets is described, it is difficult to judge these reconstructions as independent.
[Response: Long well-resolved paleo records are rare — I doubt that is a surprise to anyone. Should people not use what has been published to get the best characterisation of past climate change? Methods can be independent though, and since your earlier comments seem to revolve around methods, I don’t quite get what point you are making.]
7. The Mann et al. 2008, which purports to address all the issues raised by MM and produce a range of different reconstructions using different methodologies, still do not include a single reconstruction that is free of questioned tree rings and centered PCA.
[Response: Absolutely untrue in all respects. No, really, have you even read these papers? There is no PCA data reduction step used in that paper at all. And this figure shows the difference between reconstructions without any tree ring data (dark and light blue) compared to the full reconstruction (black). (This is a modified figure from the SI in Mann et al (2008) to show the impact of removing 7 questionable proxies and tree ring data together). In addition, there are many papers that deal with issues raised by MM — Huybers (2005), von Storch et al, (2005), Rutherford et al (2005), Wahl and Amman (2007), Amman and Wahl (2007), Berger (2006) etc.Judith, I implore you to do some work for yourself instead of just repeating things you read in blogs. (Hint, not everything on the Internet is reliable). ]
8. The divergence problem is clearly explained, including how the graphs in the IPCC report were misleading, and how the splicing of the historical records with the paleo records is misleading. I.e., the trick to hide the decline. Why should we have confidence in paleoproxies that show a temperate decrease in recent decades, in contrast to historical measurements?
[Response: The divergence problem is well known. And I absolutely disagree that the IPCC graphs are ‘misleading’. How perchance were you misled? The picture on the 1999 WMO report cover has nothing to do with IPCC, and frankly was completely unknown until November last year. Yet an incomplete caption on a report that no-one knew about is the biggest scandal in climate science? Get real. I’m with Muir Russell on this one. There is nothing wrong per se in splicing records together to get a continuous series — for instance I have just done the exact same thing in creating a series of solar forcing functions for climate model runs — but these things should be clearly explained. The divergence issue is predominantly an issue for the tree ring density measurements (Briffa et al), and while there is some reason to think that is a unique phenomena, it remains unresolved. So, feel free to ignore the Briffa et al curve if you want. This is not a general issue and doesn’t affect the MBH and Mann et al 2008 conclusions at all. ]
9. Finally, Montford asks the question as to why the scientists and the IPCC promoted the hockey stick at such a high confidence level so prematurely, and why such extraordinary efforts were made to defend it when it arguably isn’t a critical piece of the climate puzzle, rather than to learn from outside statisticians and do a credible error analysis on the data and the inferences.
[Response: Oh please. Why didn’t the first multi-proxy paper deal with all issues and try all methodologies and come to all the conclusions? Because that is not the way science works. People try new things, issues arise, issues are dealt with and a more sophisticated understanding emerges. Some data is used, more data is gathered and more complete pictures arise. No single paper is ever perfect — and I’m sure if any of your papers (or mine for that matter) got the attention that has been payed to MBH98 there’d be all sorts of potential issues as well. But you are again overstating the conclusions of those early papers, and there have been no extraordinary efforts to defend them. It is quite the contrary, there have been many and multiple extraordinary attempts to discredit them (unless you think Congressional review is ‘ordinary’). No-one is against efforts to learn from outside statisticians, that is just a strawman. People are against politically-driven hack jobs purporting to be analyses but that don’t even bother to work out what the consequences of any different choices might be. All of the data in Mann et al (2008) is online, as is all the code — where are the outside statisticians who are clamouring to have their ideas heard? They are welcome to try and do a better job.]
I’ve probably missed a few things, but those are the key points raised in the book that have stuck with me. I’ve tried to follow the debate by reading the journal articles and posts at both RC and CA. I was very frustrated in trying to sort all this out. Montford’s book sorted everything out into coherent, well argued and well documented arguments. There is a certain element of spin, so I wanted to see what RC had to say about all this. On the RC side, we have the outdated Dummies Guide to the Hockey Stick and Tamino’s review, plus the snarky replies to serious posters that include statistician Jean S. You need to do better than this to counter Montford’s book. Failing to do so will just push more people into the Montford/McIntyre corner of the ring. And how and why this issue has become so contentious and stayed so contentious is a serious issue in the field of climate science.
[Response: The reason this has become ‘contentious’ has nothing to do the MBH and everything to do with people not wanting climate change to be a problem. Icons that arise for whatever reason attract iconoclasts. Noise in the blogosphere does not correlate to seriousness in climate science. As your comments make abundantly clear, you have very little knowledge on this issue and have done no independent investigation of the wild claims being made. Yet the more smoke there is, the more you appear to want to blame MBH for the fire. A ‘certain amount of spin’? Seeing conspiracies everywhere you look is not ‘spin’, it is paranoia. Real scientific controversies get resolved in the literature for the people who actually care about getting things right. For those that don’t, continued repetition of long debunked talking points seems to be their only tactic. I, for one, am pretty tired of that and heartily bored of pointing them out.The fact of the matter is that we are far beyond the point where people need to either s*** or get off the pot. Continuing to whine about what selection rules were used in a PCA analysis 12 years ago without coming up with any constructive alternative, continuing to complain about a centering convention that makes no difference whatsoever, continuing to moan about error analyses being inadequate without doing a single stitch of work to improve them… enough, already! Science moves forward because people do actual work. Nothing happens when people just sit in a room and  complain about the state the world. The people who are actually publishing in this field are doing all of the things you seem to think are being ignored, while the people whose work you are reading are doing nothing but complain about how they are being ignored. I’m very confident about which group will make the most progress in future. — gavin]
That would be all tragic enough, but Curry then offers this final misfire:
Gavin, the post I made in #167 was a summary of Montford’s book as closely as I can remember it, sort of a review. I did not particularly bring in my personal opinions into this, other than the framing of montford’s points. So asking me to retract a point made in a book in a review of that book is, well, pointless. your attempt to rebut my points are full of logical fallacies and arguing at points i didn’t make. As a result, Montford’s theses look even more convincing. Once you’e in a hole, you can try to climb out or keep digging. Well keep digging, Gavin. My final words: read the book.
[Response: Thanks for passing by. In future I will simply assume you are a conduit for untrue statements rather than their originator. And if we are offering advice, might I suggest that you actually engage your critical faculties before demanding that others waste their time rebutting nonsense. I, for one, have much better things to do. — gavin]
Many CP regulars commented at RC and I hope they will elaborate on their points in the comments here.
[And yes, I’m aware someone else notable jumped the shark in the RC comments. I’ll address that Monday.]