Proponents of large energy-efficiency rebound effects fail to prove their case.Advocates of the thesis that “rebound” effects will offset much, most, all, or more than all energy savings from increasing end-use efficiency””a thesis popularized by David Owen’s recent and controversial New Yorker article””were asked in an early-2011 email exchange to illustrate their proposed rebound mechanisms with a hypothetical numerical example. Jesse Jenkins from the Breakthrough Institute obliged them. Jim Sweeney (Stanford) and Amory Lovins (Rocky Mountain Institute) then pointed out specific apparent errors whose correction would reduce Jenkins’s calculated rebound by about 10–20-fold (to a few percent, consistent with their own estimates). Further, the macroeconomic effects that Jenkins and his fellow-advocates had claimed were very large turned out in his example to be very small. Yet neither Jenkins nor his co-proponents rebutted the Sweeney and Lovins critiques. Jenkins now wants to abandon rather than uphold his own example, and big-rebound proponents appear to have withdrawn from the conversation. They insist that their economic calculations prove they’re right, no further proof is required, and the effects they posit are too complex for a numerical example to reflect. This behavior invites the inference that they won’t defend their sweeping claims because they can’t, and that inference will strengthen so long as they fail to do so. The exchange upholds the strength of the scientific process in clarifying understanding and exposing error, although it remains to be seen whether this goal is shared equally by both sides of the conversation. Asked for comment, Lovins quoted Harvard biology professor E.O. Wilson: “Sometimes a c oncept is baffling, not because it is profound but because it is wrong.”
That’s the conclusion Jon Koomey says journalists might well draw from a for-the-record email conversation between The Breakthrough Institute and leading energy experts. I repost his entire 8-page discussion below. It makes for fascinating reading and reveals better than anything I’ve seen just how TBI operates.
Koomey, ever the scientist, even has an “abstract,” which reads:
An e-mail conversation about whether “rebound” effects that offset energy savings are big or small reached a critical stage when a numerical example meant to demonstrate big rebounds came under decisive technical criticism””and wasn’t defended.
Recently, the Breakthrough Institute launched a major attack on energy efficiency. They used talking points that right-wing think tanks have pushed for years (see The intellectual bankruptcy of conservatism: Heritage even opposes energy efficiency). This shouldn’t be terribly surprising to longtime followers of TBI. After all, last year they partnered with a right-wing think tank, the American Enterprise Institute, to push right-wing energy myths and attack the most basic of clean energy policies, a clean energy standard.
This year, Breakthrough’s attacks on clean energy were used by the Republican National Committee as part of their overall attack on Obama’s clean energy agenda. Again, not a big surprise. TBI’s work is consistently cited by those who want to attack environmentalists and climate scientists, “George Will embraces the anti-environmentalism “” and anti-environment “” message of The Breakthrough Institute.”
Yes, I know, The Breakthrough Institute will insist it’s purely a coincidence that they are the darling of the anti-science, pro-pollution right-wing disinformers. The fact that they push right wing myths and even partner with right-wing organizations to push those myths has nothing to do with it. Nor does the fact that they spent the past two years dedicating the resources of their organization to help kill prospects for climate and clean energy action — and to spread disinformation about Obama, Gore, Congressional leaders, Waxman and Markey, leading climate scientists, Al Gore again, the entire environmental community and anyone else trying to end our status quo energy policies (see “Debunking Breakthrough Institute’s attacks on Obama, Gore, Waxman, top climate scientists, progressives, and environmentalists”). Nor does the fact that they even attacked Rachel Carson, who died decades ago after helping launch the modern environmental movement!
The only reason I point this out is that the only reason the media pay any attention whatsoever to their endless shoddy analyses and misrepresentations and smears is that Breakthrough has tried to create the impression they are a progressive, environmental organization dedicated to promoting clean energy — so that when they launch their umpteenth attack on progressives and environmentalists and climate scientists and clean energy they can be seen as “contrarians.” Stop the presses — here’s an environmental group saying environmentalists are doing the wrong thing. No, please, stop the presses already.
Anyway, some in the media have started to see through this shtick. For an excellent debunking by the media of a typically flawed TBI analysis attacking the clean energy bill, see Markey spokesman: “The Breakthrough Institute seems to believe, much as the Bush administration did, that technology will solve all, even without a market.”
After multiple debunkings in 2008 and 2009, I have mostly ignored the nonstop disinformation coming from TBI founders Ted Nordhaus and Michael Shellenberger, as well as TBI’s Jesse Jenkins, for as long as possible.
But the confusion they are attempting to spread on energy efficiency is just too much to ignore. I have had the chance to work with many of the leading experts on energy efficiency since my time overseeing the DOE’s office of energy efficiency and renewable energy back in the mid-1990s. Back in September, I asked Evan Mills, a leading scientist and efficiency expert at Lawrence Berkeley National Laboratory, to debunk some directly related myths — see (Efficiency lives “” the rebound effect, not so much). And in the coming weeks I will be doing more posts on the subject.
Today, however, we’re luck to have a detailed discussion by Jonathan Koomey, one of the world’s foremost authorities on energy efficiency. Koomey, a longtime friend and colleague who worked as a researcher and scientist at Lawrence Berkeley National Laboratory for more than two decades, is now a Consulting Professor at Stanford University.For the record, I received the document and the link from an interested third party, not Koomey.EVERYTHING THAT FOLLOWS IS BY JON KOOMEYIn the interests of space and readability, I’m not going to indent it. Footnotes (#) are at the end.LEARNING FROM AN EXAMPLE OF THE REBOUND EFFECT
by Jonathan KoomeyConsulting Professor, Stanford University
February 13, 2011
An e-mail conversation about whether “rebound” effects that offset energy savings are big or small reached a critical stage when a numerical example meant to demonstrate big rebounds came under decisive technical criticism””and wasn’t defended.
In an effort to avoid misunderstandings about the complex phenomenon of rebound, I proposed to an email list of about 30 analysts and energy/environmental reporters that those supporting large rebound effects produce a simplified example, so we could dig into the buried assumptions that always afflict such analyses.
Jesse Jenkins (of the Breakthrough Institute) offered such an example, but failed to respond to substantive critiques of the assumptions behind that model that reduced the calculated rebound effect by factors of roughly 10 to 20.
Nevertheless, there was general agreement that the relevant research question should be “under what conditions is rebound a problem, and in those cases, how big is it?” Once this question is accepted, however, making blanket categorical statements like “energy efficiency never saves energy” (as blog posts on the Breakthrough Institute’s web site routinely do) is no longer appropriate.
The normal burden of proof is on those advocating the existence of some unexpected and novel effect to show the underlying causal mechanisms that lead to that result, so the assumptions can be peer reviewed. I can’t prove that large rebounds don’t exist, just like I can’t prove that black swans don’t exist in the absence of a perfectly accurate universal census of swan colors, but if someone brings me a black swan, the problem is solved. And that’s what those of us skeptical about large rebound effects continue to request: bring us a black swan!
Our story so far
After Jesse Jenkins’ reply to my email of Friday February 4, 2011 I realized that it’s time to step back and summarize what we’ve learned from this intensive discussion, whose 30-odd participants posted over a hundred contributions in the last week of January 2011. I initially asked those supporting the existence of big rebound effects to lay out a specific example so we could avoid the usual definitional problems that afflict discussions about any complex phenomenon. The lessons that emerged from this process give insights into the meaning and importance of scientific integrity. They also illuminate claims about whether energy efficiency can contribute to solving global climate change, so this debateis deeply consequential.
This particular part of the thread began with my posing a logical conundrum, which I expressed in an email addressed to Harry Saunders on January 25, 2011:
I don’t understand the following statement in your email below:
‘the backfire comes from energy efficiency gains enabling producers to reconfigure new productive capacity to use more energy profitably.’
I don’t see how this can be true except in special cases where energy is a huge fraction of production costs. One such special case may be data centers (a sector with which I have more than passing familiarity) but there aren’t many others that fall into the same category. For most companies energy is a small (single digit) percentage of their total costs, so I don’t see how energy efficiency gains on their own can have more than a modest effect on their productive capacity.
It is this puzzle that we need to resolve. One the one hand, energy is a tiny part of total costs for most consumers and productive parts of the economy. On the other hand, the rebound advocates claim that reducing these tiny costs somehow generates additional activity leading to more energy use, offsetting the initial energy savings.
Harry and Ted Nordhaus both reacted to my request for evidence with a reiteration that the econometric and modeling studies are sufficient and that no further evidence need be produced to demonstrate that rebounds are large. As Amory Lovins subsequently pointed out, this inverts the usual burden of proof. It’s impossible for me to prove, for example, that a black swan does not exist in the absence of a perfectly accurate universal census of swan colors. It’s incumbent on one who claims black swans do exist to produce one, otherwise it is just speculation, even if based on complex calculations and models.
So I asked for a specific example (“bring me a black swan!”) and Amory seconded that request. The request was met with resistance most explicitly by Ted, but also to some extent by Harry. They said that it was complicated and difficult to create such simple examples, and that those arguing against rebounds were focusing on micro effects, when the real action, in their view, was at the macro level. For example, Ted on January 26, 2011 said “Amory and Jon know perfectly well that there is no ‘simple’ way to track the complex and manifold processes of macro-rebound. What you end up with is a very elaborate model.” Ted also responded to Amory’s reiteration of the need for a simple calculation by saying
“Getting even heuristic clarity requires making all sorts of assumptions abou tproduction functions and elasticities of demand and substitution. I think you know this perfectly well Amory. What you are really asking is that we bound the analysis in all sorts of ways that make it not an analysis of macroeconomic rebound at all.”
Waving the flag of complexity is really a distraction. One can abstract from the literature plausible ranges for various parameters and carry through the simplified calculation using those ranges. If the factors causing large rebounds are clearly understood, then they can be explained in such a simplified calculation. If someone can’t explain those factors, they don’t really understand the phenomenon.
What I and other skeptics of large rebound also suspect is that the results of the complex models projecting large rebound effects are driven by deeply buried assumptions and divergence between reality and how the models represent that reality.(1) Real firms and consumers almost invariably do not behave like the economic models predict, due to cognitive limitations, information asymmetries, split incentives, principal- agent problems, transaction costs, increasing returns to scale, and other issues that are almost never included in those models.(2)
In my experience(3) an excellent way to diagnose such modeling problems is by examining a specific example. This is why I’ve insisted that advocates for large rebound can’t just point to the previous literature or to modeling results to prove their case””they need to demonstrate that they understand the causal mechanisms and the embedded assumptions so we can think them through together and determine if we all agree. Working through an example is the best means to that end.
To spur the discussion, Jim Sweeney, who is one the world’s top energy economists, on January 26, 2011 produced his illustrative example, and explained why he thought the rebound effect was real but small in most cases. He pointed out the key role that assumptions about the cost of efficiency play in the calculation, because that cost must be subtracted from spending elsewhere in the economy and so offset (at least to some degree) the stimulating effect of the savings. He also asked again for specific examples illustrating the claims Ted and Harry made. Later that same day Jesse rose to the challenge and provided another example showing rebounds greater than 50% (i.e., more than half of the energy savings would be taken back in his example).
Interestingly, Ted agreed (on January 27, 2011) with my problem statement for this exercise, but still did not participate in analyzing the specific example itself:
I want to second Jonathan’s observation that ‘the question is better phrased as “under what conditions is rebound a problem, and in those cases, how big is it?”’
He proceeded to give a useful summary of where he thought the debate then rested:
In response to both David’s article and Harry’s blog, a number of folks on this list suggested, with a great deal of certitude, not only that backfire wasn’t proven but that rebound is “insignificant” and this has sparked a rather wide ranging debate about rebound that I think has generally arrived at some agreement that in many sectors it is probably quite substantial””we can quibble about whether it is 30% or 50% or 70% but whatever that number is, in most sectors it is probably not 5%. I am happy to concede that backfire has not been “proven,” and suspect that it may not ultimately be provable, at least through standard quantitative methods. But it is also problematic that Amory and others have for so long dismissed the phenomena as non-existent(4) or insignificant and in so doing suggested that we could expect linear reductions, or something close to it, in energy use and emissions from engineering estimates of energy savings from technical efficiency.
This is much more nuanced statement than the one he made the day before (or the ones that appear on the Breakthrough Institute web site):
Once we dispense with the canard that there is a free lunch here””that significant reductions in energy use and emissions can be achieved with net economic benefit or very low cost through technical efficiency improvements””we can get on with the business of developing realistic climate mitigation scenarios.
These statements do represent progress, however, particularly the agreement on the problem statement.
So far, so good. Science thrives on debate about such specific examples, where everything is clearly defined and laid out for all to see. Jim and Amory then went through Jesse’s example and identified empirical and methodological errors that, when corrected as per Amory’s email of January 27, 2011, reduced the rebound effect to a few percent, which is consistent with what those arguing against large rebounds have been saying all along: the effect is small except in a few special cases. Amory also pointed out that the macro effects were a relatively small contributor to the overall effect in the example created by Jesse, which is odd because Ted and Harry said at various points that the macro effects are the most important ones, and we hoped that Jesse’s example would illuminate why they are big and how they work.
Then things started to run off the rails. Instead of digging into Amory’s specific criticisms, rebound proponents went silent for about a week, with the exception of Harry, who mused on January 29, 2011 about four potential ways to achieve the same end as analyzing a concrete example without actually analyzing a concrete example. He made a telling comment at the end of this email: “I hope it is not assumed that I am running away from the numerical examples challenge; and that the above is deemed a reasonable shot at attaining the same end.” It seemed strange to me at the time that someone would write an email proposing ways not to (or reasons why not to) create a concrete example when there was already a concrete example under discussion, awaiting his comment.
Harry closed with the following sentence: “I re-state my position: rebound is real, is measurable, and is significant. And to that I will add: concrete and physically comprehensible.” Things that are “physically comprehensible” can be explained in a simplified calculation, so I was again puzzled by his reticence about the specific example.
After a week of radio silence about Amory and Jim’s critiques, I sent an email (on February 4, 2011) prompting the group to respond so we could continue our joint learning process. Jesse replied that same day””I reproduce his email below in its entirety:
You’ve caught me just as I’m preparing to leave for a week’s long vacation, so I’m afraid won’t be able to write a thorough reply to Amory’s critique.
My brief response however is this: you asked for a simple but specific numerical example of how rebound works. I provided one, expressly in generic terms, with numbers that were roughly appropriate for an industrial sector, but clearly not drawn from specific analysis of the example sector I (perhaps foolishly) decided to specify (e.g. I decided to use a chemicals plant as our example, since it could help us visualize what I was talking about more concretely, when perhaps it should have just been left as a generic industrial facility). Amory then proceeded to provide critique of the specific figures I utilized in this generic example, based on very specific experiences he has from clients within the chemical sector.
I didn’t reply with a detailed rebuttal, because the point of the exercise was not to provide a concrete and rigorously accurate description of rebound at an actual chemicals plants, but rather to provide a clear sketch of the mechanisms at work, and how they can together sum to quite significant rebound.
I provided this example because you and others had decided to completely ignore a whole set of published academic research that does provide far more rigorous calculations and depictions of rebounds at work in various industrial and end-use contexts.
Suffice it to say, I believe having clearly shown how rebound mechanisms operate in plain terms, providing you with the asked-for explanation of the casual mechanisms at work, the realm of debate should center now not on Amory’s rebuttal to my straw example, but on your as of yet absent rebuttal of any of the specific academic research on the topic. In other words, let’s not ignore the dozens of papers on the subject at hand, as we instead focus on picking nits over a straw example provided for illustrative purposes only.
Clearly Jesse misunderstood the purpose of the simple example, which was to facilitate dialogue between people whose terminology and analytical conventions differ. The only way this technique works is if there is a discussion of the validity of each empirical assumption and the structure of the calculation. That’s why Amory and Jim went point by point through the example to analyze those assumptions. Most of Amory’s comments were methodological and did not depend on his specific experiences in chemical plants. He and Jim variously pointed out apparent double-counting, a fourfold-high estimate of energy intensity (in one of the most energy-intensive manufacturing sectors), the blanket assumption of zero cost for efficiency investments, and other specific issues with Jesse’s proposed mechanisms and summation.
Instead of addressing each point of Amory and Jim’s critique with technical corrections and commentary, Jesse chose to dismiss those concerns as “nits”, which I don’t think is a fair description for a correction factor of roughly 10 to 20. He also said:
I didn’t reply with a detailed rebuttal, because the point of the exercise was not to provide a concrete and rigorously accurate description of rebound at an actual chemical plant, but rather to provide a clear sketch of the mechanisms at work, and how they can together sum to quite significant rebound.
The distinction between these two concepts eludes me. Clearly a “clear sketch of the mechanisms at work” requires a “concrete and rigorously accurate description of rebound at an actual [insert chosen industry here] plant”. And if one can’t create “a concrete and rigorously accurate description of rebound at an actual plant.” Why should we believe claims about the effect of rebound on the broader economy?
Jesse complains again (as he and Ted and Harry have done previously) that we’re ignoring all the published research, but this deflects attention from the real issue. He and his colleagues claim to be the experts in big rebound effects, but are unable or unwilling to produce and defend a specific example that accurately represents the results of that research, and that is the crux of the matter. At various points in this discussion, advocates for rebound have lectured us about the complexity of their macroeconomic calculations but failed to produce a coherent explanation of the relevant factors leading those calculations to their results. If these models are so sophisticated and clearly structured, then the driving factors ought to be explainable to other analysts, but we still await that explanation.
When someone creates an example and publishes it (in this case in an email trail), scientific etiquette dictates that the creator address serious and substantive critiques, owning up to errors where they arise and defending their facts and arguments as the evidence dictates. But on the substantive critiques of the specific example, Jesse and his colleagues remain conspicuously silent. Jesse’s reply above says, in essence, that the details don’t matter. But they do matter, both procedurally and substantively. That’s why I requested the example, and why I initially thought he provided it.
For scientific knowledge to progress, claims by competing experts need to be weighed by different peer groups as a check on groupthink. There are myriad examples where like thinking researchers go down blind alleys because of their unexamined prior assumptions. I think that this is what may be happening with rebound proponents in this discussion, although we won’t know for sure until we’ve dug more into this specific example and the models supporting the various claims.
I’ve been able to identify three distinct (high-level) types of rebound in our discussion thread to date:
1) Micro rebounds, which are zero (for those end-uses afflicted by the principal agent problem or otherwise not subject to user control), very small (a few percent) for many end-uses, or modest (10–30%) for a few special cases like space heating and personal automobiles.
2) Macro rebounds associated with re-spending of money saved by efficiency, which are capped at 6–8% of net savings (after counting the cost of efficiency) on average because that’s the percentage of GDP associated with energy
3) Macro rebounds associated with other substitution effects in economically productive sectors of the economy.(5) This is the term I’m having the most trouble understanding, given that energy is generally a very small part of the costs for most businesses, and this is the one that is least well explained by proponents of this phenomenon.
There’s also a separate discussion on the popular interpretation of rebound confusing increasing wealth, technology changes, and other trends with the actual rebound effect, but that has more to do with the New Yorker article and the Breakthrough Institute’s blog postings than with Harry’s academic paper, and we can put that aside for now.
I think one focus should now be on understanding macro rebounds of the second type, because those seem to be the ones most in dispute. I ask now (again) that rebound proponents bring forth a black swan (i.e. a simplified clearly and fully explained example illustrating rebounds, where cash flows to different parts of the economy are identified). Once you do, we’ll of course examine it closely and make sure it really is a black swan and not a white one with dyed feathers, but after such an examination we’ll say “We were wrong. Black swans exist” or we’ll tell you why we’re not yet convinced, and the discussions will continue until we reach agreement. That’s how this process is supposed to work, and I hope we can make it work in this instance, because this issue is an important one that affects society’s judgments about the costs and potential for reducing greenhouse gas emissions in coming decades.
I would also ask rebound advocates to cease writing blog posts with misleading categorical statements like “Why Energy Efficiency Does not Decrease Energy Consumption.” If we agree on the above problem statement, the question simply cannot be framed in that way. Efficiency may save less than we estimate in some cases, and we all agree that we need to identify and quantify those cases. By continuing to make blanket statements like “efficiency never saves energy” or the idea that “significant reductions in energy use and emissions can be achieved with net economic benefit or very low cost through technical efficiency improvements” is a “canard”, the policy debate is sidetracked and people without the technical background to evaluate those claims jump to erroneous conclusions (as many did after reading the New Yorker article).
We owe it to science, ourselves, and the public””some of whom will presumably hear about our conversation from the prominent energy/environment journalists present””to persevere until we reach mutual clarity and understanding. Otherwise those journalists might draw a conclusion something like this:
Proponents of large energy-efficiency rebound effects fail to prove their case.Advocates of the thesis that “rebound” effects will offset much, most, all, or more than all energy savings from increasing end-use efficiency””a thesis popularized by DavidOwen’s recent and controversial New Yorker article””were asked in an early-2011 email exchange to illustrate their proposed rebound mechanisms with a hypothetical numerical example. Jesse Jenkins from the Breakthrough Institute obliged them. Jim Sweeney (Stanford) and Amory Lovins (Rocky Mountain Institute) then pointed out specific apparent errors whose correction would reduce Jenkins’s calculated rebound by about 10–20-fold (to a few percent, consistent with their own estimates). Further, the macroeconomic effects that Jenkins and his fellow-advocates had claimed were very large turned out in his example to be very small. Yet neither Jenkins nor his co-proponents rebutted the Sweeney and Lovins critiques. Jenkins now wants to abandon rather than uphold his own example, and big-rebound proponents appear to have withdrawn from the conversation. They insist that their economic calculations prove they’re right, no further proof is required, and the effects they posit are too complex for a numerical example to reflect. This behavior invites the inference that they won’t defend their sweeping claims because they can’t, and that inference will strengthen so long as they fail to do so. The exchange upholds the strength of the scientific process in clarifying understanding and exposing error, although it remains to be seen whether this goal is shared equally by both sides of the conversation. Asked for comment, Lovins quoted Harvard biology professor E.O. Wilson: “Sometimes a concept is baffling, not because it is profound but because it is wrong.”
Presumably that’s not the sort of conclusion that Ted, Jesse, Harry, and other big-rebound proponents had in mind when they launched this conversation. The remedy is obvious: rejoin it, address Jim’s and Amory’s specific critiques, and show us where they’re wrong. We’d all love to learn.
— — — — —
1 An example of such an embedded assumption is the mapping of Dale Jorgensen’s economic data to energy-using sectors in Harry’s latest paper (here), which Lee Schipper and Skip Laitner raised as problematic for that analysis””Harry promised to investigate. That paper, which is still a working paper and has not been subject to detailed peer review, is the one that rebound proponents have most commonly been citing in this debate as containing the best evidence for rebound, but this unexamined assumption about economic sector coverage is one that may have a substantial effect on the results. There are of course many other such unexamined assumptions in any economic modeling exercise, and to truly understand the results, those assumptions must be unearthed and scrutinized. That’s the purpose of the simple calculation.
2 For details, see the excellent book by Stephen DeCanio 2003. Economic Models of Climate Change: A Critique. Basingstoke, UK: Palgrave-Macmillan.
3 For those who don’t know me, I have been working with economic models for more than two decades and was the founder of Lawrence Berkeley National Laboratory’s Energy Forecasting group. I also led that group for almost a dozen years. For more details, go to here.
4 Amory Lovins, who has been following and occasionally contributing to the rebound literature for several decades, doesn’t think he’s ever claimed rebound doesn’t exist””just that it’s generally small.
5 Jim, Amory, and others view this as mostly a micro effect (internal to firms), so there’s a terminological and definitional issue here.