Hey, people are doing the multiverse!

Elio Campitelli writes:

I’ve just saw this image in a paper discussing the weight of evidence for a “hiatus” in the global warming signal and immediately thought of the garden of forking paths.

From the paper:

Tree representation of choices to represent and test pause-periods. The ‘pause’ is defined as either no-trend or a slow-trend. The trends can be measured as ‘broken’ or ‘continuous’ trends. The data used to assess the trends can come from HadCRUT, GISTEMP, or other datasets. The bottom branch represents the use of ‘historical’ versions of the datasets as they existed, or contemporary versions providing full dataset ‘hindsight’. The colour coded circles at the bottom of the tree indicate our assessment of the level of evidence (fair, weak, little or no) for the tests undertaken for each set of choices in the tree. The ‘year’ rows are for assessments undertaken at each year in time.

Thus, descending the tree in the figure, a typical researcher makes choices (explicitly or implicitly) about how to define the ‘pause’ (no-trend or slow-trend), how to model the pause-interval (as broken or continuous trends), which (and how many) datasets to use (HadCRUT, GISTEMP, Other), and what versions to use for the data with what foresight about corrections to the data (historical, hindsight). For example, a researcher who chose to define the ‘pause’ as no-trend and selected isolated intervals to test trends (broken trends) using HadCRUT3 data would be following the left-most branches of the tree.

Actually, it’s the multiverse.

24 thoughts on “Hey, people are doing the multiverse!

  1. Strange paper. Did the models developed prior predict what happened in those years or not? I assume not since there would be no discussion otherwise.

    • I think it depends on what you mean by “predict”. For about 15 years the temperature trend moved from the middle of the error range of the forecast to lower part of the forecast envelope – but that’s somewhat depending on which model you choose and which temp set you choose. Link below is to realclimate model performance page.

      So the question is: was there a trend change? Which skeptics say yes and surely realclimate says no, just noise.

      http://www.realclimate.org/index.php/climate-model-projections-compared-to-observations/

      • I’ve always thought it strange that climate scientists talk about the observed data being within the error range of the forecast as opposed to talking about some sort of cumulative forecast error relative to that of a naive approach, as I see in other fields–most notably, near-term weather forecasting. Can these models beat a 30 year trendline?

        I’ve even seen discussions of whether the models are “rejected” at the 95% level, which they appear to mean is the case where observations are outside the 95% confidence interval of the model. This seems backwards, assuming you buy into frequentist hypothesis testing at all.

        In any case, if you want to claim your forecast is “right” when your 15-year dispersion is a factor of 3, that doesn’t seem very helpful when you’re trying to plan on multi decade time sacales. The high and low boundaries of the forecast in terms of effect size go from “meh” to “holy crap!”

      • So the question is: was there a trend change?

        This is not an interesting question to me at all. Do the climate models predict one constant rate of warming or something?

  2. Here is the last paragraph of the paper:

    “Researchers have noted that whether the ‘pause’ was real or not, it helped generate research on the mechanisms of climate variability on decadal time scales, and thus increased understanding about the climate system (Lewandowsky et al 2015a, 2015b, 2016, Fyfe et al 2016, Medhaug et al 2017, Nature 2017). While this is true, it is also important to ask what has been lost by the invention of a ‘pause’ in global warming? We will never fully know the answer to this question, but it is clear that the climate-research community’s self-declaration of a ‘pause’ in global warming has created additional confusion for the public and policy-system about the pace and urgency of climate change. This in turn may have contributed to reduced momentum for action to prevent greenhouse climate change, even if only a bit and if only by some years. That lost momentum is likely to be counted in higher total emissions of greenhouse gases before climate stabilisation (Allen et al 2009, Meinshausen et al 2009). The full costs of that are unknownable, but the risks are substantial (World Bank 2012, Hansen et al 2016). That is, there are costs, and there are perspectives upon which it matters whether the ‘pause’ was real or not. The effort to deconstruct the basis for the ‘pause’ is not strictly academic and provides some salient lessons for the science.”

    Does that last sentence mean “we were not really trying to do this objectively”?

    • Not even a little. To say something is “academic” is to say that it is of only theoretical interest only, with no practical relevance; the authors are saying that it has practical import.

      • Terry wrote:

        “To say something is “academic” is to say that it is of only theoretical interest only, with no practical relevance; the authors are saying that it has practical import.”

        OK, thanks, so now we have “the effort to deconstruct the basis for the ‘pause’ is not [only of theoretical interest], and provides some salient lessons for the science.”

        What do you have as the salient lessons for the science?

  3. “The Pause” was always highly dubious as a criticism of warming.

    If you look at the long-term temperature trend, there was clearly a sawtooth pattern. Temperature would trend up for a few decades, then pause or even decline. So even if there was a pause after 1999, it wouldn’t disprove the general upward trend. At most it would prove that models do not incorporate all decadal-scale factors that affect temperature.

    Footnote: the sawtooth pattern in the temperature data has been steadily disappearing as historical temperatures have been revised in recent years. The sawtooth pattern has highly visible a few decades ago, but not so much lately.

    • Discussion of a “pause” makes sense only as a proxy for whether there are substantial decades-long fluctuations in temperature. It’s not a particularly good proxy for this, but then directly discussing long-term autocorrelations in temperature series is not easy either.

      I’ve only glanced at this paper, but it doesn’t seem to grasp this point. Their conclusion that discussing a pause was bad because it led to a loss of momentum for their preferred political actions is not an good sign.

      You can see my own take on the “pause” here: https://radfordneal.wordpress.com/2015/12/19/has-there-been-a-pause-in-global-warming/

    • From the skeptic POV, “the pause” was exactly that – a pause, not and end or rejection of warming. The “pause” or flattening trend was significant because it implied a much lower rate of warming than the advocates were claiming.

      • The trouble with all this is that only makes sense in a world where there are no decadal or greater fluctuations in climate. As usual, the ‘skeptic’ POV turns out to be either cherry picking willfully ignorant…

        • The skeptics weren’t the first to cherry pick 1998. That honor goes to the warming activists, who jumped on the steep uptrend in the noise from 1992-1998 to claim that warming was dramatically accelerating. AS it turned out, they were wrong.

          The chart in the post you link to does show continuous warming. What it doesn’t show is that the 2016 high is on or just above the model average – meaning the projections are *very* warm compared to reality.

        • On the contrary, a “pause” would be significant precisely because it would indicate that there are indeed decadal or longer fluctuartions. And that would indicate that, taking the recent (eg, since 1970) temperature trend as being due to CO2 would be unsafe – the real CO2-induced trend might be lower (or, of course, higher). See the discussion I link to above for a detailed explanation, with simulation study.

        • I agree Radford! Except to me it makes more sense to take the full time series and model fluctuations and trends directly. What I mean is, the dichotomous questions (is there a trend? Is there a fluctuation?) are less interesting since we already know the answer is yes. I see the fixation on the alleged recent pause as, at best, a symptom of bad NHST-style thinking. It seems especially problematic to get to freely choose the end-points of some interval and then do some dichotomous testing. Moreover, In the hands of self styled (or paid) skeptics, any such answer will NOT be interpreted or used in good faith. So there’s that. Since I know you are a good faith scientist and statistician, I look forward to reading your analysis- thanks for the linkage! A priori- here’s another thought: CO2 drives warming due to well resolved physics, however the climate ^response^ to additional forcing could indeed involve fluctuations due to complex feedbacks, etc. Basically, disentangling fluctuations from ‘real CO2 trend’ might get epistemologically murky.

        • “either cherry picking willfully ignorant…”

          Not at all. There have been other extended periods with lower trends. The linked article provides a one-off explanation for the 1940-1975 period, which is widely accepted by the broader climate science community.

          The occurrence of another long pause would draw that one-off explanation into question. An alternative explanation is, as you suggest, that any extended lower trend is just noise in the uptrend. But the distinction is important because if you accept a one-off explanation for the 1940-1975 trend, then you will forecast too high in the future – which seems to be what the models do. OTOH, if the periods of lower trend are noise in the trend, then they’re a real part of the trend, which forces the forecast to include them.

          You’ll note that the trends in realclimate’s updated comparisons show models consistently warmer than reality.

          https://skepticalscience.com/global-cooling-mid-20th-century-advanced.htm
          http://www.realclimate.org/index.php/climate-model-projections-compared-to-observations/

  4. The linked multiverse paper states (first page, second column),

    “Data construction occurs when the raw data are converted into a form ready for analysis.”

    Heck, data construction starts even before that — for example, with choice of measures and other choices in data collection.

  5. Suess wiggles – and if you think I am joking, I am not. Look it up. None of the models I am aware of account for them. Account for them and the trend becomes much cleaner.

      • The Suess Wiggles have also been connected with temperature, see a paper by David Thompson (yes he of multi-taper spectral estimates). Also look at some of Radford Neal’s comments above, which while I may not agree with all his conclusions, estimates must account for all of what is going on in the data. There is some evidence of the Suess effect at roughly 52 years, 24 years and 12 years. If we have a trend super-imposed on a cycle, and we don’t properly account for the cycle, then when the cyclic term is on the upswing the trend will seemingly be increasing at a faster rate, and when the cycle is on the downswing, the trend will seemingly be slowing. Properly account for the cycle, and the trend is more consistent. I certainly see Suess wiggles in ocean temperature series. Removing the cyclic term does not make the trend disappear, as some skeptics have claimed (that it is all solar and correct for it and there is no trend), what it does do is make for a smoother and more consistent trend.

        The eastern Pacific is seeing a really significant warming, basin-wide. I can not prove this because we are in the midst of it, but a very preliminary look suggests the effect may be (and I emphasize the may be) a combination of the long-term warming trend in the ocean plus the Suess wiggles (or similar effect) on the upswing, starting about 2011 or 2012. If I am correct, starting around 2024 we should again see signs of “deceleration” or a “pause”. If that pans out, you can visit me in my old age home, where I will be wrapped in blankets on a rocking chair sipping hot toddies, and telling me if it was correct.

        • Since I am a terrible writer I hope that it is clear that I see a very clear, strong, long-term warming trend in the ocean, and the point was when you don’t account for the cyclic term(s) then that dynamic bleeds into the trend and you get all sorts of over-interpretation of what is happening. The reference to Freakonomics had me worried.

          I have been looking at environmental time-series and space-time data for over 40 years, and if there is one thing I have learned is that when data are correlated in time and space, and have complicated underlying dynamics, it is very, very easy to find patterns in noise, because of all of the correlations. Why a lot of today’s newer methods scare me. They are being used on spatio -temporal data but treating them as if the data were all independent.

          And see, in a single response I have given people two reasons to flame me.

Leave a Reply to jim Cancel reply

Your email address will not be published. Required fields are marked *