What does it mean to talk about a “1 in 600 year drought”?

Patrick Atwater writes:

Curious to your thoughts on a bit of a statistical and philosophical quandary. We often make statements like this drought was a 1 in 400 year event but what do we really mean when we say that?
In California for example there was an oft repeated line that the recent historic drought was a 1 in 600 year event though we only have a well instrumented hydrologic record of approximately 100 years. Beyond that it’s mostly tree ring studies.
I suppose this is a subset of the larger issue of quantifying rare but impactful events like hurricane, flood, terrorist attacks or other natural and man-made disasters. Generally hard to to infer conclusively from a single observation (in this case the recent droughts in South Africa and California).
The general practice is to utilize the historical instrumented record to construct a probability distribution and see where the current observation lies on that. So what we’re really saying with the 1 in 400 years line then is that this is a 1 in 400 year event assuming the future will continue to look like the past hundred years.
But with climate change there’s good reason not to believe that! So I’m curious if you have any ideas about ways to incorporate priors about future hydrology utilizing climate modeling into these sorts of headline statements about the likelihood of the current drought. And then thinking ideally would communicate through transparent tools like what Bret Victor advocates for so the assumptions are clear ( http://worrydream.com/ClimateChange/ ).
For context we emailed a bit a few years back about applied data science questions [see here and here] and I’ve been leading a small data team automating the integration of urban customer water use data in California and contextual information. You can see the resulting dataset ( http://bit.ly/scuba_metadata ) and analytics ( https://demo.californiadatacollaborative.com/smc/ ) if you’re curious.

My reply:

Get Gerd Gigerenzer on the line.

You’ve got a great question here!

When it comes to communication of probability and uncertainty, I’m only aware of work that assumes that the probabilities are known, or at least that they are constant. Here you’re talking about the real-world scenario in which the probabilities are changing—indeed, the change in these probabilities is a key issue in any real-world use of these numbers.

The point of saying that we’re in a hundred-year drought is not to say: Hey, we happened to see something unexpected this year! No, the point is that the probabilities have changed; that a former “hundred-year drought” is now happening every 20 years or whatever.

I have no answers here. I’m posting because it’s an important topic.

And, yes, I’m sure there are researchers in the judgment and decision making field who have worked on this. Please share relevant ideas and references in the comments. Thank you.

P.S. Hey, I just noticed the name thing: a hydrologist named Atwater! Cool.

12 thoughts on “What does it mean to talk about a “1 in 600 year drought”?

  1. A couple interesting points.

    First, even if the distribution is known, the implications are unclear because the events that lead to floods in a given region can be geographically and temporally correlated in complex ways, or even anti-correlated in strange cases! For example, floods from a very full river due to increased snow melt + rainfall in a given year may be regionally sand temporally correlated within a year. For that reason, among others, risk communication is hard. (A related challenge is that people misinterpret 1-in-X as “only every X years,” instead of “mean expected probability of 1/X per year”.)

    Second, the general practice in statistics for extreme value distributions was used by actuaries for a long time, but for earthquakes and hurricanes since the mid-1990s, it has been almost completely replaced by a new class of risk model based on fundamental physical models that use simulated events. The somewhat harder problem of forecasting flood risk was not addressed at the time, in part because US flood risk is insured by the US government. However, like other extreme risks, it is a place where fundamental physical / hydrology models are really important. A friend of mine started Katrisk – http://www.katrisk.com, which now builds flood models based on fundamentals. These types of are calibrated based on historical data, but (in the case of flood risks) include changing hydrology, uncertainty in levels of rainfall up-river, the existence of new dams, etc. (I believe other Catastrophe risk modelling firms now also have such models, but I am uncertain, as I no longer work in that area.)

    Lastly, on the question of how to display such information, and how to inform people about it, I’d suggest looking at the work of / asking:
    1) Howard Kunreuther, who leads the UPenn/Wharton “Risk Management and Decision Processes Center” – https://riskcenter.wharton.upenn.edu/ 2) Baruch Fischhoff at CMU, https://scholar.google.com/citations?user=GxvFzdcAAAAJ , who specializes in risk communication research, and 3) Lloyd Dixon, https://www.rand.org/about/people/d/dixon_lloyd.html (full disclosure, who I worked with at RAND), who runs the RAND Center for Catastrophic Risk Management and Compensation, which focuses more on the policy side.

  2. A colleague who recently moved here and bought a house in a 100 year flood plain was promptly flooded. He asked me (likely rhetorically) while in an angry mood “What are the odds of getting flooded the first year you move here?” I thought about all the area flood plains and all the people moving in, and about the marginalized, underappreciated mode and decided to offer a conjecture based on this image from DataColada: http://datacolada.org/wp-content/uploads/2018/04/71-Fig3.jpg

    He was having none of it and was certain that the people who came up with the maps were all idiots and that the developers and real estate agent knew it but failed to disclose it. Nevertheless, I do think that some portion of the invariable surprise voiced whenever someone encounters a rare event that occurs earlier than would have been predicted by the median or mean might be a mode of thinking in need of the mode. Or have I got it wrong (again)?

      • I assumed as much. The point I was trying to make was about the surprise factor. It rains a lot here, the topography is flat and lots of people have been moving into old and new areas for decades. If people in the Texas coastal plain are asked “How many years were you here before you were flooded the first time?” it shouldn’t be a surprise that their answers are much lower than intuition would suggest if the 100 year flood plain estimates are accurate; and it shouldn’t be a surprise that many rolled a 1 on a 100-sided die when the problem is framed as “most common number of years to first flood”. Or so I figured.

      • More correctly, the braindead simple and definitely wrong model of extreme floods as independent identically distributed poisson process in time assumes that the mean time between floods of a certain height is N years, and that therefore the time between floods is exponentially distributed with rate 1/N

        In reality, it turns out that physics rather than random number generators is what determines when floods occur.

  3. It doesn’t have any real statistical validity. It’s some kind of guess. Perhaps a useful educated guess at best.

    If you look at the IPCC SREX (Special Report on Extremes) 2012 you find a couple of papers that are highlighted to look at recent trends in droughts – Sheffield & Wood 2008 and Dai 2011. Sheffield and Wood show (over the last 50 years) droughts increasing in some places and decreasing in others and globally droughts *decreasing*.
    Dai shows droughts increasing in some places and decreasing in others and globally droughts *increasing*.

    So the conclusion is that we don’t know if droughts have increased or decreased globally over the last 50 years under various measures of drought. (Off, topic, of course all good people know that droughts have increased due to “climate change” so lucky that no one reads the actual reports).

    Dai 2011 **also** shows drought measures (PDSI), based on proxy data, for western north America over the last 1000 years and for China over the last 500 years. What you see is decade to decade and century to century substantial changes in droughts. I don’t mean 10% changes.

    So, if you look at the last 50 years you might reach one conclusion. If you look at the last 500 years you might reach a completely different conclusion.

    I don’t know if I can insert images here so instead you can see extracts of these two graphs from Dai at https://scienceofdoom.com/2017/02/12/impacts-v-climate-change-is-already-causing-worsening-storms-floods-and-droughts/#comment-116440

    Trying to form useful statistical ideas on non-linear chaotic processes where the entire time series is not known is not very meaningful. At least, that’s how it appears to me. If you take as an example the classic Lorenz 3-equation model and you have “deity like” powers to see the entire time series you can identify meaningful statistics. If you don’t have deity like powers and you can’t see the entire time series you are just guessing that the whole of the time series is like the sample you have in front of you.

  4. Hopefull you don’t find it boring to note that the Washington State Dept. of Transportation’s program manager for tunnels was Todd Trepanier. He was a significant figure in the recent project to replace the Alaskan Way viaduct in downtonwn Seattle.

  5. Not directly relevant to the topic. But, Science magazine has two interesting pieces on retractions this week. See
    http://science.sciencemag.org/content/362/6413/390 and
    http://science.sciencemag.org/content/362/6413/393.

    This latter piece conflicts with my statements from time to time on this blog that electrical engineering journals do not seem to have the same problems with bad articles and non-replicatable results as do fields such as social psychology.

    It begins:
    Some 40% of the retractions in the Retraction Watch database have a single curious origin. Over the past decade, one publisher—the Institute of Electrical and Electronics Engineers (IEEE) in New York City—has quietly retracted thousands of conference abstracts.

    Bob

  6. I’m sorry about asking this here, since this is almost certainly not the right place for it, but I can’t seem to find an FAQ anywhere – I see several posts where readers submitted statistical questions like this one. Where and how is one supposed to submit questions?

  7. The inherent bias in news stories can make it seem that extraordinary events are happening more often than they should.

    Take a simple example. Say there are 100 different watersheds in the US and flooding events are uncorrelated across them. We would expect to see, on average, one extreme flooding event each year, i.e., one hundred-year flood in one of the watersheds. Since this is the only flooding news reported (non floods are not reported), it can sound like there are too many hundred-year floods because we hear about one every year.

    Hurricane reporting is similarly biased. For more than ten years, there were very few hurricanes. There was very little reporting of this, so very little awareness that there were “too-few” hurricanes for a long time. Then, there were a few doozies, and suddenly there was a hurricane crisis. Careful studies have shown there is actually no trend in hurricane frequency or severity in the last hundred years or so.

    The reporting bias works across disaster categories as well. One year extreme flooding is reporting. Another year it is wildfires or tornadoes or hurricanes or locusts or boils and so on. If it isn’t one damn thing, its another.

  8. “But with climate change there’s good reason not to believe that!”

    You also need to account for changes that will be made in response to climate change.

Leave a Reply to Daniel Lakeland Cancel reply

Your email address will not be published. Required fields are marked *