Skip to content

In statistics, we talk about uncertainty without it being viewed as undesirable

Lauren Kennedy writes:

I’ve noticed that statistics (or at least applied statistics) has this nice ability to talk about uncertainty without it being viewed as undesirable. Stan Con had that atmosphere and I think it just makes everyone so much more willing to debug, discuss and generate new ideas.

Indeed, in statistics I’ve seen fierce disputes, but I don’t typically see people trying to dodge criticism or attack people who point out errors in their work. I’d like to think that one reason for this productive style of handling dissent is that uncertainty is baked into our way of thinking.

P.S. Lauren writes of Jazz, pictured above: “Her claim to fame is that she can open two of four door styles present in the houses she’s lived in.”


  1. No I haven’t seen outright attacks. Nevertheless, there are passive-aggressive behaviours rearing their backal and frontal sides too. LOL Individuals who are able to maintain their composure and moral compass are rare. This is not to suggest that reactions are always negative. Outrage can be quite useful.

    Rather the tendency to cliqueishness is prevalent, judging from Twitter interactions. We have some very opinionated/ideological people in every arena.

    In so far as the discussions about ‘uncertainity’, I, as an outside the field, am skeptical of some uses it in different discussion. It should have a subject I should have paid far more attention to. The reason I say this is because conflicts of interests and career considerations have played a large role too.

    It’s also the case that many domains have to rely on eclectic thinking to generate anything new, whether for the good or the bad.

  2. zbicyclist says:

    One of the unfortunate things about basic statistics is that we tend to tend to call things errors (most notably standard error) when we really mean uncertainty or variability. And this sticks (as few other things do) when these MBAs go up the management ladder.

    At a couple of points in my career, I’ve spent some weeks deriving “error terms” around complex estimations to produce “error tables”, when, of course, we aren’t so much talking about things being erroneous, just uncertain.

    Periodically, I’m asked to expand these tables to cover some additional measurement circumstance. Co-workers have occasionally asked if I mind doing this. I respond “No. I’m getting paid to make errors. And the more errors I make, the more I get paid!”

    • Jonathan (another one) says:

      A colleague of mine had her court testimony impugned by the other side: “She even admits that her analysis has standard errors!”

      • Keith O'Rourke says:

        It was reported once that in court, when the judge heard the phrase standard error, they declared that the court would not permit any errors – standard or otherwise.

        • Thanatos Savehn says:

          What usually happens is that the clever expert retorts “but my error rate is only 5% so that means I’m right 95% of the time”, and the judge says “Ah! I get it now. Since error and bias go to the weight of the evidence rather than the admissibility of the evidence we’ll leave it to the jurors to sort out. Objection overruled. Mr/Ms Expert, you may explain your answer.” I engaged in some of this insanity 20+ years ago but only because the “expert” epidemiologists used on both sides agreed that the quote in the preceding sentence is how p-values are supposed to be interpreted, because the “learned treatises” that we relied on said the same thing and because I assumed that since the opposite of error is accurate that 1-p = accuracy of the claim.

    • Martha (Smith) says:

      Yes, the used of “error” to label “uncertainty” is a barrier to learning statistics. Unfortunately, the usage has become so standard (ironic pun partially intended) that the reality is that statistics teachers have the responsibility (not always carried out) to emphasize (repeatedly) that

      i. If it involves statistical inference, it involves uncertainty.

      ii. The technical meanings of terminology often conflict with the ordinary meanings — so continual caution is needed.

    • Curious says:

      Now, I am certainly willing to acknowledge the distinction between an observed error and the uncertainty of a possible error. However, I am not willing to fully separate the idea of error from the definition of uncertainty.

      • There’s a big difference between “difference between a prediction made by a model and the actual outcome” which is one basic definition of a statistical error, and “whoops I selected the wrong cell in Excel” which is the kind of “error” that most people think of in common usage of the term error.

        • Martha (Smith) says:

          Good example to explain the general point: “Error” in common usage means “mistake”. However, the technical meaning of “error” in statistics does NOT mean “mistake” — so it’s a poor choice of the word “error” for the statistical concept, but it has become standard in the field, so it is too late to change it. But the consequence is that we need to be very careful to explain and emphasize that the technical usage differs from the everyday usage.

          • Curious says:

            Does a model that produces a result that is inconsistent with the reality being modeled not in fact produce a mistake?

            • Curious says:

              An error term that creates boundaries around an estimate is essentially claiming that the true value is within this interval but we don’t know where precisely and thus any specific value is likely a mistake in the assertion of that value as the true value.

              • It’s fine to think like that as a statistician, but it’s an unavoidable fact that models can never predict with precise accuracy outside a few narrow domains. It’s not an unavoidable part of prediction that you select the wrong Excel cell or type the wrong code into your computer, or enter the wrong formula into your calculator. Those kinds of errors are avoidable by being more careful or having a second person check things etc. Courts normally require you to take some kind of care before testifying, but no amount of care is going to eliminate the errors in weather forecasts for example.

              • Curious says:

                The same can be said for causal claims, though I think they are far more costly on average. The problems I see quite often are when causal claims are assumed for data that cannot possibly generate that level of information. The costs can be enormous, both in spending money on interventions that cannot possibly work and on high salaries to people who are quite intellectually gifted, but who seem to lack the critical reasoning necessary to properly understand that the claims they are making are blatantly false.

          • Thanatos Savehn says:

            This I think would be more like what people imagine when they read of “error”: Again, channeling Wittgenstein here, maybe statistics’ biggest problem is its language rather than its methods.

    • Mikhail says:

      I dont think the term “standard deviation” is any better.

      Hey, it looks like our statistical language is deeply based in XX century where we tended to view things as “the norm and deviation from norm” rather just “some set of different things”.

  3. Precisely because of statisticians’ comfort with and awareness of uncertainty, I see room for compatibility between statistics and the humanities (including literature and arts). The two are not at odds. In the view of some, anything involving “data” is ultimately dehumanizing in that it reduces people to numbers and averages–but statisticians are often the first to point out the fallacy of such reduction.

    And what an irreducible-looking cat!

    • Steve Sailer says:

      Is that a housecat or a lynx?

      I’d love to spot a lynx in the wild.

    • Keith O'Rourke says:

      > literature and arts). The two are not at odds.
      Well, perhaps the best subsets of each are open to each other but I had the worst experience ever giving a talk to a mixed audience of quantitative and qualitative researchers in Toronto in 1998 (it was part of the process in this new merge department which included both).

      I was basing my presentation on work by Jean Claude Gardin and Christopher S Peebles who ran a retreat on the topic. After the qualitative speakers decided to dismiss the agreed to prior definitions of terms to be used as it was to their advantage, one of their members declared that as they once tutored in an introductory statistics course they understood my subject thoroughly and another claimed that if I enrolled in their introductory sociology course – it would change my life for ever. I was too stunned to know how the respond.

      Part of the challenge may be some thinking thinking their particular perspective is all that matters or matters more importantly.

      And they might have very different goals-
      Our imagination is stretched to the utmost, not as in fiction, to imagine things which are not really there, but just to comprehend those things which are there. (Feynman et al. 1964, 127–128, “The Character of Physical Law”)

      • What a funny (and telling) story! Yes, I see your points–both about people cleaving to their own perspective and about their having different goals. That said, those who imagine things that aren’t there and those who use their imagination to comprehend things that are there have something in common. They both acknowledge that there’s something beyond what they know, see, and understand right now.

      • Keith

        What an interesting story. You remind me of many of my father’s more intellectually interesting colleagues. A rare breed.

  4. Dan Wright says:

    Here is quote from a former teacher of mine. He told it to use in class then shared to Royal Stats Society in a commentary on Goldstein & Spielgelhalter (1996, JRSS:B, 3, 385-443).

    We think that we know about uncertainty, and that when we have added a standard error or
    a confidence interval to a point estimate we have increased knowledge in some way or another. To many people, it does not look like that; they think that we are taking away their certainties–we are actually taking away information, and, if that is all that we can do, we are of no use to them.
    \dots{} Our insistence that we could not deliver certainties was regarded as a sign of weakness, if not downright incompetence. One may laugh at that, but that is the way it was \citep[p.~428]{Bartholomew1996}.

Leave a Reply to Curious