Deterministic thinking (“dichotomania”): a problem in how we think, not just in how we act

This has come up before:

Basketball Stats: Don’t model the probability of win, model the expected score differential.

Econometrics, political science, epidemiology, etc.: Don’t model the probability of a discrete outcome, model the underlying continuous variable

Thinking like a statistician (continuously) rather than like a civilian (discretely)

Message to Booleans: It’s an additive world, we just live in it

And it came up again recently.

Epidemiologist Sander Greenland has written about “dichotomania: the compulsion to replace quantities with dichotomies (‘black-and-white thinking’), even when such dichotomization is unnecessary and misleading for inference.”

I’d avoid the misleadingly clinically-sounding term “compulsion,” and I’d similarly prefer a word that doesn’t include the pejorative suffix “mania,” hence I’d rather just speak of “deterministic thinking” or “discrete thinking”—but I agree with Greenland’s general point that this tendency to prematurely collapse the wave function contributes to many problems in statistics and science.

Often when the problem of deterministic thinking comes up in discussion, I hear people explain it away, arguing that decisions have to be made (FDA drug trials are often brought up here), or that all rules are essentially deterministic (the idea that confidence intervals are interpreted as whether they include zero), or that this is a problem with incentives or publication bias, or that, sure, everyone knows that thinking of hypotheses as “true” or “false” is wrong, and that statistical significance and other summaries are just convenient shorthands for expressions of uncertainty that are well understood.

But I’d argue, with Eric Loken, that inappropriate discretization is not just a problem with statistical practice; it’s also a problem with how people think, that the idea of things being on or off is “actually the internal working model for a lot of otherwise smart scientists and researchers.”

This came up in some of the recent discussions on abandoning statistical significance, and I want to use this space to emphasize one more time the problem of inappropriate discrete modeling.

The issue arose in my 2011 paper, Causality and Statistical Learning:

More generally, anything that plausibly could have an effect will not have an effect that is exactly zero. I can respect that some social scientists find it useful to frame their research in terms of conditional independence and the testing of null effects, but I don’t generally find this approach helpful—and I certainly don’t believe that it is necessary to think in terms of conditional independence in order to study causality. Without structural zeros, it is impossible to identify graphical structural equation models.

The most common exceptions to this rule, as I see it, are independences from design (as in a designed or natural experiment) or effects that are zero based on a plausible scientific hypothesis (as might arise, e.g., in genetics, where genes on different chromosomes might have essentially independent effects), or in a study of ESP. In such settings I can see the value of testing a null hypothesis of zero effect, either for its own sake or to rule out the possibility of a conditional correlation that is supposed not to be there.

Another sort of exception to the “no true zeros” rule comes from in- formation restriction: a person’s decision should not be affected by knowledge that he or she does not have. For example, a consumer interested in buying apples cares about the total price he pays, not about how much of that goes to the seller and how much goes to the government in the form of taxes. So the restriction is that the utility depends on prices, not on the share of that going to taxes. That is the type of restriction that can help identify demand functions in economics.

I realize, however, that my perspective that there are no true zeros (information restrictions aside) is a minority view among social scientists and perhaps among people in general, on the evidence of [cognitive scientist Steven] Sloman’s book [Causal Models: How People Think About the World and Its Alternatives]. For example, from chapter 2: “A good politician will know who is motivated by greed and who is motivated by larger principles in order to discern how to solicit each one’s vote when it is needed” (p. 17). I can well believe that people think in this way but I don’t buy it: just about everyone is motivated by greed and by larger principles. This sort of discrete thinking doesn’t seem to me to be at all realistic about how people behave—although it might very well be a good model about how people characterize others.

In the next chapter, Sloman writes, “No matter how many times A and B occur together, mere co-occurrence cannot reveal whether A causes B, or B causes A, or something else causes both” (p. 25; emphasis added). Again, I am bothered by this sort of discrete thinking. I will return in a moment with an example, but just to speak generally, if A could cause B, and B could cause A, then I would think that, yes, they could cause each other. And if something else could cause them both, I imagine that could be happening along with the causation of A on B and of B on A.

To continue:

Let’s put this another way. Sloman’s book is called, “Causal Models: How People Think About the World and Its Alternatives,” and it makes sense to me that people think about the world discretely. My point, which I think aligns with those of Loken and Greenland, is that this discrete model of the world is typically inaccurate and misleading, so it’s worth fighting this tendency in ourselves. The point of the above example is that Sloman, who’s writing about how people think, is himself slipping into this error.

One more time

The above is just an example. My point is not to argue with Sloman—his book stands on its own, and his ideas can be valuable even if (or especially if!) his perspective on discrete modeling is different from mine. My point is that discrete thinking is not simply something people do because they have to make decisions, nor is it something that people do just because they have some incentive to take a stance of certainty. So when we’re talking about the problems of deterministic thinking, or premature collapse of the “wave function” of inferential uncertainty, we really are talking about a failure to incorporate enough of a continuous view of the world in our mental model.

P.S. Tomorrow’s post: I think that science is mostly “Brezhnevs.” It’s rare to see a “Gorbachev” who will abandon a paradigm just because it doesn’t do the job. Also, moving beyond naive falsificationism.

50 thoughts on “Deterministic thinking (“dichotomania”): a problem in how we think, not just in how we act

  1. Nice – I like the BB analogy, although here in Israel we have another BB.

    Where the discussion should go, is emphasize Information Quality. Dichotomisation is not the best way to generate information quality. It requires much more.

    If one agrees that Statistics is about the generation of information, how can we design, produce and assess information quality?

    My book with Galit Shmueli titled: Information Quality: The Potential of Data and Analytics to Generate Knowledge, defines information quality as the utility, U, in applying a method of analysis, f, to a data set, X, conditioned on the analysis goal, g. In other words, Information quality, InfoQ(U,f,X,g) = U(f(X|g)). To assess it we define eight dimensions that determine InfoQ. These are:
    1) Data Resolution
    2) Data Structure
    3) Data Integration
    4) Temporal Relevance
    5) Chronology of Data and Goal
    6) Generalizability
    7) Operationalization
    8) Communication

    To achieve information quality, you need to properly address all these dimensions, given your goals. Like the cover of the InfoQ book, you need to see the pattern, and then you need to know its meaning (Hint: Google The Hitchhiker’s Guide to the Galaxy).

    What is the significance of these dimensions? Let’s say you are the production manager of a company developing and producing consumer goods. Efficiencies in production and quality of products are your main goals. Do you have data with the right resolution to allow you to achieve your targets? You need to know about the content of each product in order to ensure quality and track the production yields in the manufacturing plant to manage efficiencies. Reporting aggregated quarterly yields is not enough. Process control data is needed by batch, or shift, so that you can manage the process efficiently. Data resolution is the first InfoQ dimension. Communication is the eighth dimension. If the control charts you maintain are not communicated to the right person, at the right time, in the right way, the efficiency in running processes is hampered. More in the book and many application articles. Ihe sixth InfoQ dimension of Generalizability deserves more attention in the context of Mayo’s severe testing.

    There are two types of generalizability: statistical and scientific/engineering/mechanistic. Statistical generalizability refers to inferring from a sample to a target population. Scientific generalizability refers to applying information using first principles, mechanistic models, domain expertise or intuition, i.e. reaching decisions not necessarily on the basis of data. Pearl, for example, uses structural causal networks and the do calculus to generalize findings from experiments conducted in set up A to set up B. Professionals use their experience to generalize from past events to the handling of current circumstances. My recent book with Tom Redman on The Real Work of Data Science dedicates a full chapter to it (Chapter 8).

    I was glad to see that the recent report by the National Academies also mentions Generalisability, although in more limited way than the above. http://nap.edu/25303

  2. RE: Epidemiologist Sander Greenland has written about “dichotomania: the compulsion to replace quantities with dichotomies (‘black-and-white thinking’), even when such dichotomization is unnecessary and misleading for inference.”
    ——

    I witness this practice black and white thinking and dichotomies daily even by those who caution against both: in informal conversations. Both are ingrained habits. The debates themselves are couched as binaries or false dichotomies in international relations and in historical essays/articles.

    I am not sure that ‘compulsion’ is quite the characterization I would use b/c it seems to imply that it forced.

    com·pul·sion
    /kəmˈpəlSHən/
    Learn to pronounce
    noun
    noun: compulsion; plural noun: compulsions

    1.
    the action or state of forcing or being forced to do something; constraint.
    “The payment was made under compulsion”
    synonyms: obligation, constraint, force, coercion, duress, pressure, pressurization, enforcement, oppression, intimidation; force majeure
    “he had been under no compulsion to go”
    2.
    an irresistible urge to behave in a certain way, especially against one’s conscious wishes.
    “he felt a compulsion to babble on about what had happened”
    synonyms: urge, impulse, need, necessity, desire, longing, motivation, drive; More
    obsession, fixation, addiction;
    temptation, pull;
    informal ones
    “he felt an overwhelming compulsion to tell her the truth”
    ——-

    It’s an ingrained habit seemingly, compounded by argument culture.

  3. I think you’re generally onto something, but I think you’re limiting yourself by thinking in DAGs–in one-way causation. This seems more about the examples you drew than the dichotomania problem, but fixing one may help fix the other. I also think it it may divorce “dichotomania” from “deterministic thinking.”

    Think back to your MCSim / PBPK work with Frederic Bois and others. If you have a toxin in the body, the amount excreted (assuming that’s the exit path) is caused by the amount currently in the body (positive correlation), and the amount in the body is decreased by the amount excreted (negative correlation). Under certain assumptions, you get exponential decay in the quantity of toxin in the body.

    Does the toxin in the body cause the excretion of toxin? Yes.

    Does the excretion of toxin cause a reduction in toxin in the body? Yes.

    Does it provide insight to talk of only one of those causal paths? Probably not; both are at work simultaneously. Put another way, the cause of the overall system’s performance is the (feedback) loop (the directed graph that is not acyclic), not any one piece of the system. I find it’s all too easy to fool myself if I ignore the effect of those cyclic connections.

    By modeling them with an ODE, you end up modeling a feedback system, and you get insights into its performance. Simulation is a typical approach, of course, for complex systems; feedback control theory may be useful for linear or linearizable systems.

    That leads to another interesting point: we often say that correlation is not causation. Modeled this way, the “cause” (or what we think of as being the cause) ends up being the derivative, and the effect ends up being its integral. Switching slightly, think of “cause” as the amount of gasoline you consume in your car per unit time on a road trip and “effect” as the amount of gasoline remaining in the tank. That has a similar equation, except the outflow is now a constant and the amount remaining is now a ramp down–until you get low enough to want to fill up. At that time, the gas entering the tank per unit time is probably a new constant, and the amount remaining is a ramp up.

    So, outside of the exponential case, the cause doesn’t even look like the effect, just as the integral in general doesn’t look like the argument to the integral. Outside of the exponential case, it would seem to be uncommon to find a cause and effect that /do/ look alike. (The counter-example is often found if you compare the integral of the “cause” to the effect, which is a natural integral.

    How does that tie to deterministic thinking / dichotomania? I think it moves the discussion from “Does A cause B?” and to “Does B cause A?” to “How do the interactions between A and B affect the performance of this system?” *more of a continuous question).

    I think that way of thinking /can/ help move us from dichotomania, but I think I like Greenland’s term better than “deterministic thinking,” for note the gas consumption model was purely deterministic, at least as I phrased it, and yet the cyclic graph approach was still essential to making sense of it (because it’s a familiar system, we probably did that implicitly).

    • Does the toxin in the body cause the excretion of toxin? Yes.

      Does the excretion of toxin cause a reduction in toxin in the body? Yes.

      What insight are people getting from asking/answering these questions? It seems like playing word games.

      Just say “Excretion rate is some increasing function of toxin concentration”.

    • > Does it provide insight to talk of only one of those causal paths?

      Basically there is one causal path, but we can’t discover it without thinking in terms of dynamics… what is going on NOW causes changes in what is going on *at the next moment in time* based on dynamics… But although we talk about “the amount of toxin in the body” as being a variable, in fact it’s a function indexed by time. so C(t0) and C(t0+dt) are not the same thing at all, and although C(t0) causes C(t0+dt), it’s not the case that C(t0+dt) causes anything at time t0

      • This is one of the reasons I am not particularly fond of the DAG literature… DAGS are basically good for discovering processes that can be considered as “fast equilibriation” processes. They basically model two points in time. “start” and “end”

        so you can say if “at the start the ball is above a particular point on the floor” then “after dropping it, at the end the ball will be in a particular cup” and “letting go causes the ball to be in the cup at the end”

        but this is only satisfying if the whole process you’re interested in is far longer than say 1/2 a second. Because if you want to model the first 100ms of fall, you’re going to need a differential equation and an expression for the drag coefficient, and etc etc.

        • Daniel, you stated that “DAGS are basically a model two points in time. start and end”. I think that’s quite wrong. Cycles can be unraveled into acyclic form by using multiple time-ordered nodes, and much more. Have a look at the DAGs used in the logitudinal/time-dependent causality literature, which accommodate unlimited feedback and measurement of states from whatever is start to whatever is end (which always exist in any real problem), along with frameworks and statistical methods for estimating effects along the way.
          The founding theory goes back over 20 years:
          https://www.hsph.harvard.edu/james-robins/files/2013/03/cicld-ucla.pdf
          Here’s a primer:
          https://www.med.lu.se/content/download/90155/622060/file/LongDAG_Anna_Ekman_130822_PDF.pdf
          Now of course that’s for the kind of very discrete processes actually observed in the social-sci and health-medical literature, but the underlying general theory and models for continuous time are given in the extensive writings of Robins and colleagues. The DAGs stay discrete but are good for helping spot problems like collider bias arising from adjustment or other conditioning on intermediate events.

        • Sorry I shouldn’t have been so general. It’s clear to me, given my love for nonstandard analysis, that ODEs are just extremely enormous DAGs whose structure is an enormous chain…

          y(t[i]) = y(t[i]) + F(y(t[i-1]),a,b,c,d…) dt

          I should have maybe said something like “typical applications of DAGs” or something like that. At least all the discussions I’ve seen tend to represent relatively simple models of A,B,C somehow causing D,E,F. There’s an arrow of time, but not a magnitude associated with the time duration in most of these examples.

          And it’s not just “typical DAGs” either it’s many typical algebraic models even if they don’t involve a formal DAG. Like economists think in terms of price being the solution to where supply=demand, but in reality there’s a dynamic process where imbalance in supply vs demand causes prices to move through time, and the market dynamics are often ignored.

    • Overly discrete thinking is bad, but overly quantitative thinking seems to carry risks as well. There is the danger of quantifying something that is not completely quantitative. A good example is perhaps provided by the causal diagram in your excerpt. “Beer consumed” is easily and uncontroversially quantifiable in terms of a continuous variable (liters consumed), but “socio-economic status” and “intelligence” are not as straightforward, and introducing a precise numerical measure may introduce extra mathematical structure that doesn’t correspond to anything real in the world.

    • A very good article indeed. Thanks for the link.
      A quote as a teaser for those who haven’t yet read it:

      “If a time machine could serve up to you your 200 million greats grandfather, you would eat him with sauce tartare and a slice of lemon. He was a fish. Yet you are connected to him by an unbroken line of intermediate ancestors, every one of whom belonged to the same species as its parents and its children.”

  4. I like this post (no surprise there). Andrew mentions items I wish I had cited in my 2017 article on dichotomania, nullism, and reification at
    https://doi.org/10.1093/aje/kwx259
    I also wish I had known of and cited the Dawkins 2011 New Statesman piece that Monroe mentioned, as well as the section titled “Black and White – A Dangerous Dream” at the end of p. 100 in Tukey 1991,
    https://projecteuclid.org/download/pdf_1/euclid.ss/1177011945

    I do think dichotomizing or categorizing is a compulsion in the ordinary sense of the word, not just an acquired habit, in that it has an innate, natural quality. In everyday life it’s a heuristic in constant service of decisions, like asking someone “is it cold out?” to decide whether to grab a coat. The trouble is that, like so many useful natural heuristics and much of uneducated common sense, it can and does subvert the more refined goals of scientific research, in just the way Andrew and many others describe.

    Its companion in this subversion, nullism (placing unwarranted special focus on the null hypothesis, as in NHST without equal testing of alternatives), is I think more of a habit inculcated by the teaching-research establishment. That inculcation seems to have arisen from greater fear of “false positives” and hyperactive researcher imaginations than of “false negatives” and pseudoskepticism (the “false positive/negative distinction” is of course itself a prime manifestation of dichotomania). I often see it defended via a confusion of parsimony heuristics with reality (to paraphrase Neil de Grasse Tyson, reality is under no obligation to behave simply for you). Or more dramatically, defended with the confusion of pseudoskepticism and science – per T.H. Huxley, I see science instead distinguished by a commitment to agnosticism, which is to say: spare me your commitments (even if veiled as “statistical inferences”) – show me the data.

    Reification is seen where these often-distortive modes of thinking are enshrined in methods (like NHST) that are taught as if truth indicators, or as THE “scientific method” (yes, I’ve seen professors of biostatistics claim that NHST is “the” scientific method in statistical form). More subtly, it includes focusing on conditional (subjunctive) interpretations of statistics (which take for granted implicit assumptions like the form of the data model and prior model, and so become part of “uncertainty laundering”) to the near-exclusion of unconditional interpretations (which explicitly include assumption violations in their explanations for observations).

    – The above are just my top 3 among named classes of cognitive biases that I see as plagues of statistical teaching and applications in research. Plenty more where those came from…

    • No doubt there are plenty more.

      My question is this:

      Why don’t we already have in place a tradition to study how students learn statistical knowledge? Or how working scientists communicate that knowledge to each other?

      I am thinking of something like Pim Levelt’s portrayal of a speaker and a listener, but applied to scientists writing papers and reading them.

      • Sure – my masters thesis in biostatistics was to do Herb Simon’s think aloud protocol analysis https://en.wikipedia.org/wiki/Think_aloud_protocol in sample size consultations. Everyone ran the other way and I had to change the topic.

        Much later I wrote a grant, part of which was to analyse statistical work done in the wild (in clinical research). The reviewers were extremely negative and trashed the grant. Now my director had warned – funding agencies don’t want someone documenting the people they are funding don’t know how to do research properly. Who would have thunk?

      • Doug asked,
        “Why don’t we already have in place a tradition to study how students learn statistical knowledge? ”

        There is some tradition in this — for example, the Journal of Statistics Education, the ASA Section on Statistics and Data Science Education (Formerly the Section on Statistics Education), and the ASA Teaching Statistics in Health Science Section

    • I very much appreciated your 2017 article. The synonyms for ‘compulsion’ are all over the place, as you can see from the dictionary definition I posted. Nor am I sure about what is actually meant by ‘continuous’ and ‘discrete’ as their conceptual bases are guided, seemingly, different theories in different disciplines. I guess you may be meaning these two terms as they are conceived in statistical modeling/thinking.

  5. In your critique of Sloman, you first say that you can’t “see why only one of the three models can be true” and only then offer the second statistical critique that in practice you will not have true conditional independence. But the latter is the more fundamental critique! If you did in fact have true (conditional) independence, the restrictions on different causal graphs follow as a theorem. (Admittedly, you may need an additional assumption, like Pearl’s “no fine tuning”.) In the case where you do believe in “true zeros”, like “independences from design (as in a designed or natural experiment)”, we need the theorems to successfully interpret the experiment.

    • Anon:

      There are settings where there are true zeroes, or nearly so, for example there can be a path for some biological or chemical process. But such a model makes no sense in the above example of beer etc. There, I think Sloman’s thinking is constrained by his narrow theoretical framework, leading him to make ridiculous statements.

  6. Interestingly, cognitive psychologists like Richard Nisbett have studied the general thinking patterns between cultures and found some striking differences. His book The Geography of Thought may be enlightening to probe where dichotomous statistical thinking arises from. I’d be interested in surveying Eastern and Western scientists to see if statistical biases and dichotomous thinking are more or less pronounced…

    https://en.m.wikipedia.org/wiki/The_Geography_of_Thought

    • Ironically, the section “Reception” in the Wikipedia article says:

      Cultural anthropologist Sherry Ortner wrote a critical review in The New York Times […]. She was most critical about his “relentless attempt to cram everything into the Asian/Western dichotomy…into these monolithic units of East and West” […]

  7. Making everything continuous is just quibbling over things that aren’t important.

    It’s like Daniel’s argument about voting age. It may or may not be more fair. But it’s definitely irrelevant. The harm done but disenfranchising a 17.9972 year old for one extra election is zero.

    As far as research goes, whether or not you need continuous or discrete data depends on what you’re doing.

  8. Just read this article today linked from YCombinator. This week I published an article about how people are literally lying to themselves about reality based on political inclinations. Your article actually may help explain what I have termed “self-gaslighting.” If you have a moment to read my article and tell me your thoughts, I’d appreciate it. I use probability to explain outcomes in it.

    https://thelibertarianrepublic.com/the-self-gaslighting-of-partisan-america/

    • For all 49 Democrats to believe Kavanaugh was lying, given these numbers, we arrive at:

      0.4 (40%) to the 49th power (based on 49 Democrats.)

      This assumes the privately held beliefs of democratic senators are all independent of each other.

    • When attempting to broach this subject with partisans, this response will often be encountered: “That’s false equivalence, both parties are not the same!” However, that is, in itself, a false reply. It is eminently clear that both parties stand for different causes and have different methods of doing things.

      Democratic Party: Make the government bigger and raise taxes on “the rich” to pay for it (which doesn’t happen)
      Republican Party: Make the government bigger without raising taxes to pay for it

      Same end result.

      I say inflation adjust the income tax brackets to align with the original intent. The lower 95% (about $200k per year in 2019 dollars, $3k per year in 1913 dollars) shouldn’t even file an income tax. This will effectively give everyone a 15-25% raise.

      At the same time, cut the Federal Budget by 40%. End all the “wars”. No more “war on drugs”, “war on terrorism”, “war on cancer”, “war on poverty”, etc. All of those are just self-perpetuating policies that create more of whatever they are supposedly “fighting” against.

  9. I tend to think continuously rather than discretely, and often find it irksome when people insist on presenting things discretely when I see them as continuous. I am not sure but guess that continuous thinking is less common than discrete thinking. I am wondering if there has been any research on this: e.g., developmentally: At what ages do kids start to show signs of one type of thinking or another? Or genetically? (e.g., this would be a good thing to study in identical-twins-reared-apart). One specific thing I wonder about is if continuous thinking is linked with a tendency to think spatially.
    I am wondering if any of these have been (or might be, with prompting) addressed on the Sister(‘s) Blog?

  10. Good public policy and public health policies are continuous yet their wave functions always collapse into people. People want to know “do I qualify or not?” and government agents need to decide “does she qualify or not?” So who are we trying to help here? A person or the politeia? Such arguments lie at the heart of all politics. Now, I’m terribly tempted to fly my Zeno of Elea / David Hilbert finitist Freak Flag and say that the continuum is just a model (and a false one at that) that works pretty well at scale but fails at quanta, but I won’t. Because reasons.

    • Thanatos — Right, and to flip it around, as I know you know (this is mainly for other readers), lawyers can be counted on to argue that a given legal standard is ‘incoherent” or “unworkable” because it doesn’t deal well with edge cases. But there will always be edge cases. Fuzzy boundaries. Those are the hard ones. Society draws dividing lines across continua to make enforceable rules. I argued the analogy in a brief once that the fact that a beach shifts constantly doesn’t render the concepts of “land’ and “sea” ambiguous or unworkable. I lost.

      • Btw I am not in any way arguing against Andrew, which I’m unqualified to do anyway. I share his frustration with dichotomization in research questions. I’m simply pointing out a rhetorical strategy that relies on the same sense that dichotomies “should work” in the real world, whereas of course they do not work at the boundaries.

      • Apologies for the delay in responding. I’ve been painting my new fences (/e silently curses those who outlawed arsenic in fence posts).

        Yes, the courts have for 20+ years refused to embrace uncertainty. Here’s an example. Once I tried to put on an expert who was of the following opinion: “The other side’s expert has failed to rule out a host of other evidence that indicates chemical “X” isn’t a cause of cancer.” The federal judge asked “Ok, so what caused it then?” My expert said “We can’t be sure; all I can say is that the evidence against chemical “X” is weak and serially contradicted.” The judge ruled my expert’s testimony inadmissible. “Mr. Savehn, if you deny chemical “X” caused the plaintiff’s cancer then you must by expert testimony establish what did.” This sort of thinking brought dichotomania to my world.

        • In reading journal articles, I am struck by the number of definitional variations and in different contexts. I know that there has been a call for standardization of terms. However is that being implemented? Would it improve statistics? Some terms have many contradictory connotations, as for example ‘uncertainty’ ‘compatibility, etc.

        • I talked to a cancer researcher / grad student once who told me that “everything causes cancer”.

          I had a professor once who was called as an expert witness, but said that his testimony was thrown out as “too technical”.

  11. Here, this kills Dichotomania:

    https://arxiv.org/abs/1904.06605

    Victor Coscrato, Luís Gustavo Esteves, Rafael Izbicki, Rafael Bassi Stern — Interpretable hypothesis tests

    Abstract:

    Although hypothesis tests play a prominent role in Science, their interpretation can be challenging. Three issues are (i) the difficulty in making an assertive decision based on the output of an hypothesis test, (ii) the logical contradictions that occur in multiple hypothesis testing, and (iii) the possible lack of practical importance when rejecting a precise hypothesis. These issues can be addressed through the use of agnostic tests and pragmatic hypotheses.

  12. It is worth mentioning that from a practical modeling standpoint, it is better to start with a simple model, get it working, get it in production, get feedback from stakeholders, etc. Then gradually increase model complexity so long as it improves the loss function. This could apply to starting with discrete variables, then gradually switching back to the more nuanced continuous variables they are derived from.

    You could say this about other sources of model complexity to, such as including the dependence between covariates.

    If one has any experience in Stan or a similar probabilistic programming language, then they know that diving right into a more complex model that may be closer than others to ground truth is a recipe for headaches and hair pulling.

    As an example, one might find that using a tree-augmented Bayesian classifier (which models dependence between predictors) does not provide better predictive performance than a naive Bayesian classifier (which assumes predictors are conditionally independent). If all you cared about were predictive performance, the simpler model would be preferable. They same could be true of a dichotomous vs continuous model, depending on the loss function.

Leave a Reply to Justin Cancel reply

Your email address will not be published. Required fields are marked *