Skip to content
 

DAGS in Stan

Macartan Humphries writes:

As part of a project with Alan Jacobs we have put together a package that makes it easy to define, update, and query DAG-type causal models over binary nodes. We have a draft guide and illustrations here.

Now I know that you don’t care much for the DAG approach BUT this is all implemented in Stan so I’m hoping at least that it’ll make you go huh?.

The basic approach is to take any DAG over binary nodes, figure out the set of possible “causal types”, and update over this set using data. So far everything is done with a single but very flexible Stan model. The main limitations we see are that it is working only with binary nodes at the moment and that the type space blows up very quickly making computation difficult for complex models. Even still you can do quite a bit with it and quickly illustrate lots of ideas.

In Berlin we were also talking about case level explanation. If of interest this piece figures out the bounds that can be obtained on case level effects using mediators and moderators. Different to the approach you were discussing but maybe of interest.

Any comments welcome, as always (including a better name for the package)!

I followed the second link, and in the abstract is says, “We are now interested in assessing, for a case that was exposed and exhibited a positive outcome, whether it was the exposure that caused the outcome.” This doesn’t seem like a meaningful question to ask!

But maybe some of you will feel differently. And, as Macartan says, their method uses Stan, so I’m sharing this with all of you. Feel free to download the package, try out the methods, comment, etc.

16 Comments

  1. Macartan says:

    Thanks Andy!

    You write:

    > “We are now interested in assessing, for a case that was exposed and exhibited a positive outcome, whether it was the exposure that caused the outcome.” This doesn’t seem like a meaningful question to ask!

    Our phrasing could be better but what’s meant here is “whether the exposure caused the outcome” and not “whether the exposure was the only cause of the outcome.” Or more simply, imagine X were randomly assigned and we see a case with X=1 and Y=1, we ask “would Y have been 0 if X were 0?”, or “is Y=1 due to X=1?” We have formal definitions in Section 3.1 in the paper (see also discussions in Pearl https://journals.sagepub.com/doi/full/10.1177/0049124114562614).

    Wonder if that still sounds not meaningful (as opposed to unanswerable), and if so why?

    • Anoneuoid says:

      I think Andrew meant that the answer to your question is: “yes”

      Every event is collectively caused by everything that happened earlier. Also, everything is correlated with everything else. If you accept those principles a lot of applied statistics starts looking like pointless pedantic calculations.

      Instead, you should be looking at the relationship between x and y, coming up with explanations for why it is so, and figuring out how to best distinguish between the various possibilities. Either that, or figuring out how to best predict y from the information available to you (eg machine learning).

      There is plenty of useful statistics to be done there.

      • Garnett says:

        “Every event is collectively caused by everything that happened earlier.”

        I *suspect* that this statement will seem like nonsense to those rooted in the DAG/Causal Mediation paradigm.

        • Perhaps, but it is basically true. I mean, at time t+dt whatever happens does so because the state of the world was whatever it was at time t…. it’s all a big Lagrangian or whatnot. Even quantum mechanics can be described as a deterministic nonlocal phenomenon (Bohmian)

          I realize this isn’t a useful way to describe say “romantic infatuation” or “voting” or “monetary policy” but it is nevertheless true.

          Instead of asking “did X cause Y” a better question is something like “which causes had an appreciable effect on the outcome Y”, another way to say this is rather than “there is an arrow” or “there isn’t an arrow” think “how strong is the arrow?”

          • Michael Schwartz says:

            Seems like superdeterminism in a nutshell (https://en.wikipedia.org/wiki/Superdeterminism). Barring a supernatural being, or fundamental laws of nature we don’t yet know, it seems reasonable to me.

            The issue really seems like what preceding events have measurable/meaningful effects(i.e. which of the world’s states should we try to modify at time t in order to impact the world at t+dt).

          • Anoneuoid says:

            Instead of asking “did X cause Y” a better question is something like “which causes had an appreciable effect on the outcome Y”, another way to say this is rather than “there is an arrow” or “there isn’t an arrow” think “how strong is the arrow?”

            I’d go a step further and ask “why is this arrow as strong as it is?”

        • Macartan says:

          Thanks Garnett. I think also for people using the potential outcomes framework it’s a bit hard to make sense of the claim that everything matters for everything else. Certainly at the case level. 

          Consider the binary world we focus on in the paper. With two potential causes, X1, and X2, we might consider potential outcomes of the form Y(X1, X2). The claim that both X1 and X2 always matter for a given outcome, whatever it is, implies Y(0,0) != Y(1,0) != Y(1,1) and so, for Y binary, we must have Y(0,0) = Y(1,1). So the claim that, at the case level, everything matters for everything requires strong and unusual restrictions on causal relations. The restrictions are weirder still with more potential causes.

          I take it though that this and the comments that follow are really about understanding population models, which is something else. The attribution question (which we focus on) is not about *whether* there is an arrow but about case level inference. These other questions, around how strong a causal relationship is in a population or how a relation operates, are different questions, but the good thing is that learning about these things is exactly what gives a handle on the attribution problem.

          • for example you choose 1,2,3,4,5,6 as your lottery numbers and they draw 6,5,4,3,2,1 and of course order doesn’t matter… so which of these balls “caused” you to win the lottery?

            Even at an individual level, whether you combed your hair right or left had an effect on whatever you are measuring. for example perhaps if you had combed your hair right instead of left it would have fallen down over your eyes during a right turn in your vehicle causing you to be unable to see the parked car you would have slammed into… making you miss your doctor’s appointment and causing you to not get your meds in time to save you from a serious adverse reaction to a bee sting leading to death…. or not.

            whatever your model is, it’s a far oversimplification of such things, so it doesn’t represent the effect of combing your hair right… so you call this stuff it doesn’t represent “random”

            if such random stuff is unimportant for your process it can only be because you control an experimental environment where such things can’t happen… move outside the controlled experiment and suddenly hair and centrifugal forces and parked cars and allergies and beestings all combine to cause everything.

            • Garnett says:

              Macartan and (Daniel and Anoneuoid):
              Thanks for taking the time to comment. It seems that you are not even speaking the same language, but I strongly believe that everyone would benefit from a healthy discussion on this issue.

              Macartan: Do you hold the view that “cause” is something that can be discovered/identified through suitable data analysis?

              Daniel and Anoneuoid have provided several helpful posts for understanding their points of view: e.g.

              https://statmodeling.stat.columbia.edu/2019/08/18/i-feel-like-the-really-solid-information-therein-comes-from-non-or-negative-correlations/

              • To be fair, I think the idea of causal models is still a realistic thing… because although there are lots of causes for every outcome, some of those causes are consistent and have large effects, and other causes are time-varying, fluctuating, context dependent, etc. I see science as the search for rules that fairly consistently provide reasonably good guidance to predict what will happen if we try to do something, and to quantify what is “left over” in terms of the uncertainty of that prediction is also important.

                But asking “did X cause Y” is not the right question, it’s more like “how much did X affect the outcome Y and why was the effect as large as it was in this context, and how large could we expect it to be in other contexts of interest vs how much the other myriad inconsistent effects would alter the outcome?”

              • Garnett says:

                Daniel:
                Am I correct in interpreting your use of the word “context” to include statistical model specification (among other things)?

              • Ideally you’d specify a model that identified the consistent effect of the variables of interest… Meaning that whatever context dependencies which are identifiable and of “significant size” are included explicitly in the model… all the highly context sensitive stuff whose context variables are unknown go into the “error” term (like, combing your hair right is context sensitive on you being in a rush, going around a curve, having a parked car sticking out a little too far, happening to be allergic to bees, having a bee sting you when you get out of the car… all put together)

                But in a more realistic setting where things become apparent through time, yes, you may realize later that there are context dependencies that you can identify which weren’t in your earlier model… and so respecifying your model to include them will alter the estimates that you got from the old model.

              • Anoneuoid says:

                It seems that you are not even speaking the same language, but I strongly believe that everyone would benefit from a healthy discussion on this issue.

                It is just reasoning from different premises. One set is wrong and constantly leads to the wrong conclusions (or in the case of DAGs has never been shown to predict anything). The other generated all the benefits we see around us today.

  2. Tommy says:

    Dags? Oh dogs! Sure, I like dogs.

Leave a Reply