The people who go by “Slime Mold Time Mold” write:
Some people have noted that not only does correlation not imply causality, no correlation also doesn’t imply no causality. Two variables can be causally linked without having an observable correlation. Two examples of people noting this previously are Nick Rowe offering the example of Milton Friedman’s thermostat and Scott Cunningham’s Do Not Confuse Correlation with Causality chapter in Causal Inference: The Mixtape.
We realized that this should be true for any control system or negative feedback loop. As long as the control of a variable is sufficiently effective, that variable won’t be correlated with the variables causally prior to it. We wrote a short blog post exploring this idea if you want to take a closer look. It appears to us that in any sufficiently effective control system, causally linked variables won’t be correlated. This puts some limitations on using correlational techniques to study anything that involves control systems, like the economy, or the human body. The stronger version of this observation, that the only case where causally linked variables aren’t correlated is when they are linked together as part of a control system, may also be true.
Our question for you is, has anyone else made this observation? Is it recognized within statistics? (Maybe this is all implied by Peston’s 1972 “The Correlation between Targets and Instruments”? But that paper seems totally focused on economics and has only 14 citations. And the two examples we give above are both economists.) If not, is it worth trying to give this some kind of formal treatment or taking other steps to bring this to people’s attention, and if so, what would those steps look like?
My response: Yes, this has come up before. It’s a subtle point, as can be seen in some of the confused comments to this post. In that example, the person who brought up the feedback-destroys-correlation example was economist Rachael Meager, and it was a psychologist, a law professor, and some dude who describes himself as “a professor, writer and keynote speaker specializing in the quality of strategic thinking and the design of decision processes” who missed the point. So it’s interesting that you brought up an example of feedback from the economics literature.
Also, as I like to say, correlation does not even imply correlation.
The point you are making about feedback is related to the idea that, at equilibrium in an idealized setting, price elasticity of demand should be -1, because if it’s higher or lower than that, it would make sense to alter the price accordingly and slide up or down that curve to maximize total $.
I’m not up on all this literature; it’s the kind of thing that people were writing about a lot back in the 1950s related to cybernetics. It’s also related to the idea that clinical trials exist on a phase transition where the new treatment exists but has not yet been determined to be better or worse than the old. This is sometimes referred to as “equipoise,” which I consider to be a very sloppy concept.
The other thing is that everybody knows how correlations can be changed by selection (Simpson’s paradox, the example of high school grades and SAT scores among students who attend a moderately selective institution, those holes in the airplane wings, etc etc.). Knowing about one mechanism for correlations to be distorted can perhaps make people less attuned to other mechanisms such as the feedback thing.
So, yeah, a lot going on here.
By “correlation” here, do you mean a linear relationship, or any relationship (like a U-shape)?
This is what Scott Sumner is talking about when he says ‘never reason from a price change’. For example the quantity traded of a good is uncorrelated with the price. If the price rose that could be because supply fell, in which case the quantity traded would also fall, but it could also be because demand rose, in which case the quantity traded would rise.
In the previous thread with the “confused comments,” Jim wrote:
“If [you] change the accelerator pedal [setting] to a metered valve [setting] and plot speed, resistance and flow in real time, there will be some sort of correlation. But the point is that a standard accelerator pedal and standard gages render the causation invisible, and in the real world many causes are hidden by similar indirect linkages.”
The point that the statisticians want to drive home is that in a world where correlations can be invisible, which includes all cases where we are working back from incomplete observational data, there can be cases where there appears to be causation without correlation. The examples prove that they are not wrong.
Later in the same thread, I was making the point that pedal setting correlates to velocity in a properly constructed work function, and “Anonymous” responded with:
“The problem with your argument is that correlation between “step on the accelerator” and “speed increases” has nothing to do with work. It’s just a correlation between two behaviors. IN effect, you’re cheating by including your knowledge of other parameters. These parameters may not be known or measurable in an analogous situation in the real world.”
…and that is correct (and in no way refutes the argument I presented). The problem is that it is impossible to understand or design a control system without “cheating” by knowing what is going on! If you are staring blankly at accelerator pedal setting vs. speed graphs trying to find a correlation, well let’s just say that you are never going to design a control system.
I don’t see where anyone was or is confused, despite numerous comments that others are WRONG! We are simply talking past each other. Yes, the statisticians are right, if you are feeling around inside a black box full of observational data, you can easily find pairs of variables that seem to be related through causation but not correlation. But yes, the technical folks who have weighed in are also right, if you have adequate knowledge of how the control system actually works, you will clearly see that the input signals correlate to the output signals though mathematical functions that you can write down if you know enough. Velocity is a nonlinear summation of throttle setting and rolling resistance via a discrete work function. Ignorance of how the system works doesn’t change that.
Thanks Matt, now I don’t have to say it.
I can just add basically that social science is *woefully* lacking in dynamic models.
I’d add that feedback loops can lead to opposite acute vs chronic effects.
Eg, a high-sodium meal (or short term diet) can raise blood pressure, while a chronically high-sodium diet lowers it.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7832857/
In fact, this type of acute-chronic reversal is what we should expect by default from systems with feedbacks.
“The stronger version of this observation, that the only case where causally linked variables aren’t correlated is when they are linked together as part of a control system, may also be true.”
i think the converse described here is false by th following counterexample. for a mean zero random variable X with a symmetric distribution, the correlation between X and X^2 is equal to zero. maybe you could say this is a control system, but then the term becomes meaningless. or maybe you could say X doesn’t cause X^2? (kind of like how people say 2 isn’t a prime number). anyways, selection is another good counterexample showing that it isn’t always control.
I have many examples of negative feedback destroying correlations.
Feedback loop of any predictive model, e.g. people who drive red cars tend to be have more auto insurance claims, and are thus less profitable. Insurer X learns this and either raises prices on red car drivers or avoid them. The next time you look, the correlation between red car drivers and claims has weakened. This applies to every predictive model for which action is taken on the predictions.
Here’s my favorite example. An ecommerce website discovers that people who have credit cards on file tend to purchase more compared to those who don’t. The marketing team runs a promotion giving people a coupon if they are willing to provide their credit card numbers. Now the proportion of customers who have credit cards on file went from 50% to 70%. What happened to the correlation between credit cards on file and purchase amount?
Interestingly, the opposite is also possible, i.e. feedback loop reinforces correlations. For example, Netflix’s model decides that you like horror movies featuring deep-water creatures based on other things you have seen and what other people who watched those other things have seen. Soon, everywhere you look on Netflix, you find horror movies featuring deep-water creatures. You may end up watching some of them. Suddenly, the correlation between you and horror movies featuring deep-water creatures increases.
I’m pretty sure this can be analyzed under the usual causal framework. There is correlation. There is action (i.e. intervention) that affects both variables of that correlation, directly or indirectly. The effect of the intervention can be any direction (positive or negative feedback).
Kaiser:
I agree that this can be understood with causal inference. The key for me is to think of the causal effects of actions rather than the causal effect of variables. So, if variables X, Y, and Z are in some feedback loop, you don’t think of the causal effect of X, say; instead you think of the causal effect of an action that can alter X (as well as doing other things).
I don’t think it’s true that the price elasticity of demand is negative one in an idealized system, unless it includes additional constraints I’m not thinking of. Under perfect competition, the elasticity can be anything (it just depends where S hits D). Under a single-price monopolist, the elasticity is just the price divided by the markup times -1, via the Lerner index.
Jason:
Yeah, I’m probably thinking of some special case.
>The stronger version of this observation, that the only case where causally linked variables aren’t correlated is when they are linked together as part of a control system, may also be true.
Another commenter has pointed out that non-linear relationships can destroy linear correlations.
Another example is the relationship between a variable and its derivative. If both are bounded, then their correlation must tend to zero over a time, and if they are periodic, the correlation over one period will be exactly zero. (The proof is simple, and one can be found in the appendix to https://arxiv.org/pdf/1505.03118.pdf.)
For a causal example of this relationship, consider a voltage source connected to a capacitor. As the voltage is varied, a current is produced, proportional to the rate of change of voltage. Alternatively, a current course connected to a capacitor will produce a voltage proportional to its integral. In one case the voltage is causing the current, in the other the current causes the voltage. In both cases, the correlation is zero.
I believe this sort of thing is well-known to people who deal with time series; perhaps less well-known outside that field. Even Brownian motion can confuse things: two independent Brownian motions with zero drift typically have a substantial correlation with each other. The “expected” correlation is of course zero, but the distribution of the correlation has a substantial spread that does not diminish with the number of samples. The autocorrelation is the problem: according to a simulation I once did it reduces the effective sample size to about 5 or 6.