Madison Fansher, Tyler Adkins, and Priti Shah write:
Media articles often communicate the latest scientific findings, and readers must evaluate the evidence and consider its potential implications. Prior work has found that the inclusion of graphs makes messages about scientific data more persuasive (Tal & Wansink, 2016). One explanation for this finding is that such visualizations evoke the notion of “science”; however, results are mixed. In the current investigation we extend this work by examining whether graphs lead people to erroneously infer causation from correlational data. In two experiments we gave participants realistic online news articles in which they were asked to evaluate the research and apply the work’s findings to a real-life hypothetical scenario. Participants were assigned to read the text of the article alone or with an accompanying line or bar graph. We found no evidence that the presence of graphs affected participants’ evaluations of correlational data as causal. Given that these findings were unexpected, we attempted to directly replicate a well-cited article making the claim that graphs are persuasive (Tal & Wansink, 2016), but we were unsuccessful. Overall, our results suggest that the mere presence of graphs does not necessarily increase the likelihood that one infers incorrect causal claims.
A paper by Wansink didn’t replicate??? Color me gobsmacked.
Hey! Wansink! That’s the dude who created the perpetually refilling bowl of cereal! :)
Hmmm.
Seems to me the test of whether graphs lead to erroneous inference of causation from correlation is to use a test condition where you first assess that no inference of causation is being assumed, and then show a graph of correlation, and then assess whether causation has been assumed.
That, it seems to me, is a different question than asking whether graphs increase an inappropriate conflating of causation with correlation when a text already evokes such a conflation.
I gotta admit that I’ve got strong “prior” to reject the contention that: “Graphs do not lead people to infer causation from correlation”
But maybe I’m just conflating causation and correlation, because I’ve seen so many people so many times use graphs to try to convey causation when the graphs merely illustrate correlation.
The study by Wansink probably didn’t replicate because it’s bunk, but I would suspect that the propensity of graphs to prompt an incorrect inference of causation could change with time. It seems like “correlation does not imply causation” is a message that is spreading, so 5 years later, less inference of causation could plausibly be due to greater awareness. If the study was written by anybody else, I would consider the two an interest snapshot into the evolution of public thought about statistics. Unfortunately, due to Wansink’s reputation, we’ll never know.
> correlation does not imply causation” is a message that is spreading,
With the caveats that 1) the degree of spread would be really hard to measure and, 2) I have no basis for an opinion other than pure conjecture…
I would disagee along a particular axis. I think that on issues where there is a moderate- to large-degree of polarization, it doesn’t seem to me that there’s a decrease in conflating correlation and causation and in fact if there’s any change over time it would be towards an increase. I think the trend towards polarization increases the kinds of biases that “motivate” conflating correlation and causation and if I had to guess, the explosion of social media – which I’d guess dramatically increases the number of graphical representations most people are exposed to on a regular basis, as well as the number of graphs being used to conflate correlation and causation that people are exposed to – interacts with greater polarization to create a trend of increase in the phenomenon despite perhaps more messaging about the fallacy problem.
My reading of the original post is that the quoted article is concerned with false conclusions about correlation, but that the Wansink article was about graphs being _persuasive_. Basically, the blokes who wrote the quoted article assumed Wansink was reasonable, wanted to go one step further and show that not only were graphs persuasive, but that they can also sometimes persuade folks of something that’s problematical.
But the quoted artticle’s conclusion is wacky, in that what they actually ended up showing (by doing the work of showing that Wansink’s article failed to replicated) was that graphs don’t do squat. But they still insisted on talking about the nonexistence of the incorrect inferal of probably nonexistent causality. (Thus becoming a pitiful, frantic, and desparte attempt to rescue a bad idea into which they put a lot of work, only to find that it was based on a foundation of quicksand.)
To further beat a dead horse into even higher qualirty glue (hmm: am I the only one here who has actually used horsehide derived glue?), they were trying desperately to rescue something, anything, from an idea that was based on something that was dead wrong. IMHO, they should have stopped talking about “the likelihood that one infers incorrect causal claims”.
I’m torn between giving our heros here credit for fixing the scientific record (good work guys, thanks!), having sympathy for them for being backstabbed (by the bogus article) into doing the work of thinking up an idea, setting up and testing that idea whereas all along it was hopeless, and being irritated with them for not simply saying, oops, our idea was bad from the start (which it was).
“It seems like “correlation does not imply causation” is a message that is spreading,”
Really? There’s a whole branch of computer science that’s dedicated to the belief that correlation does imply causation, and that furthermore the use of that implication is adequate to reproduce and exceed human intelligence.
David –
> Really? There’s a whole branch of computer science that’s dedicated to the belief that correlation does imply causation, and that furthermore the use of that implication is adequate to reproduce and exceed human intelligence.
That’s interesting point. Indeed, the question of whether correlation implies causation could be considered a semantic question.
The real problem, imo, is whether correlation is conflated with causation. Or the question of at what point does “implying” causation become conflating the one with the other.
Here’s an article on point, the type of article to which access has exploded with social media, where I think “implication” becomes equated to conflation:
https://dailysceptic.org/2022/10/27/mrna-vaccines-injure-the-heart-of-all-vaccine-recipients-and-cause-myocarditis-in-up-to-1-in-27-study-finds/
Whether graphics impede or accelerate the conflation of correlation with causation will depend on a lot of factors. First: a low (but “statistically significant”) correlation would, in a typical graph not look very correlated and hence not very causative, no matter what the statistical significance was.
Second: where the correlation is obviously induced by a jointly causative factor, the graph, to the extent that it gets people to focus on what they’re looking at, might well reduce the conflation. So no graph of football games played vs. ambient temoerature in Green Bay WI is going to convince people that playing football makes it cold, or that cold weather causes football games.
Third: On the other hand, where there is some underlying causal theory, accurate or not, the graph might well convince someone in a way that a regression would not that the theory was true. But it’s the causal theiry that really convinces, not the graph alone.
One of those conflating factors should involve the subject matter – what correlation is being described/portrayed. It says a “realistic online news article.” In this age of polarized everything, I’d be interested to know what sort of news article was involved, how large the sample was, who were the participants, etc. However, I don’t have access to the publication (which should have some answers). Can anyone access it and tell us?
The question as to whether graphs make scientific claims more persuasive to the general public than the scientific claim by itself is a different one than whether graphs reinforce an assumption of causality in a scientific claim. Unless I misunderstood the purpose of the Wansink study, if the study about persuasion did not replicate, it doesn’t follow that graphs do not increase assumptions of causality.
I see a few different types of claims that could affect results in the causality question. Let’s say they test the classic “ice cream consumption is highly correlated with murder rates” claim. Most people would probably not infer causality after seeing a murder vs. ice cream chart. In other words, they wouldn’t tend to assume causality more after seeing a graph/relationship that doesn’t feel causal in the first place. Yet, seeing the graph of a relationship they believed was causal prior to the study might make them feel “more right” about their set beliefs.
It seems to me that the degree of causality inferred from a graph would depend on the credibility level, on whether the causal relationship is “true”, on whether the claim fits within a coherent narrative, and on prior beliefs. An interesting space to test (imo) would be claims that were not already believed by the respondent but that fit a compelling narrative, and whose “true” level of causality unclear. The type of claims found in pop science/freakonomics-like books, like Levitt’s claim that abortions reduce criminality. I don’t think most participants would come in with a preconceived idea about this, but there’s a simple narrative easily weaved from the claim. Let’s say respondents get to read a small paragraph describing the relationship solely in correlational terms, with some information about the socio-economic status of women who get abortions. I wonder if the group that read the text and see a graph would be more likely to come out believing the relationship crime vs abortion is causal.
It’s an interesting question.
For readers who have not been following this forum a long time: see https://www.vox.com/science-and-health/2018/9/19/17879102/brian-wansink-cornell-food-brand-lab-retractions-jama
Hey! Wansink! The guy with the refilling soup bowl…no, wait…cereal bowl…no soup…no! cereal! Maybe that’s it! The soup-cereal correlation is 1.0!! Cereal causes soup!! No wait, soup causes cereal! Dangit! Phil! Help!
Sample size was too small then. Whenever you find “no correlation/effect”, that is the correct conclusion. NHST doesn’t magically start working when it is used to debunk Wansink.
They could instead tell us the approximate range of the effect, but this potentially useful info did not make it into the abstract for some reason.