Dean Eckles sent an email, subject line “PPNAS: learning math changes your brain,” saying:
Nice example of a paper with a mix of obvious, perhaps useful, and somewhat absurd.
Great commentary by Thomas Lumley.
Part of what I thought you might like is the availability of the raw data, albeit without any analysis code, perhaps making it difficult for non-fMRI experts (well and them too) to readily reproduce the results.
He adds:
Overall I’d say I’m not an expert so can’t fully evaluate this. But seems like a good example.
“Overall I’d say I’m not an expert” . . . that’s pretty much my motto!
P.S. Note the title above. We’re not saying this project seems all bad. It’s a mix! One problem with the system we have of scientific publishing is that the good and the bad are all mixed together. It’s not enough to publish data and some speculations; you’re also supposed to include statistical analysis. Not every research group has the capabilities to do all this well, but they’re essentially required to do it all in any case. I’d prefer more of a division of labor.
Re: “perhaps making it difficult for non-fMRI experts (well and them too) to readily reproduce the results.”
Here’s a great study about how well (or not) even fMRI experts can replicate results based on identical datasets: https://www.nature.com/articles/s41586-020-2314-9 (Upshot: asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses …. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests.)
Well, that “great study” with meta analysis based on 70 teams is itself controversial: https://www.biorxiv.org/content/10.1101/2021.05.09.443246v2
The “difference in the brain” stuff is classic 90s/00s cognitive neuroscience. Good to see there are still folks carrying on that tradition.
I see the bigger issue as the fundamental logical error of trying to ascribe causation with purely observational data (including another classic, throwing an arbitrary set of predictors into a regression and claiming causation when one is “significant”). All they can really say is that students who decided not to keep taking math are also students who may have lower concentrations of GABA in the regions they looked at.
So it’s not even, “learning math changes your brain”, because that causal claim cannot be supported by the data they have. It’s “different people who make different choices have different brains”.
> All they can really say is that students who decided not to keep taking math are also students who may have lower concentrations of GABA in the regions they looked at.
They tested for this, and it looks like GABA was not correlated with the choice.
The quoted section was part of their attempt to establish a causal relationship between subsequent math exposure and GABA, by saying that there was not a significant difference prior to the choice. This is the relevant section earlier which is the basis for their claim of the relationship between math and GABA:
> We then used GABA and glutamate concentrations in the MFG and IPS to classify students based on their present lack of math education (nonmath students vs. A-level math) using a binary logistic regression. Lower MFG GABA concentrations increased the likelihood that a student lacked math education rather than continued their math education [Fig. 2D, n = 83, standardized beta(β) = −0.3, P = 0.009, Exp(pβ) = 0.524, Exp(β) = 0.742].
Their whole argument is therefore based on this: They fail to find a significant GABA predictor early and do find a significant GABA predictor later. So that’s another error, treating a difference between significant and non-significant as significant.
So a more precise statement of their claim is as an interaction, “students who chose not to study math may eventually have different GABA concentrations in the regions we looked at after a while.” There is still no way to support a causal claim unless the lack of math experience was the only difference between groups and this is not well established.
Not really on topic but I am quite taken with the seeming correlation (or lack) of numeracy, logic, and personal worldview. Numeracy says that to draw useful inferences from observations you must count (broad view), integrate like with like (only), scale appropriately and only then begin to infer meaning. Basically that is what differential equations say (and more). Basically rules of logic. Does this view result mainly from education or is it innate to some extent? I suspect that it is the latter, e.g. in social context some deliberately refuse to look at the big picture, ascribing results to single causes, conflating effects with virtue etc.
I have a very smart/accomplished retired doctor friend with physics envy (arts premed) who has taken calculus as a hobby and reads pop science books. He exhibits no credence in the value of recognizing boundary limits e.g. or that calculus alone is simply necessary procedures for further application. Socially he is very much the single cause believer and value of effects depend on actor’s virtue etc. That is not a minority view.
As for education I am skeptical that high school math has a significant impact on understanding or brain development. It may be counter-productive in some ways. Algebra is too abstract absent application (e.g. matrices) and is perhaps confusing as it implies that it’s OK to sum X’s and Y’s which do not appear to be like things (in fact are in that context).
How is this sentence:
“Learning changes the brain”
at all different from this one:
“Learning changes the mind”
The second one cannot be counted as a discovery, not insofar as the categories to which it refers, “learning” and “mind” are linked by virtue of the very definition of “mind”. A “mind” (as opposed to say an apparatus composed of levers and gears) is — as part of its definition — is something which “learns”; and “learning”, along with “thinking”, is a defining property of “minds”.
A “brain”, on the other hand, is something that can sit in a jar; or in a skull. Minds think, brains live or die; vegetate or are pickled. Let’s put it by way of a (close) analogy: ones and zeros are properties not of silicon gates on chips; but of our interpretations of their physical states. Computer programs are not properties of machines; they are our interpretations of sequences their physical states.
The first statement is tossed around by the as if it were an exciting tidbit, as though they’d stumbled upon something remarkable; by the over-educated chatterers who think by their chattering that they have won the race, leaving the dunce Aristotle and the rest of the dusty antique thinkers far back in their wake.
As it happens Aristotle published a great deal more rubbish than the brain-chatters can conjure up in two generations, let alone thirty; and a good part of his speculative and somewhat tentative conjecture isn’t even rubbish. But calling “minds” “brains” and chalking it up as a “scientific” coup comes under his chapter on sophistical refutations; that is, how one might make a jury’s head spin, by playing with words.