Under the heading, “please blog about this,” Shravan Vasishth writes:
This book by a theoretical physicist [Sabine Hossenfelder] is awesome. The book trailer is here.
Some quotes from her blog:
“theorists in the foundations of physics have been spectacularly unsuccessful with their predictions for more than 30 years now.”
“Everyone is happily producing papers in record numbers, but I go around and say this is a waste of money. Would you give me a job? You probably wouldn’t. I probably wouldn’t give me a job either.”
“The story here isn’t that theorists have been unsuccessful per se, but that they’ve been unsuccessful and yet don’t change their methods.”
“And that’s the real story here: Scientists get stuck on unsuccessful methods.”
She deserves to be world famous.
I have no idea who deserves to be world famous, but Shravan’s email was intriguing enough to motivate me to follow the link and read Hossenfelder’s blog, which had some discussion of the idea that the quest for beauty in mathematical theories has led physics astray.
Hossenfelder also draws some connections between the crisis in physics and the reproducibility crisis in social and behavioral science. The two crises are different—in physics, the problem (as I see it from the outside) is that the theories are so complicated and so difficult to test with data (requiring extremely high energies, etc.), whereas in the human sciences many prominent theories are so ridiculous and so easy to refute that this creates an entirely different sort of crisis or panic.
Hossenfelder writes, “In my community [in physics], it has become common to justify the publication of new theories by claiming the theories are falsifiable. But falsifiability is a weak criterion for a scientific hypothesis. It’s necessary, but certainly not sufficient, for many hypotheses are falsifiable yet almost certainly wrong.” Yup. Theories in the human sciences are typically too vague to ever be wrong, exactly; instead, they are set up to make predictions which, when falsified, are simply folded back into the theory (as in this recent example). Rather than say I think a particular social science theory is false, I prefer to say that I expect the phenomenon of interest to be highly variable, with effects that depend unpredictably on the context, hence trying to verify or estimate parameters of these theories using a black-box experimental approach will typically be hopeless. Again, I can’t really comment on how this works in physics.
Back to beauty
I will not try to define what is beauty in a scientific theory; instead I’ll point you toward Hossenfelder’s discussions in her blog. I’ll share a few impressions, though. Newton’s laws, relativity theory, quantum mechanics, classical electromagnetism, the second law of thermodynamics, the ideal gas law: these all do seem beautiful to me. At a lesser level, various statistical theories such as the central limit theorem, stable laws, the convergence of various statistical estimators, Bayes’ theorem, regression to the mean, they’re beautiful too. And I have a long list of statistics stories that I keep misplacing . . . they’re all beautiful, at a slightly lower level than the above theorems. I don’t want to pit theories against each other in a beauty context; I’m just listing the above to acknowledge that I too think about beauty when constructing and evaluating theories.
And I do see Hossenfelder’s point, that criteria of beauty do not always work when guiding research choices.
Let me give some examples. They’re all in my own subfields of research within statistics and political science, so not really of general interest, but I bet you could come up with similar stories in other fields. Anyway, these are my examples where the quest for beauty can get in the way of progress:
1. Conjugate prior distributions. People used to use inverse-gamma(1,1) or inverse-gamma(0.001, 0.001) priors because of their seeming naturalness; it was only relatively recently realized that these priors embodied very strong information. Similarly with Jeffreys priors, noninformative priors, and other ideas out there that had mathematical simplicity and few if any adjustable parameters (hence, some “beauty” in the sense of physics models). It took me awhile to see the benefit of weakly informative priors. The apparent ugliness of user-specified parameters gives real benefits in allowing one to include constraining information for regularization. That said, I do think that transforming to unit scale can make sense, and so, once we understood what we were doing, we could recover some of the lost beauty.
2. Predictive information criteria. AIC was fine under certain asymptotic limits and for linear models with flat priors but does not work in general; hence DIC, WAIC, and other alternatives. Aki and I spent a lot of time trying to figure out the right formula for effective number of parameters, and then we suddenly realized that there was no magic formula. Freed from this alchemic goal, we were able to attack the problem of predictive model evaluation directly using leave-one-out cross-validation and leave the beautiful formulas behind.
3. Lots of other examples: R-hat, R-squared, all sorts of other things. Sometimes there’s a beautiful formula, sometimes not. Using beauty as a criterion is not so terrible, as long as you realize that sometimes the best solution, at least for now, is not so beautiful.
4. Five ways to write the same model. That’s the title of section 12.5 of my book with Jennifer, and it represents a breakthrough for us: after years spent trying to construct the perfect general notation, we realized the (obvious in retrospect) point that different notations will make sense for different problems. And this in turn freed us when writing our new book to be even more flexible with our notation for regression.
5. Social science theories. Daniel Drezner once memorably criticized “piss-poor monocausal social science”—but it’s my impression that, to many people who have not thought seriously about the human sciences, monocausal explanations are more beautiful. A naive researcher might well think that there’s something clean and beautiful about the theory that women are much more likely to support Barack Obama during certain times of the month, or that college students with fat arms are more likely to support redistribution of income, without realizing the piranha problem that these monocausal theories can’t coexist in a single world. In this case, it’s an unsophisticated quest for beauty that’s leading certain fields of science astray—so this is different than physics, where the ideas of beauty are more refined (I mean it, I’m not being sarcastic here; actually this entire post and just about everything I write is sincere; I’m just emphasizing my sincerity right here because I’m thinking that there’s something about talking about a “refined” idea of beauty that might sound sarcastic, and it’s not). But maybe it’s another version of the same impulse.
Why care about beauty in theories?
Let’s try to unpack this a bit.
Constructing and choosing theories based on mathematical beauty and simplicity has led to some famous successes, associated with the names of Copernicus, Kepler, Newton, Laplace, Hamilton, Maxwell, Einstein, Dirac, and Gell-Mann—just to name a few! Or, to move away from physics, there’s Pasteur, Darwin, Galton, Smith, Ricardo, etc. The so-called modern synthesis in biology reconciling Mendelian inheritance and natural selection—that’s beautiful. The unification of various laws of chemistry based on quantum mechanics: again, a seeming menagerie is reframed as the product of an underlying simple structure.
Then, more recently, the quest of mathematical beauty has led to some dead ends, as discussed above. From a sociology-of-science perspective, that makes sense: if a method yields success, you keep pushing it to its limits until it stops working.
The question remains: Is there some fundamental reason why it makes sense can make sense to prefer beautiful theories, beyond slogans such as “Occam’s razor” (which I hate; see here and here) or “the unreasonable effectiveness of mathematics” or whatever? I think there is.
Here’s how I see it. Rather than focusing on beautiful theories that make us happy, let’s consider theories that are not so beautiful and make us uncomfortable. Where does this discomfort come from? Putting on my falsificationist Bayesian hat for a moment (actually, I never took it off; I even sleep with it on!), I’d say the discomfort has to come from some clash of knowledge, some way in which the posterior distribution corresponding to our fitted model in not in accord with some prior belief we have. Similar to the Why ask why? problem in social science.
But where exactly is this happening? Let’s think about some examples. In astronomy, the Copernican or Keplerian system is more pleasant to contemplate than an endless set of spinning epicycles. In economics, the invisible hand of Smith etc. seems like a better explanation, overall, than “the benevolence of the butcher, the brewer, or the baker.” Ugly theories are full of moving parts that all have to fit together just right to work. In contrast, beautiful theories are more self-sustaining. The problem is that the world is complicated, and beautiful theories only explain part of what we see. So we’re always in some intermediate zone, where our beautiful theories explain some stylized facts about the world, but some ugliness is needed to explain the rest.
P.S. In response to the above post, Hossenfelder writes:
I think though you have some things backwards. Take the epicycles: People held onto these because they thought that circles were beautiful. Also, I do not consider Occam’s razor to be an argument from beauty. Besides, note that the context in which I am discussing problems with arguments from beauty is in the development of new theories. The ‘discomfort’ that you refer to (which I discuss in my book as an argument from experience) is something you can apply to when you refer to theories you know, but not if you want to develop new ones.
That’s interesting that epicycles could be considered beautiful. I suppose that the characterization of beauty will depend a lot on context.
Nice post! Had me thinking back to this wonderful essay by John Maynard Smith, trying to identify the most beautiful completely wrong idea in science (“too good to be true”). Not wrong in the sense that it’s only partially correct, or only correct in some contexts, or only an approximation to an uglier reality, but just flat out 100% wrong.
https://www.nature.com/articles/22238
Here’s something ugly in science: yesterday I noticed the Wikipedia page “Selective Sweeps” got JMS’ name wrong in two different ways. I tried to log in to Wikipedia for the first time in five years, and was immediately told I was blocked from doing anything on W starting last week, for two weeks. Some 23-year-old bedroom boy in Finland with a dozen multi-coloured stars to his name, has either obliterated a tranche of IP addresses including mine, for some imagined offence, or worse, is actually blocking me. Don’t underestimate the damage caused by Wikipedia. Might exceed the “damage” Sabine says is happening to physics.
Yes, the allocation of DNA codings to proteins which that failed beautiful theory didn’t make stick, made way for a more complex but satisfying arrangement with so much more potential for effectiveness.
I think it is inherently beautiful when a simple fact explains many different phenomena — that one force explains both elliptical orbits and the falling apple, that the hypothesis of light quanta explains blackbody radiation and the photoelectric effect, that group theory can explain so many different things (the unsolvability of the quintic, etc.).
So from that point of view, it seems reasonable for scientists to go chasing after beauty: simple stories that explain a wide variety of phenomena. “Beauty” might just be a name for a certain kind of mismatch between what you put in (hypotheses, models, equations) and what you get out (explanations).
This almost follows the path of physics over the last two centuries or so. Verification of the theory goes from relying on evidence that is easily verifiable by almost anyone, to more abstract evidence that requires understanding specialized equipment, to having no relationship to evidence at all.
I’ve been enjoying teaching the median voter theorem as a really elegant (if not beautiful) model that has also recently proved spectacularly wrong. I love being able to say “look at these few assumptions that seem to make sense, and then look at this clear, intuitive, deeply pleasing logic that falls out of that…. and then ask yourself if American politics seems to be moving towards the center these days.”
Now putting on my pragmatic/perspectival anti-realist hat (I never took it off!), I might say something like “since all models are false and the world inherently unrepresentable by mathematical expressions conceivable by human beings, we might as well choose attractive ones at least up until the point they prove to have less explanatory power than uglier, darker theories… and then we should keep using them where they work even once we know they are false.” I mean for real, Newton’s equations work just fine for most things regular earth-people need to compute, even if the metaphysics is totally wrong about the world. And if people had abandoned Ptolemy right when Copernicus came along, it would’ve been decades when no one knew where the stars would be or when there would be an eclipse (that would’ve been a problem for navigation too i suspect, but I don’t actually understand sextants.)
A sextant is a fancy protractor, it measures angles between two visible objects (when you look through the eyepiece you see a split view, on one side is the horizon the other side is some celestial object, like a star). A useful one to measure is the angle between polaris and the horizon. This angle is obviously 90 degrees at the north pole, and about 0 degrees at the equator. So it can tell you your latitude if you’re in the northern hemisphere. You could also measure say the angle between the sun and the horizon (if you have a sextant with an extremely dark solar filter, so you don’t burn out your eyes) which if you have an accurate clock could also let you calculate your longitude.
If you think of it as just a really fancy protractor you don’t go far wrong.
I’ve never read this guys book, but he claims you can do this without a clock as long as you have an accurate ephemeris for a reference longitude and can measure the angle between various stars and the moon well enough:
http://www.rexresearch.com/millercelt/millercelt.htm
Then he goes on to say the celtic cross with a plumbline would be a perfect, simple tool for this purpose. And further that the people who carried around and understood these tools were well-respected in their time. Eventually it devolved into cargo culting where carrying the cross represented political/social power. The original use was eventually lost. Then via folk etymology, telephone effect, etc a mythology formed to explain why all these crosses were around: https://en.wikipedia.org/wiki/Sun_cross
At least his first two claims seem to make sense and would be testable.
Beauty is in the eye of the beholder – but the eye can be trained.
Pierce’s training (I never take that hat off) led him to take aesthetics to be the topic of what you should value above all regardless of any ulterior purposes. The ultimate thing to value above all he took to be the grasping of reality as being reasonable (capable of being understood ). See here for more http://statmodeling.stat.columbia.edu/2017/09/27/value-set-act-represent-possibly-act-upon-aesthetics-ethics-logic/
From this we take as beautiful anything which accelerates getting less wrong about reality (which we never have direct access too). So ugly things would be those things which decelerate getting less wrong – such as arguments by authority (for instance, quoting Peirce as if he had the right answers or insight rather than possibly helpful ones).
This is a really great post, maybe even beautiful.
In economics the core micro model is widely regarded as comely, and whenever a phenomenon can be explained (I’m tempted to use scare quotes) by that model with nothing else thrown in, economists praise the “parsimony” (gave in) of this explanation. What’s striking is that, while the core model is based on just a few assumptions, they are gargantuan and largely contradicted by findings, or at least maintained assumptions, in adjacent disciplines. The whole consistent preferences/rational choice/utility bit flies in the face of what we know about psychology. The convexity assumption in preference and production sets (my favorite target) is grossly inconsistent with pretty much every other social science, ecology, management theory, etc.
The reason I bring this up is that beauty, in the sense of a parsimonious model that possesses wide explanatory power, can be deceptive. It’s not the number of the assumptions that matters but their reach and plausibility. You could say that the explanatory power of a critique of just a handful of standard economic assumptions is beautiful too. I feel that way about the critique of the convexity assumption.
There was a chapter on Stephen Hawking’s “A Brief history of time” in our English textbook, which we had when I was in the elementary school, and that sparked a great interest towards popular physics in young me. While devouring a vast selection of more and less popular classics from the available books in the local library, I always found it kind of strange that in many of those, the authors would indeed refer to the theories or calculations as “beautiful”.
Now, there’s nothing wrong in seeing beauty in things–I’d be delighted if people would be more open to beauty, aesthetics and so on–, but it has always seemed to me awkwardly self-congratulatory, maybe slightly… uh, I don’t know the word, but that this law isn’t just a pragmatic equation to equate what ever it is that we are in a dire need to equate, but it is also BEAUTIFUL–at which point I can imagine the writer beginning to almost float in the air, maybe shedding few tears: partly due to experiencing the beauty so strongly; partly because there are just too many people who are blind to what only the initiated can truly see and experience so deeply…
I’m sure no physicist really thinks about it that way; that is just the impression I’ve always seemed to get. I think it is just (from my point of view) awkward rhetorics. Maybe partly because–as someone else noted earlier–the sort of annoyingly pragmatic, instrumentalist influence has tainted my soul.
Just one another point… which is mayhaps slightly apropos in this blog, but in the linked blog–ah, gosh, I forgot the name of the writer, since I was waiting for a Stan model to compile for so long (using my travel laptop which is no good mrr)–the writer is concerned that the fanciful nature of e.g. multiversum/string theories is not sufficiently communicated to the reader. Having read a couple books by Greene, the string theory guy, and some by the multiverse fella, whose name has oozed out of my brain on to the pillow, I’ve never gotten the impression other than that these ideas are highly controversial/untested/should-not-be-taken-as-proved.
Ha… I happened upon this today too:
http://osc.centerforopenscience.org/2013/11/20/theoretical-amnesia/
Nice. I have a blog post on a similar theme: our statistics don’t suck, our theories do:
https://grasshoppermouse.github.io/2018/01/16/the-replication-crisis-our-statistics-don-t-suck-our-theories-do/
Ed:
I agree! What you say is related to what I said here.
Ed’s blog dovetails nicely with some of Paul Rozin’s critique of social psychology, implicating weak theories.
https://cpb-us-w2.wpmucdn.com/web.sas.upenn.edu/dist/7/206/files/2016/09/socpsysci195PSPR2001pap-25bdsu0.pdf
I think the quality of insights in various fields, for the last 30 years more or less, has been demonstrably weak really haven’t produced modern day equivalences of the originality of Dewey, Freud, Jung, Adler, Allport, Bruner, etc. Kahneman & Tversky quite possibly, although Tversky claimed that their research held out nothing really new. It’s that they laid them out systematically. Of the thinkers I’ve listed, I think Jerome Bruner and Kahneman & Tversky have brought to fore the most relevant research. I would also put Philip Tetlock’s work in that category of relevant thinking. Serge Lang as well, though a mathematician. His book Challenges was really thought provoking.
Dr. Gelman, perhaps this has been said before on this blog, but I always took Occam’s razor to mean not that parsimony per se is good, but that given two equally “good” (based on whatever criterion you please) models, the one with fewer assumptions is superior. In a way it seems that adding parameters to a model is making a certain kind of assumption.
Interested in your thoughts on the matter.
Jackson:
I’m with Radford Neal on this one; see here.
Occam’s Razor has been misused and abused in international relations field.
I’m somewhat skeptical of using ‘beautiful’ and ‘ugly’ in the manner they are being used here.
You argue that ugly models suffer a clash between prior and posterior knowledge. Do you agree that as a consequence their marginal likelihoods would be small relative to models that didn’t suffer such a clash? If it were, the intuition that ugly models are implausible would be supported by comparison of marginal likelihoods, i.e. Bayes factors.
Other Andrew:
No. Except in the case of fully generative models (which we rarely see), the marginal likelihood is close to uninterpretable because it depends on arbitrary aspects of the prior distribution that have essentially no effect on predictions; see for example the discussion in chapter 7 of BDA3 or this article.
Sorry about using your name here! I am somewhat familiar with your criticisms of Bayes factors. I don’t understand how you can reconcile those criticisms with your argument that it is reasonable to suggest ugly models are bad because of a clash between prior knowledge and the posterior.
Could you be more precise? What exactly should I calculate to show the clash of knowledge and/or that an ugly model is ‘bad’/implausible/unworthy of study?
Other Andrew:
I don’t think it’s reasonable to suggest ugly models are bad. I’m just trying to explore, statistically, what is meant by “ugly.” I don’t think so-called ugly models are bad/implausible/unworthy of study; what I think is that if we feel that a model is ugly, we should try to explore what exactly about is bothersome, as that might point us to prior knowledge or prejudices.
That hasn’t entirely cleared things up, since in your post you argue that there is a fundamental reason to prefer beautiful theories (and presumably therefore disfavor ugly ones):
Is there some fundamental reason why it makes sense to prefer beautiful theories, beyond slogans such as “Occam’s razor” (which I hate; see here and here) or “the unreasonable effectiveness of mathematics” or whatever? I think there is.
but you now say
I don’t think it’s reasonable to suggest ugly models are bad … I don’t think so-called ugly models are bad/implausible/unworthy of study
Can you understand why I find these two things slightly contradictory? By the first passage, I understand that beautiful theories should be preferred over ugly ones. From your response above, though, you don’t think it’s reasonable to disfavor an ugly model because it’s relatively ugly.
Other Andrew:
I guess it would be more precise for me to say “There can be a reason to prefer beautiful theories.” To put it another way: If you have a preference for a particular beautiful theory, you should interrogate that preference and try to get at the more fundamental reasons why the ugly theory seems bothersome.
Hmm. I’m actually more confused now than I was before – I don’t find your article and responses consistent. We began in the post with (I paraphrase) ‘there is a fundamental reason to prefer beautiful theories’, and we’re now at ‘the fact that a theory appears beautiful/ugly is worthy of further investigation/thought, but doesn’t indicate it should be favored/disfavored’. They are very different positions! Am I being unfair? Nevertheless, thanks for taking the time to reply.
Other Andrew:
I wrote: “Is there some fundamental reason why it makes sense to prefer beautiful theories. . . . I think there is.” Based on your comment, I will change “makes sense” to “can make sense.”
I do not see a contradiction in those two statements. In the first, Andrew describes some examples and rationale for the use of “beauty” as a criterion by which theories have been judged and perhaps continue to be. In the second he suggests criteria such as “Occam’s Razor” may lead us astray by eliminating the true model in favor of the simple model. In the case of Occam’s Razor, the “ugly model” would be the more complex, but closer to reality model.
> In the case of Occam’s Razor, the “ugly model” would be the more complex, but closer to reality model.
No, the really ugly model is the simpler one that decelerates moving to and learning with the model closer to reality.
I have no idea if Andrew agrees but [real] beautiful is what accelerates getting less wrong about reality and [real] ugly that which decelerates.
Now perceptions of [apparent] beauty are in the eye of the beholder…
Why do you think that epicycles are less beautiful? Isn’t there a fractal element to it which has quite a lot of beauty? I know the argument about parsimony, but isn’t that just a variation of Occam’s razor? So isn’t beauty very much a matter of perspective? If so, to what extent is “beauty” contingent on culture, e.g., western culture? How might e.g. native Americans judge this aspect of beauty?
Sometimes even “true” models deliver unreliable predictions because of the “beaty problem”: Copernicus assumed, like the Tolomeus system, that orbits were perfect circles, which would entail the same errors under the Copernican system that motivated Tolemaic astronomers to introduce epicylces
The previous issue of Nature has a book review on that book.
When someone says a thing is “beautiful”, it isn’t a statement about the thing, it is a statement about the person’s ‘beauty detection pattern recognition neuroanatomy’; that it is maximally activated by what ever this “beautiful” thing is. It is a feeling. It is an intuition that is ‘felt’ through the person’s neuroendocrine system, not something that is understood cognitively.
Until you actually understand what the thing is (and can describe it with facts and logic, in closed form, i.e. mathematically), that activation can only be a false positive (to which human pattern recognition systems are prone).
The analogy I like to use is from Nietzsche’s quote; “if you stare too long into the abyss, the abyss stares back into you”. What that means, is that when you stare at something, your pattern recognition neuroanatomy automatically tries to make what ever you are looking at fit into patterns that “make sense”. Even when you stare into an abyss, your pattern recognition neuroanatomy tries to find patterns in the abyss. Eventually it does, and you end up with a ‘pattern’ of the abyss in your neuroanatomy.
That is why you need to be “careful” (when looking at abyss-like things) and run that pattern recognition as an “emulation”, as a “virtual machine” (based on facts and logic via cognition), and not “native” (based on feelings and emotions in your neuroendocrine system). If you let it run “native”, you can lose track of what is “real” and what is spurious, because the neuroendocrine system doesn’t keep track of the “details”, so why something is “beautiful” can’t be checked (and rechecked).
This is what Sabine Hossenfelder is talking about when “beauty” is used as a criteria for what kind of physics to do. Beauty is in the pattern recognition neuroanatomy of the beholder. If your pattern recognition neuroanatomy tells you something is “beautiful”, unless you have defined criteria that you can cognitively analyze (with facts and logic), the “beauty” is just a feeling and could easily be spurious.
There is lots of faulty reasoning in many different kinds of fields. Sometimes newcomers to a field have greater perspective and can see the faulty reasoning, but they don’t have the social power to get anyone else to see it.
Social power frequently doesn’t have anything to do with being right, or having correct reasoning. Once fields get “taken over” by those who use social power to enforce what is “important” or “valid”, then those fields are doomed. This happens when resources in a field are allocated according to social power by “leaders” and not by other means. Power corrupts and absolute power corrupts absolutely.
It is not surprising that some of the most successful religions (the Abrahamic Religions) are set up as Patriarchies. They are top-down zero-sum social power hierarchies. Humans have a cognitive bias to try and find “the cause” and to then try to fit causation into a top-down causal hierarchy.
They are “successful” in the sense that the person (virtually always a man) at the top of the social power hierarchy has a lot of power and people defer to that person because he can increase their social standing and power. Feynman’s discussion of Cargo Cults bears on this. The “leader” of a cult is always the most successful person in the cult, even if the cult only redistributes cult member resources and doesn’t generate anything productive.
This seems to be the way to actually evaluate these things, where the priors are explicitly included in the Occam’s razor analysis.
https://arxiv.org/abs/1701.00213
Interesting paper in the link – maybe I’ll try to put into the context of these: http://statmodeling.stat.columbia.edu/2017/11/29/expediting-organised-experience-statistics/ and http://statmodeling.stat.columbia.edu/2017/09/27/value-set-act-represent-possibly-act-upon-aesthetics-ethics-logic/
Some good points here.
Re: Sometimes newcomers to a field have greater perspective and can see the faulty reasoning, but they don’t have the social power to get anyone else to see it.
—-
I am not sure that newcomers are necessarily devoid of social power. They can get a hammering from the seniors if they veer from influence of seniors. True. But if they are creative or analytically creative they can make sustainable contributions.
RE: Social power frequently doesn’t have anything to do with being right, or having correct reasoning. Once fields get “taken over” by those who use social power to enforce what is “important” or “valid”, then those fields are doomed. This happens when resources in a field are allocated according to social power by “leaders” and not by other means. Power corrupts and absolute power corrupts absolutely.
———–
That’s why I developed a passionate interest in the sociology of expertise. Statistics and psychology appear rife with relational dynamics that are not helpful. Dynamics that lead to poorer knowledge base. That is why it helps not to be so ensconced in cliques.
There are two articles that really crystallized my thinking around a field getting hijacked by people pursuing social power and not a deeper understanding of reality.
The field is intelligence testing, which still has the idea that intelligence is something that depends on something called ‘g’, which can be measured by doing ‘intelligence tests’. The field maintains this myth, which is actually wrong. A good article discussing why and how it is wrong is here: titled “ ‘g’ a statistical myth”
http://bactra.org/weblog/523.html
Look especially at note 2. Even if there was such a thing as ‘g’ using a linear approach can’t find it because there are too many degrees of freedom. The system is indeterminate, even if everything to do with intelligence is linear. We know intelligence is highly non-linear. Non-linear systems of a few coupled variables are completely intractable. How many non-linear variables does intelligence depend on? Hundreds? Thousands? Hundreds of thousands? Trying to use any kind of linear approach in a system of so many coupled parameters is silly, and completely flawed. How could the field have missed this obvious error? Other than through motivated reasoning?
How has the field of intelligence testing not corrected itself over this fundamental flaw? Pretty clearly it is because collectively, the “leaders of the field’ won’t allow the field to be corrected. Which leads to the second paper.
For example: here is a paper by Peter Schonemann who was a recognized expert in the field, with tenure, who couldn’t get something (which was correct) published because “the peers” and journal editors didn’t want it to be published. Not because they could demonstrate that it was wrong (it wasn’t wrong, it was correct) but because it contradicted their (incorrect) world view about intelligence.
http://ferris-pages.org/ISAR/schonemann-obit/Better-Never.pdf
The flawed thinking of the intelligence testing field has propagated into the genetics of intelligence field.
All of the Genome Wide Association Studies, use a model that is linear and purely additive to regress the association between DNA SNPs and measures of “intelligence”. All DNA interactions are non-linear. Protein-protein interactions are non-linear. GWAS intelligence studies specifically exclude gene-gene and gene-environment interactions. The variance due to those interactions is imputed to be exactly zero.
How can gene-gene and gene-environment effects be zero? They can’t be. Transcription factors which regulate gene expression are themselves products of genes.
It is pretty obvious that human emergent traits, like intelligence can’t be largely genetic. There isn’t enough information in the genome (6×10^9 bases) to specify 10^11 neurons with 10^15 connections.
It is willful blindness to ignore effects that are known to be large, and then to hide that you have ignored it in your analysis.
I’m sick of all this: “Oh – I’ve just discovered groupthink in science! What a scandal! What a surprise!”
I said this in my book right on 6 years ago, and I don’t remember getting all this fuss that Sabine’s getting. What’s more, I don’t make elementary philosophy of science errors like she does: she accuses people of coming up with untestable theories. What if a theory becomes testable next year?! Today it’s an nonsense theory, is it? What does SKDH think it is that makes people DEVISE tests? They don’t happen automatically.
I didn’t spend the effort working out these guideleines just so someone can come along and pretend they’re doing it for the first time, even after they’ve been told:
11) An untestable theory is one which is intrinsically logically untestable, not one for which no
technique for testing it is yet known to some person, or indeed anyone. Deducing the scope of
implications, effects, or influences of a hypothesis (via which it might be tested) can be slow and
unending. ‘Untestable’ is a rare category, not to be routinely flung at everyone else’s new theory.
12) Tests a new theory can uniquely pass are best offered, and may be needed for superiority, but
not for pseudo-criteria like theory status, false ‘testability’, or truth. Insisting on a mechanism for a
theory is a classic error. A theory often inspires the discovery of its mechanisms and special tests.
(From: https://sciencepolice2010.files.wordpress.com/2011/06/sciencepolice-14-latest.pdf )
And it’s not just in physics, she says. Well, at least in physics, people going into it have some kind of relevant skill, unlike in palaeontology. Her four quotes at the start of this posting all apply to palaeontology. If people had paid attention what I’d said about that then physics would have benefitted. I also made some pertinent comments about the physics situation, in my chapter 2.
“But falsifiability is a weak criterion for a scientific hypothesis.” Oh? There is only the predicitve/explantory power of a hypothesis (and judging falsifiability is so easy to get wrong, it seems). Refutation is in fact an aspect of failure to explain adequately. If theories aren’t judged on their predicitve/explanatory power then the alternative is some kind of style in which they’re created (NEVER an adequate criterion) or stuff like beauty.
Funny thing is I really admire German scientists, German women (shouldn’t say that, I suppose!), any scientist who goes against the grain, and anyone who can do all she’s done and raise multiple children. But it pisses me off when people think they’re the only right-thinking rebel in the world. A bit of unity would work wonders. Yet another unherdable cat.
Here’s what I said about this kind of stuff last year. You don’t have to read all of it ;-) :
https://sciencepolice2010.wordpress.com/2017/11/15/sabine-hasnt-killed-popper/
I have beauty as a perceptive phenomenon that combines two things: closeness to a particular ideal; and in particular, to an ideal that is particularly desirable. So a scientific theory that explained a lot and was very simple would satisfy the second half; also, as you understood it for the first time, if all sorts of perceptions like consilience, isomorphisms with other good theories etc occured to you, that would be the first half.
Occam’s Razor? It cut palaeontology’s throat.
Fact: I just quoted this.
Strangetruther:
1. I think Popper is great, as long as he’s interpreted in a Lakatosian way; see my 2012 paper with Shalizi.
2. Do you think Hossenfelder is really claiming that she’s “just discovered groupthink in science” or that it’s a “surprise”? It seems to me that she realizes that many people have been talking about groupthink in science for a long time, and she has some things to add to that discussion. Which seems fair enough. Her discussion of beauty is new to me—and, even if it weren’t new, there’s virtue in explaining things clearly.
Anoneuoid:
” ‘There are no absolute facts, proofs or truths; outside mathematics at least,
they are convenient untruths that can simplify our thinking.’
Fact: I just quoted this”
Define ‘quoted’!
Is it some kind of “being and time” bs where you have noted the limitations of language but misattribute that problem to some aspect of reality?
Hi AG –
“1. I think Popper is great, as long as he’s interpreted in a Lakatosian way; see my 2012 paper with Shalizi.”
Crumbs! Thanks. I’d better dig in I suppose. I’ll get back later if I have anything to say!
“2. Do you think Hossenfelder is really claiming that she’s “just discovered groupthink in science” or that it’s a “surprise”? It seems to me that she realizes that many people have been talking about groupthink in science for a long time, and she has some things to add to that discussion. Which seems fair enough. Her discussion of beauty is new to me—and, even if it weren’t new, there’s virtue in explaining things clearly.”
I haven’t read her book. I’ve read enough of her blog and tweets to convince me of a couple of things, though I may be wrong on both counts. But I get the impression that she shares the usual hopelessly poor understanding of many about testing and testability. Getting that wrong can make you get the science wrong. If I could give half a…sixpence…about big physics, I’d be very worried about it on that account. As it happens I’m only worried about the money it’s doomed to waste. The other thing is the impression I have got that she doesn’t see the groupthink thing as a science-wide – indeed world-wide – issue. You may have read other stuff she’s said or interpreted her differently; I felt she lost an opportunity to attack the issue on a broader front, and I did rather get the impression she was saying she’d just discovered it. I’d hoped Gillian Tett’s two books, one of which was:
The Silo Effect: Why Every Organisation Needs to Disrupt Itself to Survive
…and the other one was one of her other two books, I can’t remember which now, but maybe it was both… anyway, they prepared the field for a grand understanding and assault on groupthink, in an area no-one can ignore, and nothing happened. No-one wants to attack their own field, and few understand fields they’re not in well enough to do so. And if like Hossenfelder they try, they do run risks. Tett at least is an investigative reporter, so is in a rare position to be able to criticse any field.
But mainly I’m annoyed that my offering on this got the reception I expected instead of what it deserved. And the most annoying thing is that this is because Hossenfelder comes from the Max Planck… whereas I’ve chosen independence. The moral is: critics and other commentators judge work by circumstantial stuff surrounding the author, instead of the work. Which is one reason why so much effort is made towards achieving position and prestige, and once there it’s weird as well as dangerous to criticise your own career and institutions.
As for the beauty thing, I haven’t tried to sort out exactly what she’s saying about what other people are saying about the usefulness of beauty in any science, but I’d guess other physicists’ beauty criteria are worth more than her understanding of testability and other vital issues in the scientific process because everything she’s said about that seems rubbish to me.
Looks like an interesting read however.
It’s a mistake to frame it as Copernicus vs epicycles because he actually added MORE epicycles in his model! Copernicus was opposed to the equant, and thought that he could get rid of it to produce a model consisting entirely of combinations of perfect circles (including epicycles).
This time I agree with Gelman entirely. You can’t expect beauty from evolution and you don’t get it. What you have in humans is a jumble of inconsistent and tortuously complicated systems, generated by random mutations over the ages, which just works, most of the time.
I often describe evolution as “survival of the fit enough to have survived so far”. And every once in a while it comes up with something that is beautiful (at least for a while).