Many fields of research can be justified based on the argument that their object of study exists, and that denying its existence won’t make it go away. For example:
Economics: Denying the existence of economics (for example, by trying to set up a command economy) doesn’t resolve the fundamental problems of economics. Issues such as scarcity, opportunity costs, etc., will just arise in other form; they can’t be legislated away.
Political science: There is no such thing as a political vacuum. Conflicts about power, resource allocation, etc., still need to resolved, one way or another, even in the absence of a formal government.
Sabermetrics: People make judgments about baseball statistics. As Bill James put it, the alternative to “good statistics” is not “no statistics,” it’s “bad statistics.”
Causal inference: Everybody cares ultimately about causal questions. As Jennifer Hill says, even if you claim to be just studying association or descriptive statistics, really this is motivated by have underlying causal questions.
Bayesian inference: Every analysis uses prior information; the only question is whether you want to acknowledge it explicitly.
Defaults: We all have defaults, so let’s try to set them well. Yes, it’s true that no default is perfect, or close to perfect—any default has its zone of effectiveness, outside of which it fails—but defaults are inevitable, so the only way forward it so choose good defaults and then understand where they work and don’t work.
Workflow: Theoretical statistics is the theory of applied statistics. In real life, researchers learn from a dataset by fitting lots of models, including lots of mistakes. Let’s recognize this is what we do and design our procedures accordingly.
Much, perhaps most, of statistical practice is tacit. We make lots of decisions without thinking about them. Let’s study statistics so we can do it better: it exists, and it’s not going away.
Now that you’ve pointed it out I can’t help but see this principle everywhere.
I think these are all ‘practice-based’ disciplines: The underlying practice will exist, so might as well optimize it.
But I am reminded me of a debate between formalism and substantivism in economics [1]. The formalists would try to model a situation abstractly, often with an eye towards what the optimal practice would be. And the substantivists would try to describe the practice as it currently exists.
In my opinion, in all these practice-based disciplines, we need more substativists. (For example, a substantivist approach to statistics is looking at how people actually reason about statistics etc). A cool use of big data is not optimizing practices but describing them.
[1] https://en.wikipedia.org/wiki/Formalist%E2%80%93substantivist_debate
I should say, I’ve identified the phenomenon in niche areas of my own research. But I’ve never heard it articulated as a general principle. Nice!
This certainly applies to lots of things. (Physics exists; without studying physics we’d just do bad physics, like people did for millennia.) I’m trying to think of fields for which this *doesn’t* apply. I came up with:
(i) much of engineering, whose existence is tied to the invention of new sorts of things. (Would electrical engineering exist without electrical engineering?)
(ii) Astrology and similar things. This one is more interesting: Astrology “exists” in that people use it, and denying this existence hasn’t made it go away.
You can choose not to believe in astrology and therefore not do any astrology. If you don’t believe in economics, you still make decisions about economics just by existing as a human in society.
Andrew wrote:
“Many fields of research can be justified based on the argument that their object of study exists, and that denying its existence won’t make it go away.”
Raghu wrote:
“I’m trying to think of fields for which this *doesn’t* apply.”
Right now “gender studies” seem to be at this particular crossroads.
Ah. I’d interpreted Andrew’s argument differently, which explains my confusion about political science in my post below. I’d thought he was defining the “existence justification” as “if we ignore this thing, it is at our peril, because it exists and will undermine our efforts.” But I see now Andrew was defining it as “if we ignore this thing, which exists in the universe, we will be doomed to ignorance about the universe.”
You might say that some fields are justifiable *only* or *primarily* by the existence of their subject matter (e.g., astronomy), others are justifiable *only* or *primarily* by their practical usefulness (e.g., electrical engineering), and others still are justifiable by both (e.g., economics). Thus, we have a two-parameter model of scientific field justification.
Decision theory: you, or the people paying you, always have a decision problem in mind. No one studies air pollution or drugs or ‘himmicanes’ because they deeply appreciate the abstract beauty of a scatterplot. It is not easy to answer that decision problem, but you will be choosing an answer to it (inaction is also an answer), whether you like it or not.
Engineering exists as a discipline independent of what you are engineering – electricity or chemistry or software. If you don’t study engineering to understand sound practices for the manufacture of complex things (design, modularity, testing, …) you build bad products.
I agree that astrology and its ilk are more interesting examples.
Of course, this principle can be used to justify a methodology even if the methodology is inapposite. You have criticized, for example, the economic theory of rational addiction. See, for example, https://statmodeling.stat.columbia.edu/2007/10/05/more_on_signifi/ . Addicts have things going on in their brains, right, whether economists study them with their methods or not.
Jonathan:
Sure, there’s lots of bad political science, bad economics, bad sabermetrics, etc. Just because there is a use or need for a field of study, that doesn’t mean that everything done in that field will be sensible or useful.
May I play devil’s advocate? Didn’t even Keynes write in 1930 that “the economic problem may be solved, or be at least within sight of solution, within a hundred years. This means that the economic problem is not-if we look into the future-the permanent problem of the human race”?
Might one go even farther than Keynes, seeing economics as justifying imposed scarcity of money as a necessary proxy for assumed real scarcity? Has economics carved out a niche for itself, providing academic justification for those in power who seek to impose artificial scarcity, being handsomely rewarded by the powerful for playing their prescribed “it’s just Econ 101” role? (I.e., currently, isn’t inflation really all our fault for wanting too much, so isn’t the Fed justified in tightening money?)
While you’re setting arbitrary defaults, can you try to listen for those of us, so easy to ban, who are negatively impacted by policies based on those defaults? If you just can’t imagine wanting to sleep outside and make it hard to hear from those of us who prefer it, is it a big problem for me (restrictions on public camping) but a comfortable outcome for you (“they’re much better off indoors where we can lock them down in a pandemic”…)?
Rsm:
I have no idea why you think I want to stop you from going camping! I’m still annoyed when they locked up the basketball courts a couple years ago. The outdoors is great.
AFAIK, statistics is easy – almost anyone can use R or a spreadsheet to calculate statistical parameters. So what? What shines through time and time again in all the poor research that’s critiqued here is fundamental ***scientific*** incompetence. If the science is bad, then no statistics can fix it. Worse yet, if the science is bad, statistics often can’t even **detect it**.
So, I say no: study **science**. If you understand science, statistics will follow.
Economics exists in reality, it impacts all our lives, and those who study it or who rely on those studies will do better than those who do not. Is that true of political science? I don’t see it. Yes, obviously political events and interactions occur with or without their study, and obviously they impact our lives, and obviously understanding things that have huge impacts on our lives is intellectually enriching. But I’m not aware of an innovation or understanding that’s arisen from political science that makes politics less rancorous, say, or more just. Demagogues, strongmen, and charismatics often prosper without any scientific understanding of politics, and their rivals rarely overcome them by means of superior applications of political science results. Also, it appears to me that most political institutions change due to popular movements or monied efforts, not by scientific intervention.
Arguably, electioneering is applied political science, but my sense is that’s just applying principles of stats, social psych, sociology, econ, etc. to political strategy, as opposed to, say, marketing strategy. And I don’t think we can use the existence and influence of marketing to justify establishing the field of ad sci.
I’m not trolling–I’m not saying political science is trivial, just that it may have to be justified on other grounds, like comprehension. Astronomy is inevitable, and it impacts all our lives (we orbit a star in a galaxy), but there’s little evidence we can do much about it. After it established that constellations aren’t magic, most of astronomy is completely ignorable. Nonetheless, astronomy is a hugely important field.
“Even if you claim to be just studying association or descriptive statistics, really this is motivated by have underlying causal questions.”
I’m going to have to respectfully disagree with the above. The clearest counterexample is theoretical physics. Causation makes no appearance in fundamental laws like Maxwell’s equations, the electroweak Lagrangian, etc. If your goal is to discover and apply such laws, you don’t need to worry about causation at all. A solution to Maxwell’s equations is a solution to Maxwell’s equations, regardless of whether changes in the electric field cause changes in the magnetic field or vice versa.
I’ll go much further and say there are quite a few interesting questions that don’t involve causation. Want to predict elections or baseball games? Correlational factors are all you need. They might also be causal factors, or they might not; either way, they’ll have just as much predictive power. Want to make conditional predictions about what will happen provided that such-and-such other thing happens? Conditional correlations are all you need, and again it doesn’t matter whether they’re causal.
The phrase “causal explanation” gets thrown around pretty often, but a lot of good explanations are purely correlational. What explains the giraffe’s long neck? Ancestral giraffes with longer necks were able to eat more high-up leaves, and so more of them survived to leave descendants. The explanation is just as good if it merely relies on a correlation between longer necks and eating more leaves, rather than a causal relationship.
Causal questions are well worth considering, of course. But I submit they’re far from the be-all and end-all of everything.
> I’ll go much further and say there are quite a few interesting questions that don’t involve causation. Want to predict elections or baseball games? Correlational factors are all you need.
Well, to play devil’s advocate here, my impression is causal stuff sort of inevitably comes up as you formulate questions and talk about comparisons better.
If there’s a tendency to seem too picky in causal thinking (RCT or bust), I think maybe there’s a tendency to want to be too carefree in prediction. So sure we could predict baseball games with a mixed bag of covariates that come from anywhere — but what does that buy us really? I’m not sure it’s a whole lot. It still takes a lot of work to get those covariates and set them up in a way that is useful for regression so that’s not a free opportunity.
My impression of causal stuff is you’re trying to be careful about the specific questions you ask, and this same sort of thing happens in prediction. Like, baseball outcome X is this — okay why? Is that good? What to compare against? What’s the difference in the groups we’re comparing? And you end up slipping into it. Even if you found some weird correlational thing — if that is important it’s probably worth pinning down, or at least investigating some.
> Causation makes no appearance in fundamental laws like Maxwell’s equations
Well, you certainly can ask causal questions here. Like, if I change the electric field — what change in the magnetic field do I get? There’s a good model here where in a controlled-enough setting we could calculate out what those changes will be. That’s fine — you can still have the causal question. And the existence of another causal question (what change in electric field happens if I change the magnetic one) is totally fine too.
Oh yeah so it’s not just me rambling, I think the age thing here is good: http://observationalepidemiology.blogspot.com/2021/07/counterfactuals-and-age.html
Like age causes things, right? Okay then what’s the comparison? Not aging? Huh. Weird. Seems worth thinking about, even if age is very predictive of things.
Ben:
I’ve thought about the age thing too! Maybe I wrote a blog post about it once, or I was going to write a post, or something like that? I’m not sure now. I’ll have to think a bit more about it. As I recall, I had some nontrivial thoughts about how to use causal inference models to account for the passage of time.
Isn’t all causal modeling actually about X now causes Y later? But mostly people ignore the time thing because they’re usually modeling an asymptotic steady state, X now caused eventually at some future unspecified time Y to happen.
Ultimately if you want some kind of improvement in causality and time you should think about dynamics.
Nope. Every cause is also an effect. What exactly would the cause of age be? Time? Can’t be, because time and age are perfectly confounded. Age = time from birth to present (or death, whichever comes first). Time is the duration, and age the duration between temporal parameters of interest, within which causes and effects occur.
Unless you want to get relativistic and talk about the impact of time passing at a certain rate in a particular gravitational frame of reference…but then, the counterfactual there is obvious (different frames of reference).
“Causal inference: Everybody cares ultimately about causal questions. As Jennifer Hill says, even if you claim to be just studying association or descriptive statistics, really this is motivated by have underlying causal questions.”
If you’ll excuse a cheap shot, isn’t this bad news for AI (as she currently exists)???
David:
There’s a lot of research in AI on causal inference so I don’t see why this is bad news for that field.
This framework reminds me that I’ve long described my role in science as one of harm reduction. Like a needle exchange. I hope that I do more good than harm, anyway, at least as an average treatment effect. Nodding to your lecture yesterday, there is almost certainly heterogeneity.