In social science, we’ll study some topic, then move on to the next thing. For example, Yotam and I did this project on social penumbras and political attitudes, we designed a study, collected data, analyzed the data, wrote it up, eventually it was published—the whole thing took years! and we were very happy with the results—and then we moved on. The idea is that other people will pick up the string. There were lots of little concerns, issues of measurement, causal identification, generalization, etc., and we discussed these in our paper, again hoping that these will be useful leads to further researchers.
And that’s how it often goes. Sometimes we return to old problems (for example, we wrote a paper on incumbency advantage in 1990 and followed up 18 years later), and we’re still working on R-hat, over 30 years after I first came up with the idea), but, even there, we’re typically not working with continuous focus.
The opposite approach in science is to drill down obsessively on a single phenomenon, to really pin it down. I think this is what historians do when they immerse themselves in some archive for a decade and then emerge to write the definitive book on the topic.
Here’s an example, not from history but from cognitive psychology, by Andrew Meyer and Shane Frederick:
This paper presents 59 new studies (N=72,310) which focus primarily on the “bat and ball problem.” It documents our attempts to understand the determinants of the erroneous intuition, our exploration of ways to stimulate reflection, and our discovery that the erroneous intuition often survives whatever further reflection can be induced. Our investigation helps inform conceptions of dual process models, as “system 1” processes often appear to override or corrupt “system 2” processes. Many choose to uphold their intuition, even when directly confronted with simple arithmetic that contradicts it – especially if the intuition is approximately correct.
The paper contains the charming Ascii graphic reproduced above (for example, page 8 here). I love Ascii graphics! Regarding the paper, Frederick writes:
One thing I’m proud of is summarizing 59 studies in just 9 pages. Another thing I like, and you’ll probably like, is that when sample sizes get large enough (and we have some pretty large ones), psychology starts to look like physics.
What really impresses me about the paper is not the sample size but the obsessiveness of the project. And I mean that in a good way.
I guess there were 59 studies reviewed and 70k people answering the bat-ball question.* But did any participant get asked *why* they chose 10 cents?
I don’t see it in this review. Its like the only data that could possibly exist is survey responses.
Id guess a good chunk are trying to answer quickly so stop at “the bat costs $1.00”, or insert a comma: “The bat costs $1.00, more than the ball”.
Theres also so many 1s and 0s to store in memory. Like $1.10 – $1 = $0.10. Then $0.10 x 100 = 10 cents. And 1.00/10 = 0.10. That dang ten keeps popping up.
Btw, I did it “10 cents? Nope that doesn’t work. Do x + x + 1 = 1.10, so x = 5”. Ie, a guess followed by a sanity check, then check failed so work it out the long way.
*For those not clicking the link:
Anon, you might want to read the paper, it’s not very long. It contradicts some of your suppositions.
The paper is trying to use aggregate responses to study a phenomenon happening at the individual level.
That’s just as goofy as monitoring the firing of individual neurons to see how a person solves the problem.
Instead I’d like to see stuff like individual learning curves after repeat exposure to the same or similar questions, and narrative accounts of the thought processes. From that we *can* develop models predicting results at the aggregate level. But, the reverse (trying to guess whats going on from such data) doesn’t work except in some very simple cases.
So I don’t think my suppositions can be contradicted even in principle by that paper, but maybe they went into individual results and I missed it.
To be clear, I definitely think there is something interesting to be learned from the answers to this question. I am 100% in favor of figuring this out.
IUltimately, I’d guess most is people don’t really care what the answer is because it is someone else’s problem, or really not a problem anyone actually cares about. So they are only willing to devote minimal resources to solving it.
This refers to trying to answer to the “bat-ball question.” The research question of why everyone wants to answer 10 cents is definitely interesting.
Most psychological research uses aggregate data to study individual level phenomena because we need many observations to distinguish signal from noise and people recall their previous responses.
I don’t think “narrative accounts of the thought process” are likely to be accurate, but Barnabas Szaszi has some of those data: Szaszi, B., Szollosi, A., Palfi, B., & Aczel, B. (2017). The cognitive reflection test revisited: Exploring the ways individuals solve the test. Thinking & Reasoning, 1–28.
I don’t think “individual learning curves after repeat exposure” are particularly helpful. But Wim De Neys has short term versions: Raoelison, M., & De Neys, W. (2019). Do we de-bias ourselves?: The impact of repeated presentation on the bat-and-ball problem. Judgment and Decision making, 14(2), 170. And Shane and I have long term versions: Meyer, A., Zhou, E., & Frederick, S. (2018). The non-effects of repeated exposure to the cognitive reflection test. Judgment and Decision making, 13(3), 246.
This is a really consistent phenomenon though, so deserves more robust study. So far everyone I asked answered 10 cents initially except a guy who grew up working in casinos.
The Szaszi et al paper is close but still hides the raw data by aggregating into a few categories.
I do think some kind of psychological “law” may have been discovered here. Whatever it is can’t be figured out from an aggregation of some people doing it one way, others another way, and so on because their responses all get grouped together. Its basically using a lossy compression algorithm on the data.
Forgive my ignorance, but what is the meaning of the ASCII graphic? I have seen ASCII art before (although I only learned the term ASCII art a few minutes ago). Is the purpose of the graphic just to look pretty, or is it to convey a message beyond aesthetics?
Raphael:
It’s in the article that’s being discussed. Just follow the link!
Somehow, in my mind, the bat and ball problem reminds me of the so-called invisible gorilla problem.
https://psycnet.apa.org/record/2010-14410-000
In each case, there is a tendency for some (many) to focus on one aspect of a situation and ignore that which is obvious to others. Also, each involves some aspect of arithmetic getting in the way and each has successful problem solvers who are unable to believe that people can make that kind of mistake/error.
I agree. We should have made the connection.
I don’t think they’re quite the same, if you mean the original gorilla experiment as opposed to what might be many other examples in the book linked. For the gorilla experiment, it is more about how focusing on a particular task leads our attention to neglect even quite striking things going on that are not task relevant. In the bat and ball case people are trying to solve a single problem, and for many people there is an immediate and intuitive seeming but incorrect solution. The question is then to what extent people question/check that intuition (and presumably some people, probably a minority, don’t even have the false intuition and automatically convert it to a proper calculation)
This is so off-topic that I almost feel strange saying it but: it seems weird to me to suggest that a bat and ball could cost $1.10 total. I dunno, just maybe there’s some kiddie plastic bat-and-ball combo for which that would be possible, but for a real bat and ball it seems low by at least 10x. Even a whiffle ball costs something like $1, and that’s if you buy in bulk.
I understand why you’d want to use a real(ish)-world example: I presume it’s because it makes the problem more accessible to people who aren’t too familiar with a more math-class type of construction (such as “A + B = 1.10” etc.), and also makes it more clear that there are real-world situations in which you’d like to solve a problem like this. But if a real-world problem is to be chosen then it seems strange to make the prices so incompatible with reality. Would the problem be changed in a meaningful way if you said “a bat and a ball together cost $11. The bat costs $10 more than the ball. How much does the ball cost?”
I can’t see how the unrealistic prices are harmful, really. Maybe just slightly distracting, like if you started a problem with “A man is 11 feet tall…”
Phil:
I assume it’s just a really old problem, originally written when the actual prices were something close to that. But, yeah, I agree that it’s time to update!
For what it’s worth, I just asked ChatGPT “How much do a bat and a ball cost?” and it confirmed that the pricing is indeed accurate: “A bat and a ball together cost $1.10. The bat costs $1 more than the ball. So, the ball costs $0.05, and the bat costs $1.05.”
Dick’s has a whiffle bat and ball for $8.99, so I’m betting they are available at the dollar store for at most $1.25 ea. I can’t say for certain but a picture in my brain tells me I’ve seen them there. I mean you can get a swim noodle at the dollar store for $1.25, which probably weighs more and has greater volume (e.g., has more material, is larger and thus more expensive to ship and buy) than a whiffle bat and ball…
Sooo…I guess in this round between ChatGPT and Phil, ChatGPT wins the round, having come closest to the actual price without going over…
The one at Dicks is an “official” Whiffle bat and ball complete with 9″ diameter regulation size ball! That’s it. No doubt, a chinese knock is available at the dollar store for a few bucks or even less.
Winner: ChatGPT
Footnote 5 says solution rates are the same whether they use $1.10 or $110.
David:
$110 . . . that’s a lot to pay for a bat and ball!
https://www.slugger.com/en-us/product/2024-meta-5-2-3-4-wbl2846#axis=101925
That’s only a bat. And I thought golf equipment was expensive.