The importance of “bumblers” and “pointers” in science, and the division of labor

A few years ago, I received some angry emails from a psychology professor I’d never met, who was annoyed at me for criticizing published work in his field. In one of these, he wrote:

Of Dan Wegner’s many wonderful papers, I think I still like best his little theory of science paper in which he says there are two kinds of people in science: bumblers and pointers. Bumblers are the people who get up every morning and make mistakes, trying to find truth but mainly tripping over their own feet, occasionally getting it right but typically getting it wrong. Pointers are the people who stand on the sidelines, point at them, and say “You bumbled, you bumbled.” These are our only choices in life. If one is going to choose to do the easier of these two jobs, then one should at least know what one is talking about.

This came up a few years ago, and I remarked that this was a pretty ridiculous thing, especially coming from a psychology professor! [I refer here to my correspondent, not to Wegner, who I can only assume had a more nuanced take on all this.] I hope my correspondent doesn’t teach this sort of simplistic theory to his students. Also it was funny to see him directing this to me, given that I have personally spent so much time doing both of these things. The idea that I might do research and also criticize research . . . that was somehow beyond him.

Recently this exchange came to mind and I realized a couple other things.

The first is that my correspondent referred to standing on the sidelines and pointing as “easier” than getting up very morning, making mistakes, trying to find truth but typically getting it wrong.

But why does he say this? Even accepting his (ridiculous) idea that any person can only do one of these two things, why is he so sure that “pointing” is easier than “bumbling”?

Look at his statement more carefully. What’s so hard about “bumbling”? “Getting up in the morning,” that’s not so hard—we all do that! “Trying to find truth,” that’s not hard either. Anyone can try to find truth. “Mainly tripping over their own feet, occasionally getting it right but typically getting it wrong”: hey, that’s sounds pretty damn easy indeed! Admittedly, his definition of the activities of “pointers”—to “stand on the sidelines, point at them, and say ‘You bumbled, you bumbled.'”—that sounds pretty effortless too! So I’d guess the two tasks, as my correspondent described them, are equally easy.

To me, science is hard. To me, the “bumbling” part is not just about “trying to find truth but mainly tripping over their own feet.” It’s all about putting my ideas to the test, which includes doing my best to shoot down my inspirations and also learning from criticism rather than acting defensively. And “pointing” is really hard, because when I criticize others, I want to be very careful not to be wrong myself. Sometimes I make mistakes when criticizing—“pointing” itself has some aspects of “bumbling”—but I try hard to avoid them. So the pointing that I do, like the bumbling that I do, is effortful, as science should be. It’s satisfying and often enjoyable, but it’s work. And neither of these things is inherently easier or harder than the other. I do think it reveals a deep misunderstanding on the part of my correspondent to think otherwise.

The other thing that came to mind just now is division of labor. As I said, I do my own (collaborative) research and I also criticize the research of others. That works well for me—I think that my own research allows me to be a better critic, my criticism opens up new research ideas for me, and both these things improve my textbooks and other expository writing.

But my approach is hardly the only way to go. There are some excellent researchers who don’t waste any time criticizing bad work; they just focus on making progress, and they expend their critical effort on improving their own research. That’s just fine. From the other direction, there are some excellent critics who don’t do much original research but devote lots of time to careful criticism of work by others. That also is important. Good criticism (what my correspondent would call “pointing”) can take a lot of work, and also attention and persistence. You might have to make data requests over and over, continue bugging journals and other institutions, do all sorts of things that I won’t have the patience to do.

And that gets to the point about division of labor. Some people tell me I shouldn’t be wasting my time trying to understand and write about what’s wrong with bad research. Maybe they’re right; I don’t know. But, if you have that attitude, you can’t then turn around and criticize various research critics for being “pointers,” “terrorists,” “Stasi,” “second-stringers,” etc. You can’t have it both ways! It’s just fine to have prominent critics who do not themselves have active research agendas. Criticism is important, we learn from it, and someone who’s really good at criticism might not be really good at original research, just as the reverse might be the case. We need high-quality “bumbling” as well as high-quality “pointing”; that’s how science works.

P.S. Also from that above-linked post, I notice that Tyler Cowen in his blog had linked to that terrible, terrible ovulation-and-voting paper, calling it the “politically incorrect paper of the month.” This was amusing for two reasons:

1. Nobody uses the term “politically incorrect” anymore: like an old stick of gum all of whose flavor has been chewed out, this term has no rhetorical value anymore. Which suggests to me that it never had much real value, and that it was only serving as a political weapon.

2. That ovulation-and-voting paper was, I’d say, scientifically valueless, although it did have the virtue of serving as an example in our multiverse paper. A good reminder that a published article can be “politically incorrect” or serve some other political purpose and still be no good. This wasn’t so clear at the time, but now that we’ve been thinking about forking paths and small effect sizes for awhile, these problems are becoming easier to spot. Just to be clear: I’m not saying the paper was bad because it was “politically incorrect”; I’m saying it was bad, and the political angle gave it some attention and approval it did not deserve. Similar things have happened with bad papers that scratch an itch on the left side of the political spectrum; see for example this article purporting to demonstrate “The Bright Side of Unionization.”

22 thoughts on “The importance of “bumblers” and “pointers” in science, and the division of labor

  1. I agree with all of this, except one point. I don’t think there are people who are good at critique but not original research, or vice versa. If someone is good at critique, they can certainly do good original research (and vice versa) – but they may choose not to. Specialization is a choice they make, but not a property of their capabilities. I’m not sure how someone can be good at only one of these two things since they are two sides of the same coin. Plenty of people specialize for all sorts of reasons (personal preferences, incentives) but I don’t think it is because their abilities only align with one of these two tasks.

    The reason I think this point is important is that my teaching philosophy has always been that to be capable of good critique of data analysis, one must be able to do such analysis themselves. They may not find it easy, nor may it be something they will end up doing themselves, but it is difficult to critique effectively what you can’t do yourself. I violate this principle regularly – like many, I am an armchair quarterback, quite willing to criticize choices coaches and managers make. But that is entertainment – I don’t really know better than them, I am only pretending.

    • Dale:

      I agree with all of your comment, except for one point. Yes, anyone who is good at doing original research should have the ability to be good at critique, and vice versa. Indeed, being a good critic should help your research, and doing research should help your criticism.

      That said, not everyone does both. If someone just does critique and not any serious original research, then such a person might actually not be any good at original research. Having the potential ability to do something is not the same as actually being good at it. Or if someone just does original research and not any serious critique, then such a person might actually not be any good at critique. And, as with my correspondent, such a person might also not understand what really goes into critique.

  2. Andrew wrote:

    “It’s all about putting my ideas to the test, which includes doing my best to shoot down my inspirations […]”

    To my surprise, I discovered that this is a controversial approach that is not the norm in some fields. Certainly if you are working in a field like chemistry, you need to murderboard – consider every possible way you could be wrong and develop some evidence that refutes them all – your ideas before you publish, or you will be eaten alive by critics.

    In other fields, your paper might be celebrated because of an approach you pioneered that makes your results shine like a thousand suns, with little interest in the actual results. I think partly this is collateral damage from NHST abuse. There is a mindset that if you are doing a psych study of a heterogeneous population, it is simply impossible to control all the variables, so murderboarding is beyond your reach. Fortunately you don’t need to worry about that because NHST will pull you through. From this perspective, critics are not just criticizing a single paper built on NHST, they are tugging at a thread that just might unravel the entire sweater.

  3. The focus of interest in today’s blog concerns the (possible) dichotomy between scientific/mathematical people who engage in research vs. those who do criticism in science/mathematics. Whatever validity this split has or has not in science/mathematics, what about other fields of endeavor? Playing a sport as opposed to coaching the sport, writing as opposed to editing, leading a moral life as opposed to [merely?] preaching about it, etc. After all, a road sign may never get there, but it presumably does point in the right direction.

    • Paul:

      I remember that Bill James wrote that the best baseball managers were often marginal players. He attributed this to the idea that if you’re a great player, you can just play, but if you’re marginal, you have to look around you all the time to figure out how to contribute to the team. That could be, or it could just be a simple issue of base rates, that there are lots more marginal players than great players to choose from.

    • Dale:

      Yes, the quote has come up a couple times before on this blog (see here from 2014 and here from 2020).

      I think I’ve kept coming back to this one, partly because it’s a funny example of the sort of discrete thinking that I don’t like but which seems to come natural to so many people, and partly because it relates directly to what I do in my job.

      • I mean the whole post (with comments) was a repeat. Of course, nothing wrong with repeating things, it just that I haven’t seen you repeat entire posts with comments before.

        • The post appeared with 5 comments already included – one of them mine, which I did not submit today but submitted when you first ran the post. So, the whole post, and I believe the first 5 comments, were a repeat. Again, nothing wrong with a repeat – but in the past you have labeled them as repeats. This time you reference it coming up a few years ago, but that makes it sound like this was a new post – which it is not.

        • Now I’m really confused. The date on my first comments was Nov 28 (and I do recall making that comment). Your response – which I recall seeing after I made that comment – is dated Jan 28. But I saw that comment back in November.

        • The date thing is weird. I can attest that I wrote my comment this morning, but after posting thought “uh oh, did I just make the same point that I made on a very similar post a few months ago?”

          The blog search function choked on “posted on November 28,” returning a random mix of posts.

        • OK, I think I know what happened: I wrote it a couple months ago to appear now, but at the time I accidentally pushed Send on it without first setting the date. When I realized, I fixed the date, but in the meantime Dale’s comment had already appeared. The comment was saved in the system, and then when the post showed up, the comment showed up with it.

  4. Wegner never said the roles were mutually exclusive- Here’s a quote:

    Wegner:
    Over time, I realized that this was not just a matter of individual differences, but that people could move in and out of bumbling and pointing “modes” and thus assume either condition in proper circumstances.

  5. May I say that you critique because you are in the tradition of Jewish intellectuals who have learned the essential methods of criticism which were taught through Jewish culture and family. In the Jewish methodology, there is a distinction between the one who enacts and the one who expresses doubts. It’s a conversation which mimics the one held between humans and themselves, between humans and their beliefs in themselves and others, and thus between humans and the Jewish conception of God, which is or at least used to be the deification of the highest idea that we can manage to come up with which expresses what we call God (though we avoid applying the actual label, choosing instead to say words like Adonai or HaShem as if the conceptions we have of God can’t see the connections). This means our intellectual achievements are seen by God, and thus we have an obligation to God in our achievements. That means we must look within ourselves for that which stands in the way of seeing the truth, whatever truth that happens to be, and that means we must find fault because otherwise we cannot hope to lift the level of achievement.

  6. Andrew,

    Your post makes me think of two extremes in ways to look at the world. For lack of better terms, I’ll call the people on the extremes Type P and Type S.

    “Type P” is – a bad term but its the best I can manage – “physical rational”. They try to have insights about the world they live in and improve their life through acting on those insights. They view “success” as executing this approach at the best level. “Type S” is “social rational”.. They view their job as a thing to do to get money and status. The outcome of their work is the status they achieve from it. The results of the actual work aren’t important to them – the work and it’s results are just a tool to get money and social status.

    Type P is the person who, before a team meeting, is trying to work out a plan to improve the product. Type S is the person who, before a team meeting, tries to come up with things to say that sound smart to impress people. Type S people fear criticism of their work because they don’t recognize the work itself as being independent of the social context. To them, criticizing their work is always a personal attack because it impacts their social standing, which is the only benefit they get from doing it.

    You are type P. Your correspondent is Type S. The correspondent is offended because you violated his social rule of not criticizing others.

    The P/S distinction is what thwarts surveys. Most surveys are designed with the expectation that the person will respond in “P” mode. However, many people have no idea what P mode is. All they know is S mode. Every response is S-mode – irrational and unpredictable by scientific standards.

    • I’m finding this very plausible. Science has a problem IMHO in that it has been heavily infiltrated with type S over the last 120 years or so. Before then science was something done by people who were interested in doing it, and it was considered weird and not prestigious even potentially socially dangerous (think Galileo or Copernicus). A lot of the best science of the 1800s was done by aristocrats like Lord Kelvin or whatever. Of course this is a generalization, but I think with the incredible prestige boost science received in WWII and the cold war and the silicon valley microcomputer revolution etc increasingly science and technology is seen as similar to lawyering or doctoring, witness the “coding camps” being set up for kindergartners. There’s a big difference between someone like John McCarthy (inventor of LISP) or Edgar Codd (inventor of relational database theory) and a tech bro at Uber or whatever.

      Of course the world needs people to crank out applied technology but it’s not clear the world needs people to crank out NHST fueled “discoveries” in medicine or psychology or cancer biology or “wormhole” experiments in physics etc

      • It was pointed out to me that Kelvin got his peerage as reward for his science… So bad example. But for example Lavoisier was a French Aristocrat, Maxwells family were Baronets with inherited property, DeBroglie was an aristocrat, and Faraday was motivated largely by his devout christian beliefs (and he turned down a knighthood) so I feel like it’s still reasonable to say that a lot of science was done for motivations other than money or social prestige. Darwin neglected his medical studies and probably relies on his father’s wealth to be able to do his naturalist stuff for example. His father supposedly was annoyed that he wasn’t studying medicine.

Leave a Reply

Your email address will not be published. Required fields are marked *