A few months ago we had an interesting discussion about evaluation of pollsters, following up on some thoughts of Elliott Morris and Nate Silver, two analysts I respect and with whom I’ve collaborated (on separate occasions). In recent years I’ve become annoyed with Nate from time to time, but, hey, nobody’s perfect and I still think he’s generally a reasonable person.
I had a new feeling of frustration, though, when in one of his recent posts involving the pollsters, Nate wrote, “So take that as a signal that I don’t intend this a back-and-forth.” And then, more recently, in the context of a completely different dispute with someone else, Nate wrote, “And while I don’t really want a back-and-forth . . .”
I get it that everyone’s busy and you don’t have time to respond to every argument that comes across your desk, but . . . back-and-forths are good, no?
Some googling turned up this quote from 2012, which I agree with:
Silver tells TechCrunch that intelligent prediction is messy, biased, and iterative — all the characteristics that don’t lend themselves to grand pronouncements in 30-second soundbites. Blogs, instead, lend themselves to an honest back-and-forth about the sausage of statistical conclusions, which can, hopefully, create a more respected class of experts and a more informed public. . . .
The “iterative” thing is good too. In complicated problems, our methods are always flawed. So, wherever we happen to be right now, we should welcome criticism and opportunities for improvement.
This is a point that I and others have made over the years, for example:
Blogging also has the benefit that the discussion can go back and forth. In contrast, the journal reviewing process is very slow, and once an article is published, it typically just sits there. . . .
This was before twitter, which has the different problem that most of the volume of posts is people cheering or booing. See enough of that and you too will want to cut short all the back-and-forth.
What happened to Nate between 2012, when he explicitly talked about the benefits of “an honest back-and-forth about the sausage of statistical conclusions,” and 2020, when he avoided discussions about problems with our forecasting models, and 2023, when he “doesn’t really want a back-and-forth”?
I don’t have any specific information regarding Nate, but in any case I’m more interested in the general phenomenon of when it is that public figures want to engage in back-and-forths and when they don’t.
My theory is that if you’re a pundit and you become famous, you attract lots and lots of stupid criticism. This happened to Paul Krugman, it happened to David Brooks, and it happened to Nate Silver too. You get lots of uninformed people attacking you because they don’t misunderstand what you’re doing or you were pooh-poohing their favorite JFK-assassination theory or UFO’s-as-space-aliens theory or you’re not taking their exact political position on the issue of the day or whatever. And so you develop . . . not a thick skin, exactly, but an acceptance that you have neither the time or interest in responding to all the uninformed and possibly insincere criticism that you’re receiving.
At some point you realize that you’re piloting a submarine through a poop-filled sea and you can’t spend the rest of your life trying to keep the hull clean.
At the same time, you’re getting some thoughtful criticism! Some of it is framed in a very deferential way to you, some of it is direct and polite, and some of it is downright rude but still thoughtful. But it doesn’t matter: you’ve already turned off your reply instinct. So even when you feel forced to reply (for example, person A criticizes something you said and then B, C, D, E, F, etc. join in the twitter thread and ask what is your response), you do so reluctantly, with annoyance, and you reiterate that you “don’t really want a back-and-forth.”
In summary, the “I don’t really want a back-and-forth” attitude makes me sad, but I think I understand where it is coming from. And if people don’t want that sort of discussion, that’s their choice.
My experience is that most people don’t want back and forth under any circumstances if they consider themselves to be of superior status to their interlocutor.
Unfortunately this includes cases where they are wrong, despite their superior status.
James:
Could be! But Elliott Morris is not of lower status than Nate Silver. Both are data journalists who’ve done work in election forecasting. Indeed, a quick characterization of Elliott would be, “younger version of Nate.”
My take on Nate’s position is not that he considers himself to be above it all. Rather, I get the impression that Nate’s been overwhelmed with feedback and criticism over the years, and it exhausts him, so he makes the reasonable decision to not take criticism seriously anymore but just to move on. I can see how this position makes sense: to the extent that the criticisms he’s receiving and dismissing are valid, he’ll still have a chance to do better in his future work. By not responding to criticism, he’s following a sort of implicit natural-selection-of-science attitude to his own work: criticism that is relevant in the future will continue to pop up, and at some point he can address it and do better, while criticism that is wrong or that won’t affect his future work can safely be ignored. This strategy could be an efficient use of his time—even if it annoys me a bit when he applies it to the thoughtful and serious criticisms that I’ve given him. From Nate’s perspective, Nate’s time and effort are valuable, and it’s his take how to allocate these resources. Similarly for David Brooks, Malcolm Gladwell, etc.
I think you’re missing some key indicators.
Nate spends a LOT of time pumping stuff out on Twitter. That’s indicative of an attitude. It’s not merely a question of his best use of time. How could spending a lot of time pumping stuff out on Twitter ever really be considered a useful expenditure of time? Why do you spend time on a blog? Why don’t you spend time on Twitter, which is explicitly designed to boost an echo chamber effect? Why are you offering your substack for free?
I think you’re giving Nate way too much credit here. I think that in part, that’s because you don’t spend much time on Twitter and so you don’t see how much time Nate spends pumping out total crap there.
Joshua:
People have asked me why I spend time blogging (and, even worse, responding to blog comments!), rather than doing research or exploring new ideas via books and research articles, or even simply making money by writing intro textbooks or something like that. The quick answer is that I blog because often I’m too lazy to do work, and when blogging I can reach a wider audience than sending emails. The longer answer is that I learn from blogging, both from the effort taken to try to express my ideas logically and clearly, and by engaging with feedback. I don’t agree with all the criticism I get, but often I learn a lot.
I can’t speak for Nate, but I expect he’s getting the same thing from twitter: it’s an informal way for him to work out ideas and to get feedback. It may be that much of what he “pumps out” there is “total crap”—and I guess some people would say this about this blog—, but this might be what he needs to do to work out his ideas. Pop out whatever comes to mind 100 times, and once or twice something sticks that’s valuable. I do think he’s losing out by not engaging with serious criticisms that come from me and others; hence the above post where I explore some possibilities.
Finally, the reason I offer the substack newsletter for free is that the purpose of the newsletter is to reach more people. I assume that free reaches more people than if I charged. Also I’m well paid and I don’t need the money. That said, it would be great to have a stream of money to support postdocs and programmers, and if I thought the blog and newsletter could bring in hundreds of thousands of dollars a year, I probably would charge for it. To go to all the trouble of charging and then only bring in $20K/year or whatever, though, that would hardly be worth it.
Andrew –
Point taken with you putting “pump out” and “total crap” in quotes, but as an observe with all my biases intact, there’s a world o’ difference between what you offer on this blog and what Nate puts out on Twitter (can’t speak to his Substack as I’ve rarely read it).
I agree with James above what comes across is that Nate thinks that he’s above it all. I see his engagement as one of walling himself off from engagement through aggressive and condescending interaction. That’s not to say that he doesn’t get aggressive and polemical engagement in return from others, no doubt that does, but there’s a reason why he gets so much more of that type of response than you do – and I’d say that the reason is largely a reflection of your different approach to respectful engagement.
I totally agree about working through ideas through engaging in less than fully-thought out experimentation; but it’s just really hard to believe that Nate’s working through ideas with his shallow and seemingly triggered engagement on Twitter. Maybe you haven’t looked at it much, but I say this as someone who has greatly enjoyed and respected the stuff he used to put out on 538.
I respect your inclination to give it all the benefit of the doubt and you may well be right. But I think you’re wrong on this and if you looked at more of his output you might change your mind. Again, there’s a reason why he engages on Twitter and you don’t, and I don’t think it’s because he thinks Twitter is a way to learn new things and work ideas out. Even to whatever degree that’s possible on Twitter (not much, imo), there’s nothing about the character of his engagement there that gives me an indication that it’s motivated that way.
Andrew:
In my view blogging, at least in the form you do here, should also be considered a form of work in your case (though perhaps that’s not a view you wish to take as it might ruin the fun)! As a research professor you live a life of the mind and as such, engaging in discourse which has a non-trivial chance of impacting your views, whether formal or informal, is part of the job.
I suppose what I’m saying here is that your view of “too lazy to do work” actually is…you still doing work. Perhaps this is one reason why you have as many solid publications in a single year as many others have in a decade of work.
I am very grateful for it in any case. My views of the world, and science, have been forever altered (for the better) due to reading this blog and engaging with your commentators. I’m happy to hear you’ll be keeping it free as I am not sure I would have stumbled upon this place if it was behind a paywall, and I hope others continue to have the same opportunity I did.
> This was before twitter, which has the different problem that most of the volume of posts is people cheering or booing
FWIW (anecdote, not evidence) I have found it almost impossible to engage in back and forth on Twitter although I do try to do so. In my perception, very few on Twitter are interested in back and forth. I even think I’ve noticed a trend away from back and forth. Most times, when I try to critique opinions on Twitter, I get “blocked.” At best I get ignored. It’s hard to evaluate what’s going on with the ignored user because I never know how the algorithm affects visibility of my comments.
As for this –
> And so you develop . . . not a thick skin, exactly, but an acceptance that you have neither the time or interest in responding to all the uninformed and possibly insincere criticism that you’re receiving.
I don’t agree that’s a dominant casualty chain.
I think more explanatory is a kind of capture from fame and a kind of audience capture. People like being famous. They like getting positive feedback for their views. They like the money that comes among with that. It’s just more beneficial to keep pumping out hot takes and getting money and positive feedback doing so. And there are increasingly more market mechanisms for this process. Podcasts. Substacks. Lots and lots of people pay you $7 a month to read your hot takes because they agree with you. No back and forth needed. The people that disagree are a pain in the ass. And what’s more, they don’t pay you money!
I admire your lack of interest in following that path. It’s to your credit.
There are some exceptions. Some who fall into a middle ground. Scott Alexander might be one. He, and commenters at his site, actually engage in back and forth rather than slip into a more echo chamber dynamic. But then again, the stuff he does is on a big much more than on substack or Twitter.
“You get lots of uninformed people attacking you because they don’t misunderstand what you’re doing or you were pooh-poohing their favorite JFK-assassination theory or UFO’s-as-space-aliens theory or you’re not taking their exact political position on the issue of the day or whatever”
Yet he is playing the role of all-pundit on topics he is clueless about like covid origins with minimal research beyond “stuff I read off twitter”
I don’t think this is just an issue for pundits, anyone who has read argumentative mailing lists or forums can see the basic problem. Because of Brandolini’s law, if you give every criticism an open-minded consideration, people of ill will can force you to spend hours of concentration with a few minutes of BSing. This is one reason why the most useful critical environments are relatively small and force all parties to follow certain norms.
If Scylla is ignoring all criticism and new evidence, Charybdis is paying close attention to every random Internet person who wants to argue. Sailing between them is hard.
Very nicely worded.
I think Andrew’s model here is generally correct. Obviously people attempt to select what feedback they engage with (positive, rather than negative), but you can only explain yourself to various iterations of the same question so many times before you just stop taking questions. Nate also cites to the bullshit asymmetry principle in explaining his anti-back-and-forth position. Does he think critiques are poop because of status or because they really are often poop? I think it is popular to see someone with status and say it’s because of their status, but it could very well be largely due to the (lack of) quality of the critiques. Even if you think of a situation of relatively high quality critiques with journal articles + commentaries, there are only so many rejoinders that are appropriate.
Daniel:
Yes, and then it becomes a habit, as can be seen when Nate dismisses statistics discussions coming from me!
Yes, to deal effectively with high volumes of criticism from people who don’t know or care what they are talking about, you have to learn to instinctively slot criticism into categories and make one of a short list of pre-programmed responses (“ignore it” is one of those responses). And learning to do this with “system 1” gets in the way of using “system 2” to carefully consider the merits of a specific argument from a specific person. I know very few people who can both respond rhetorically to high volumes of low-quality criticism, and respond scientifically to low volumes of high-quality criticism.