For the climate change question, I’m well situated to have an informed opinion: I have a degree in physics, two of my closest friends have studied the topic pretty carefully, and I’ve worked on a couple related research projects, one involving global climate models and one involving tree ring data.
In our climate modeling project we were trying to combine different temperature forecasts on a scale in which Africa was represented by about 600 grid boxes. No matter how we combined these precipitation models, we couldn’t get any useful forceasts out of them. Also, I did some finite-element analysis many years ago as part of a research project on the superheating of silicon crystals (for more details of the project, you can go to my published research papers and scroll way, way, way down). We were doing analysis on a four-inch wafer, and even that was tricky, so I’m not surprised that you’ll have serious problems trying to model the climate in this way. As for the tree-ring analysis, I’m learning more about this now–we’re just at the beginning of a three-year NSF-funded project–but, so far, it seems like one of those statistical problems that’s easy to state but hard to solve, involving a sort of multilevel modeling of splines that’s never been done before. It’s tricky stuff, and I can well believe that previous analyses will need to be seriously revised.
Notwithstanding my credentials in this area, I actually take my actual opinions on climate change directly from Phil: he’s more qualified to have an opinion on this than I am–unlike me, he’s remained in physics–and he’s put some time into reading up and thinking about the issues. He’s also a bit of an outsider, in that he doesn’t do climate change research himself. And if I have any questions about what Phil says, I can run it by Upmanu–a water-resources expert–and see what he thinks.
What if you don’t know any experts personally?
It helps to have experts who are personal friends. Steven Levitt has been criticized for not talking over some of his climate-change speculations with climate expert Raymond Pierrehumbert at the University of Chicago (who helpfully supplied a map showing how Levitt could get to his office), but I can almost sort-of understand why Levitt didn’t do this. It’s not so easy to understand what a subject-matter expert is saying–there really are language barriers, and if the expert is not a personal friend, communication can be difficult. It’s not enough to simply be at the same university, and perhaps Levitt realized this.
My friend Seth has a similar problem but even more so; he’s reduced to supporting his opinions from ideologically-minded journalists (see here and here). This is not to say he’s wrong on the substance–and I recognize that Seth’s attitudes on this topic come from diverse sources, not just one or two news articles–it still looks to me that he’s flailing around, grabbing all sorts of talking points off the web, without having the opportunity to talk with an actual expert. (I should also add that Seth’s primary interest here is not the climate science or even the policy issues–as Seth has written more than once, he supports restrictions on carbon emissions in any case–but in the more general issue of how scientists and others can be fooled into following a consensus.)
In any case, it’s not easy to evaluate expert arguments if you don’t have the right connections. For example, I don’t know what to think about macroeconomics. The economists I know do micro and, for that matter, I actually feel the ability to evaluate arguments in microeconomics all by myself (as regular readers of this blog may be aware). Macro is tougher, though: it confused me in 11th grade econ class and it’s confused me ever since. Sure, Paul Krugman’s an expert and has strong views, but I don’t know him. If Paul Krugman were my good friend, I’d probably be relying on his opinions here. By this I don’t mean that, just because someone’s my friend, that I trust him. Rather, when talking with close friends, I know enough about how they think that I can evaluate their arguments and get a sense of where they know more and where they know less.
Unfortunately, this mode of thinking isn’t “scalable,” as the expression goes. There aren’t enough experts around for everyone to have an expert as a personal friend. Even a person as well-connected as myself doesn’t know any experts in macroeconomics, and even someone as well-connected as Steven Levitt doesn’t know any experts in climate science, so what hope do the rest of youall have. I’d like to hope that you all think of me as your personal friend and trust everything I write on statistics, but, from your point of view, I guess that doesn’t make much sense!
Where have I disagreed with the scientific consensus?
Phil in his blog entries mentioned some National Academy of Science panel, and more generally, I think it makes sense to respect expert consensus, especially if, as with Phil, you have a sense of where the expert consensus is coming from. For example, when Seth links to a news report of some scientist saying that sea levels are falling, and the expert consensus says the opposite, I’m inclined to go with the experts.
But I think it would help to round this out with some discussion of areas where I’ve opposed the expert consensus, situations where I’m pretty sure that I’m right and the consensus was wrong.
For example, consider Bayesian statistics, which nowadays is standard if not hegemonic in many areas of statistical application and has a high enough status outside the world of statistics that it is often loosely used as a synonym for “rational.” It didn’t used to be so. When we came out with Bayesian Data Analysis in 1995, there was only one applied Bayesian textbook out there (Box and Tiao’s, from 1973, which had a pretty limited range of applications), and even as late as 1997, you had respected statistician Leo Breiman writing that “when big, real, tough problems need to be solved, there are no Bayesians.” Breiman was wrong even then–even setting aside Laplace and restricting oneself to the modern era, he was ignoring a couple of decades of hierarchical Bayesian work in application areas ranging from education to toxicology–but he was going with the scientific consensus. Or, to be more precise, the local scientific consensus of where he worked (the UC Berkeley statistics department) and the general scientific consensus of the statistics profession in his formative years.
Unfortunately, Leo Breiman didn’t have a “Phil” to educate him about Bayesian statistics. He worked in the same department as me, but he was on the 4th floor and I was on the 3rd floor, and we rarely saw each other–unfortunately, that’s just the way things were around there. Breiman was not a specialist in Bayesian methods or the foundations of statistics, and unfortunately the “Phil” figures he did rely on in this area were not well informed themselves.
Here’s the point, though: it’s not just that Breiman was wrong in a way consistent with the scientific consensus of his era; it’s that he was wrong in a way that was characteristic of being part of the consensus. Look at it this way: Suppose Breiman had had the same views about applied Bayesian statistics but without this consensus behind him. Then, at the very least, he would’ve been a bit more careful in his pronouncement, or the journal editor would’ve been a bit more careful about publishing his article. Suppose, to use Phil’s example, that a prominent biologist were to write that “when big, real, tough problems need to be solved, there are no Darwinists.” The author would need to defend his statement (and, no, keyword search on the word “data” is not enough), and the journal editor would do a double-take. But, given the scientific consensus, it was considered ok to say this. It’s not that the scientific consensus is stupid, it’s that some statements are so stupid that they only come because the speaker has processed some aspect of the consensus in a particularly ugly undigested form.
Now, I don’t want to overstate this. The scientific consensus in Leo Breiman’s mind in 1997 wasn’t much of a consensus at all. Bayesian statistics were everwhere, and the leading journal of applied statistics had already been edited more than once by Bayesians. Breiman was fighting a rearguard action. Nonetheless, he spoke in the voice of a consensus that once had been, and by shutting down his filters, it did not serve him well
I’ve also done battle with smaller consensuses, with one clear example (for me) being the consensus in the statistical literature that it makes sense to judge model fit using the distribution of discrete contingency tables conditioning on their margin. This is a difficult enough problem in high dimensions that it has become somewhat of a classic in Monte Carlo computation, but I think the underlying question makes no sense. To me, it’s another case where the existence of the consensus has switched off people’s brains.
From the other direction, coming from the outside, I tend to trust scientific consensuses when they are available. For example, I know little about biology, but I don’t see much need to listen to the supporters of Lysenkoism, or creationism, or whatever–what I have seen along those lines doesn’t seem very convincing. And, despite what the conspiracy theorist say, I think that an academic biologist with real evidence in favor of such offbeat ideas could go far these days. (One can distinguish between different sorts of consensus. In biology, my impression is that there is a consensus about evolution by natural selection and genetic drift, there is vigorous debate about speciation, and then there are various fringe ideas that are not so much rejected by the scientific consensus as living alongside the consensus, in different ways. I’m thinking here, for example, about various offbeat ideas on sex ratios and ESP that have been discussed now and again on this blog. There’s no biological reason these things can’t be true in some way, but they haven’t been demonstrated very convincingly, and they live in little worlds of their own.)
I imagine that if I were not a statistician, I’d have consensus views about statistics that were wrong, in the way that some of the received wisdom of consensus is wrong. Much as I feel uncomfortable with the conventional misconceptions about statistics held by non-statisticians (including people such as physicists, chemists, etc., who in some way should “know better”), I don’t see much room for escape from holding conventional views about other sciences that I have not been able to seriously investigate.
I don’t really have any strong conclusions here. On climate science, I can take the lead from the experts whom I know. On topics such as macroeconomics, I’m in the same boat as Seth and have little choice beyond going with the popular or semi-popular media. Most statistics questions I feel pretty confident in judging myself–although if causal inference is involved, I like to run things by Jennifer before expressing a firm opinion. When it comes to political science, I can ask the experts down the hall.
What do I recommend youall do? On subjects where Phil and I are the experts, I suggest you listen to what we have to say. Beyond that, I dunno.