Recently in the sister blog

Meritocracy won’t happen: the problem’s with the ‘ocracy’

Does the sex of your child affect your political attitudes?

More hype about political attitudes and neuroscience

Modern polling needs innovation, not traditionalism

Who cares about copycat pollsters?

The mythical swing voter

Mythical swing voter update

No, all Americans are not created equal when it comes to belief in conspiracy theories

Six does not just mean a lot: preschoolers see number words as specific

7 thoughts on “Recently in the sister blog

  1. Hi Andrew,

    You correctly identified a subtle error in how Moore et al. assessed a Krugman claim, so it seems fair to observe a similar error that you appeared to make assessing one of their claims in “No, all Americans are not created equal when it comes to belief in conspiracy theories.”

    You wrote:

    “[Moore et al.] also write, ‘When science means nuclear weapons, innovation and winning the space race, conservatives love it.’ Actually, when I last looked at the data, I found that ‘support for the space program does not seem particularly associated with conservative or Republican positions.'”

    However, the fuller quote from Moore et al. is “When science means nuclear weapons, innovation and winning the space race, conservatives love it. And when they associate it with the EPA, regulation, and global institutions, they hate it.”

    You assessed Moore et al.’s claim about conservative support for the space program by linking to a description of evidence of a small correlation between support for federal spending on space and party identification or ideology. But the Moore et al. claim is not comparing conservatives to liberals; instead, the claim is an absolute claim about conservative support for the space program, or arguably is comparing conservative support for one type of science to conservative support for another type of science. In either case, it does not seem that these claims can be assessed with correlations between support for federal spending on space and party identification or ideology, given that the Moore et al. quote did not use Democrats or liberals as an implicit or explicit comparison category.

    The larger point of the Moore et al. quote is that conservatives are not anti-science, but that claim cannot be assessed through the cited correlation, either; if anything, the small correlation can be interpreted as evidence that conservatives are not any more anti-science than liberals are, at least with regard to support for federal spending on space.

  2. Got to disagree with you on not worrying about copycat pollsters. There are plenty of ways to manipulate the data that would still work even if you released the raw data.

    For instance: last minute changes in the PID targets in the weighting scheme, dropping some inconvenient “outlier” observations or otherwise messing with the sample or the weights.

    This wouldn’t be impossible to identify but I imagine some organizations would get away with it for some time before anyone noticed and I suspect that most journalists would never notice.

    • Jon,

      Sure, you’d want them to release the raw data plus whatever methods they used to get their estimate. Again, no need for them to share all this information to the general public, but they should make it available to the news organizations. Then, if a news organization wants, they can always go back later and check, or get a professional to check.

      • I guess my concern here is that there is a large set of “broadly defensible” ways you could run and weight a poll (or even by just releasing the 1 in 10 polls that are closest to the gold standard pollsters). The issue is if those ways are not chosen because the pollster thinks they are best but because they lead to the right result that time. So a professional being consulted would not flag up any issues with that poll because it still falls within the mainstream set of techniques.

        This is analogous to p-hacking and researcher degrees of freedom. It’s not that the published methods and results in those papers are wrong in themselves (sometimes), it’s just that they chose them post-hoc to reach a particular result. And in both cases it’s very hard to identify a case of hacking in isolation.

        It’s harmful to consumers of polling data because we’re faced with a polling-hacked set of information that may lead us to have greater confidence about the state of a race than is merited from the actually independent sources of information.

  3. The big difference in conspiracy theories is what gets labeled a “conspiracy theory.” For example, not believing in global warming makes you a conspiracy theorist, but not believing in IQ testing makes you a respectable, responsible science-minded person.

    Similarly, Eric Holder ordering a third autopsy of Michael Brown only makes sense under the craziest conspiracy theory assumptions, but who calls out the Obama Administration for conspiracy theorizing.

Leave a Reply

Your email address will not be published. Required fields are marked *