No, I don’t buy that claim that Fox news is shifting the vote by 6 percentage points

Tyler Cowen writes:

This is only one estimate, from Gregory J. Martin and Ali Yurukoglu, but nonetheless it is backed by a plausible identification stragegy and this is very interesting research:

We find that in a hypothetical world without Fox News but with no other changes, the Republican vote share in the 2000 election would have been about half a percentage point lower. By 2008, the effect of there being no Fox News rises to more than six percentage points – a result of the channel’s increasing viewership and increasingly conservative slant over this period.

I am skeptical, for reasons discussed in this post from a couple years ago. It’s a problem of extrapolation.

Here’s what I wrote in 2016:

[Martin and Yurukoglu] found that Fox’s impact took off during the decade that followed: It increased Republican vote share by more than 3.5 percentage points in the 2004 election and more than 6 percentage points by 2008. . . .

Where did that 6-percentage-point estimate come from? I clicked through to Martin and Yurukoglu’s article, which begins as follows:

We measure the persuasive effects of slanted news and tastes for like-minded news, exploiting cable channel positions as exogenous shifters of cable news viewership. Channel positions do not correlate with demographics that predict viewership and voting, nor with local satellite viewership. We estimate that Fox News increases Republican vote shares by 0.3 points among viewers induced into watching 2.5 additional minutes per week by variation in position. We then estimate a model of voters who select into watching slanted news, and whose ideologies evolve as a result. We quantitatively assess media-driven polarization, and simulate alternative ideological slanting of news channels.

That 0.3 percentage points is a lot less than 6 percentage points. You can, however, get pretty close if you scale up from 2.5 minutes per week to 1 hour per week: 0.003*60/2.5 = 0.072, and maybe the difference between that and 0.006 can be explained by rounding. (For example, if the reported 0.3 is a rounded 0.026, then you get 0.026*60/2.5 = 0.0624, which rounds to 0.06.)

I then searched the Martin and Yurukoglu paper for the 6-percentage-point estimate. I found it in table 14, which reports the estimate that were Fox News to have disappeared in 2008, the Republican candidate would’ve lost 6.3 percentage points of the vote.

President Obama beat John McCain in 2008 by the margin 53 percent to 46 percent, so according to this model, had Fox News that year disappeared (or, I suppose, switched to a politically neutral format), Obama’s electoral margin would’ve been a Reaganesque 59 percent to 40 percent.

I don’t believe it.

What, then, went wrong in this analysis? Lots of little things, many of which are indeed mentioned by Martin and Yurukoglu. First is the extrapolation from 2.5 minutes per week to an hour per week, which assumes a linear effect (no diminishing returns) and also which takes the estimate far from what can be seen directly from the data.

Second, there’s the assumption that a change in Fox News would happen in a vacuum.

Third, there’s uncertainty. I’ll take the authors’ word that these estimates are statistically significant—that is, that one would not see such a pattern from chance alone—but there’s still going to be a lot of variation in these numbers. And in such settings, effect sizes tend to be overestimated. Estimates near zero are discarded and high estimates are reported. We call this the “statistical significance filter.” . . .

I think Martin and Yurukoglu’s paper is interesting and I think they’re admirably careful both in their presentation and their summary. They very appropriately gave their empirical estimate of 0.03 percentage points in the abstract, putting the larger claims deep in the paper with lots of qualifiers. [In their report of this study in Slate] Fisman and Prat were also careful to report that their conclusions were based on a model.

Still, something went wrong in the presentation, because the Slate article reads like this out-of-control 6-percentage-point extrapolation is real. . . .

For further background, see this 2014 paper by political scientists Dan Hopkins and Jonathan Ladd, who analyze data from a 2000 pre-election poll and find a positive effect of Fox News on support for George W. Bush, but “only on the vote intentions of Republicans and pure independents.” In summarizing this study, Hopkins writes that media influence “fosters political polarization. For Republicans and pure independents, Fox News access in 2000 reinforced GOP loyalties.”

I recognize that Cowen expressed mild skepticism in his post (“This is only one estimate”) but I don’t think he was skeptical enough! All the identification in the world doesn’t resolve the problem of extrapolation.

Again, I say this not to criticize the work of Martin and Yurukoglu. To say that we have to be careful about overintepreting a research result is not to say that the research is bad.

10 thoughts on “No, I don’t buy that claim that Fox news is shifting the vote by 6 percentage points

  1. “To say that we have to be careful about overintepreting a research result is not to say that the research is bad.”

    Exactly, thank you. Not every research paper definitively determines the future of the universe! :). In fact, lets be honest, most papers dont tell us much definitively at all. They’re small pieces in the making of something larger, sure, but many and probably most will end up on the cutting room floor. That’s expected. Call NPR!! LATEST RESEARCH ISNT THE LAST WORD EVEN IF IT CONFIRMS OUR BIASES!!

  2. Another potential flaw with their methodology is that fox news channel positions might be correlated with CNN, MSNBC, etc. channel positions, but I think I’d expect this to actually bias effect estimates toward the null…

  3. I guess this is more of a reply to your 2016 blog post than what you’ve added now, but I think you’ve misinterpreted what they’re saying.

    They consider a “model of voters who select into watching slanted news, and whose ideologies evolve as a result”. So they’re not saying that if Fox News had disappeared in 2008, Obama would have won 59 to 40. After all the “ideologies evolve” part presumably takes a couple of decades at least. Rather this is what would have happened if Fox News had never existed.

    And of course if Fox News had never existed but market forces had resulted in something pretty similar (“Wolf News: Balanced. And Fair.”), then I don’t think they’d predict that there’d be a shift from what we have now.

    I think they are saying that if Fox News had never existed and if something had stopped right-wing media from developing, then you’d see the 6 point shift. Stopping right-wing media from existing would require a large difference in culture or state intervention. I’m not an expert, but it seems that state control of media can result in a much bigger shift in popular opinion than 6 points (e.g. Putin’s popularity), so their model is at least in the correct range.

    And it still wouldn’t have resulted in Obama beating McCain by 19 points. Instead, McCain would have run more or less on Obama’s platform, and Obama would have advocated something closer to the UK’s Labor Party. (After all, in the days before Fox News, Nixon’s preferred tax rates and solutions to the healthcare problem were a bit to the left of Obama’s).

    • “it seems that state control of media can result in a much bigger shift in popular opinion than 6 points”

      Good point. And Fox News is essentially state controlled media when Republicans are in power. I know, it’s not like other news sources are banned, but that only mostly (not completely) undermines my argument by rhetorical elision.

      • Well, I certainly agree that Fox News often seems to function simply as the mouth piece of the Republican Party (and as such might be expected to have a large influence on popular opinion).

        But I was trying to make something like the opposite point: Given the evident market for something like Fox News, not having Fox News (or an equivalent) would require either a huge cultural shift or government intervention (to prevent right-wing media). In either case, a shift of public opinion of 6 points or more shouldn’t be surprising.

        • Maybe a DAG is in order here, or not. It seems like a FUQ (Fundamentally Unindentified Question). What does it mean to stop fox news to exist? If it means that state banned right wing news, then how is it possible that? What other changes in politics and society may cause this? And all this other changes will have an impact on Republican vote through other channels (other arrows from other variables), and the effect is not identifiable anymore. Is is?

  4. “President Obama beat John McCain in 2008 by the margin 53 percent to 46 percent, so according to this model, had Fox News that year disappeared (or, I suppose, switched to a politically neutral format), Obama’s electoral margin would’ve been a Reaganesque 59 percent to 40 percent.”

    Saying Obama would have had a victory margin similar to one that was observed just 30 years earlier is supposed to make the result seem implausible? In the midst of a huge recession presided over by an extremely unpopular incumbent? Seems like exactly the conditions where we’d expect another landslide, no? Of course, politics became much more polarized in those 30 years, making big victories more unlikely. But I know many would argue that Fox News is a major cause of that polarization…

    (All that said, I agree that the authors’ extrapolation is extremely suspect. I just don’t have such a low prior on the impact of Fox News.)

  5. “I recognize that Cowen expressed mild skepticism in his post (“This is only one estimate”) but I don’t think he was skeptical enough!”

    Hmm. Did you read the rest of it? Cowan’s entire post is expressing skepticism of the very extrapolation you’re talking about. Here’s the rest of it:

    “Unfortunately, that is followed by a real clunker of a paragraph:

    ‘All of these results suggest that citizens and regulators have reason to be concerned about media consolidation and the non-market objectives of media owners. A hypothetical monopolist controlling all three channels and interested in electoral influence would have enormous power over election outcomes.’

    How many things are wrong in those two sentences? How can a profession supposedly devoted to rigor allow such sloppy thought to continue? Here are a few of my objections:

    1. The real story in this paper is about Fox News, and Fox — whether you like it or not — is very much an alternative to the mainstream media approach. If you don’t like Fox, you might have preferred the “bad old days” of three dominant and pretty similar networks.

    2. Do the authors have any argument that “the non-market objectives of media owners” are bad? No. In fact, there is a longstanding literature that “the market objectives of media owners” are bad, whether you agree or not. Do they really just mean to say “I don’t like Fox News”? Just say it. Don’t worry, I don’t think most authors, especially of media studies, are objective to begin with.

    3. Don’t the results suggest we should perhaps be worried about polarized news rather than consolidated news ownership?

    4. Is it possible to consolidate news ownership in a world with so many cable channels and so many news alternatives to cable? I strongly doubt this, but in any case it is not something the authors have shown. Instead, they have shown that a renegade news channel can rise to a position of great political influence.

    5. Might it have been better simply to have written?: “I am really worried that Rupert Murdoch, in the absence of regulation, could buy up all the news channels and implement political outcomes I do not like.” That is an entirely coherent argument, and I wonder if it isn’t what the authors were getting at but couldn’t bring themselves to write it and thus were forced into the most illogical two sentences I have read this week.

    6. By the way, Murdoch owns a lot of media properties and most of them have political stances, and most of all tones, fairly different from that of Fox News. Worth a ponder.”

    • Kandal:

      No. Cowen is expressing belief in the empirical claim and skepticism about some of the interpretations of the empirical result. I’m expressing skepticism about the empirical claim itself. If you don’t believe the empirical claim, then all the later discussion is not so relevant.

Leave a Reply to Clifton Cancel reply

Your email address will not be published. Required fields are marked *