Stop screaming already: Exaggeration of effects of fan distraction in NCAA basketball

John Ezekowitz writes:

I have been reading your work on published effect sizes, and I thought you might be interested in this example, which is of small consequence but grates me as a basketball and data fan. Kevin Quealy and Justin Wolfers published an analysis in The NYT on fans’ effectiveness in causing road teams to shoot worse from the free throw line in college basketball.

In the piece, Wolfers notes that players shoot better at home than on the road, but then compares “fan effectiveness” by looking at how much worse opponents shoot at a given arena vs. their home arena. I think it is pretty clear that the correct comparison is opponents’ road FT shooting, not their home shooting.

When I asked him about this, he admitted that the road vs. road effect was smaller. It looks like he just picked home vs. road because he could show a bigger “effect size.” This feels symptomatic of the larger problems you have continued to highlight on your blog.

By the way, not sure if you watch any Columbia basketball, but Maodo Lo can really, really play.

Good point (not about Maodo Lo, that I have no idea about, but regarding the NYT article). The goal of newsworthiness can get in the way of clear communication.

Specifically, Quealy and Wolfers wrote:

On average, college basketball players are about one percentage point less likely to make a free throw when in front of a hostile crowd than when at home. . . . On average, the sixth man’s ability to distract opposing free throwers is worth about 0.2 points per game.

Hmmm, 0.2 points a game is pretty irrelevant anyway. But they get estimates of over 1 point per game for a few teams, most strikingly Arizona State and Northwestern.

Or is it just (or mostly) noise? Quealy and Wolfers write:

Some of the tremendous variation among teams may reflect statistical noise, given that we’re evaluating only five seasons’ worth of data. But that’s still enough to suggest that the overall patterns are real.

They provide no quantitative evidence that for this claim. All they give is this graph:

Screen Shot 2015-07-20 at 10.39.14 AM

This graph looks consistent with a small difference attributable to home-court advantage (recall Ezekowitz’s point), but I see no evidence, from this graph alone, that the differences between stadiums are real. I just don’t know.

Quealy and Wolfers write:

There are also a handful of arenas where visiting teams have actually hit a greater share of free throws than they typically do in front of their home fans. Boston College and Notre Dame are two prominent examples. It’s unfair to suggest that these fans actually hurt their team; rather, it’s more likely that they were of little or no help, and random luck means that visitors hit a few extra free throws.

Whoa baby. Hold up right there. First, according to the graph, it’s not “a handful” of teams, it’s about 110 of them. Second, that’s fine to credit these patterns to random luck. But then shouldn’t you also be considering random luck as an explanation for the success of certain teams?

And what’s with this sort of data dredging:

Duke’s Cameron Crazies are among the most famous fan groups in any sport in the country. And to some extent, they live up to their hype. Our data ranks them as one of the more distracting teams in the nation, although they’re outside our top 10. It could be that they’re actually better than that, and that their numbers will improve with more seasons of data. Or perhaps their creativity does not match their intensity.

Here’s the bad news for Duke fans: Their main rivals, the fans in Chapel Hill, have them slightly beaten here. North Carolina’s fans help the Tar Heels to the tune of about two-thirds of a point per game, relative to a typical home crowd.

This is getting ridiculous. These guys could give a story to coin flips.

What’s really needed here is a hierarchical model. Or, simpler than that, let’s just try computing these summaries for each arena in each season, and see if the arenas with these free-throw patterns in season 1, also show the patterns in season 2. At its simplest, if the differences between arenas are all noise, the year-to-year correlation between these results will be essentially zero. Next step is to fit a hierarchical model with arena effects and arena*year interactions.

7 thoughts on “Stop screaming already: Exaggeration of effects of fan distraction in NCAA basketball

  1. Here’s a test: what about gyms that don’t have seating behind the baskets so opposing free throw shooters would be less distracted? Are there any like that? It was pretty common when I was at Rice U. in the 1970s to draw a big curtain in front of the seats behind the baskets because the crowds were so small until Ricky Pierce showed up.

  2. There are so many intervening factors here that this result is not believable. First, patterns of fouling will vary from team to team depending on personnel and coaching philosophy. Second, some fouls are strategic either targeting specific players because they’re terrible at free throws that it’s better to foul than let them score (i.e., hack a Shaq) or the losing team is fouling to slow the clock at the end of a game. Third, whether teams are over 6 fouls or not means they’ll be taking 1-1 free throws (player gets another shot only if he makes the first) or 2 guaranteed. Fourth, a player shooting a free throw to tie or win the game at the end is under more stress than one shooting earlier in the game. Other things I can think of are margin of victory and conference (Big East vs Big Sky). There are probably things that I overlooked but a measly .2 points is easily washed away or strengthened in any of these scenarios.

Leave a Reply to Steve Sailer Cancel reply

Your email address will not be published. Required fields are marked *