Heat map

Posted on by

Jarad Niemi sends along this plot:

heat.png

and writes:

2010-2011 Miami Heat offensive (red), defensive (blue), and combined (black) player contribution means (dots) and 95% credible intervals (lines) where zero indicates an average NBA player. Larger positive numbers for offensive and combined are better while larger negative numbers for defense are better.

In retrospect, I [Niemi] should have plotted -1*defensive_contribution so that larger was always better. The main point with this figure is that this awesome combination of James-Wade-Bosh that was discussed immediately after the LeBron trade to the Heat has a one-of-these-things-is-not-like-the-other aspect. At least according to my analysis, Bosh is hurting his team compared to the average player (although not statistically significant) due to his terrible defensive contribution (which is statistically significant).

All fine so far. But the punchline comes at the end, when he writes:

Anyway, a reviewer said he hated the figure and demanded to see a table with the actual numbers instead.

Oof!

12 thoughts on “Heat map”

  1. The scale looks like points per 48 minutes. It's a nice plot, the only change I would make is to present the defensive numbers as negative defensive numbers, since values below zero indicate good defense, and Miami has one of the best defenses in the league.

  2. Niemi (and edk) you're very right about reversing the sign for defensive ratings. Other than that, I like the plot a lot. I really don't believe anyone would prefer a table. I know you say someone does, and I can't see why you'd lie about that. I also can't see why the reviewer would lie about it. And yet, I don't believe it. Maybe you made up the story about the reviewer, just to have a punch line?

  4. I can believe the reviewer didn't like it – though I agree that it's a useful plot.

    I've struggled to get similar displays included in scientific papers – as a replacement for (familiar) grouped barplots. Once people take a few seconds to orient themselves to the graph, they're fine, but getting that far involves overcoming a lot of inertia.

  5. Prof. gelmand and commenters,

    When faced with such a reviewer, do you just change the graphic for a table? Or you put both, or yet you use a footnote to say that the table was requested by a reviewer? Or you argue with the editor?

    thanks fot the advice,

    Manoel.

  6. I can see the reviewer's point.

    Any plot with "good" going in two directions is a difficult plot.

    There's no ordering given on the Y axis (salary? minutes? overall contribution?)

    Is there some way in which the x axis is equivalent on offense and defense (not just zero point, but scale)?

    The major point seems to be Bosh bashing; if that's the case why not have him first? (or at least the big 3 first).

  7. zbicyclist, I agree that the plot needs to be changed, but I think Niemi is saying the reviewer didn't want a plot at all, he wanted a table instead! I think Andrew goes too far when he says _all_ plots should be tables instead, but this is a case where a good plot would be, conservatively, a zillion times better than a table.

  8. Manoel: it depends. What are the chances the paper be accepted, either way? How much do I think of the paper's results, or the journal – enough to present my message less-than-perfectly? Will the other authors even let me submit the paper with a figure not a table? What other things in the paper are likely to cause disputes?

    Sorry to be so pragmatic, but often other battles have to be prioritized.

  9. zbicyclist and phil: the ordering on the y axis seems to be the lower limit of the overall player contribution interval. Admittedly, that is a pretty arbitrary ordering. As it is, the list shows players ranked on their worst night.

  10. zbicyclist's list is great for improving this chart. Sorting out the order is the most important thing. I'd also use a small-multiples format instead of having all 3 series on one chart.
    why can't journals accept charts in the main text, and tables in the appendix?
    typically tables that try to convey both mean and intervals are unreadable.
    the problem with interpreting this model (I'm going off topic) is that the intervals are so wide that there are few statistically significant pairwise differences; even in cases where they exist, we run into the nontransitive problem which is difficult to explain to non statisticians.

  12. I have been behind on my RSS reading and just saw this post come up and thought I would respond to some of the questions. First off, I went back to the original referee response and the quote is

    The pictures are nice, but it would have been useful to also have some tables with estimates and standard errors.

    So contrary to my email to Andrew (and therefore my memory), this suggestion is very reasonable. I apologize for my inaccurate paraphrasing and can only believe that because the rest of the review was so negative (although also very reasonable) that I took this comment much harsher.

    Yes, the list is ordered by lower end of the 95% CI. If the median is used to order, the list makes much less sense since you have players with huge uncertainties having large medians (despite the model shrinkage). So the lower end of the 95% CI is closely related to the probability that the estimated parameter is greater than zero, a less arbitrary cutoff.

    Yes, the x-axes are similar in that it represents points per game relative to an average player. So 10 is 1) 10 more points for your team, 2) 10 more points for your opponents, and 3) 10 point differential [your team minus opponent]. This is the reason I originally did not switch to -defense.

    The main complaint of the reviewer and Kaiser's point is absolutely true. Namely despite lots of data, it appears difficult to distinguish player abilities.

Comments are closed.