Eric Loken writes,
Criteria Corp is a company doing employee testing (basically psychometrics meets on-demand assessment). We’re also going to blog on various issues relating to psychometrics and analyses of testing data. We’re starting slowly on the blog front, but a few days ago we did one on employment tests for the NFL.. A few scholars have argued that the NFL’s use of the Wonderlic (a cognitive aptitude measure) is silly as it shows no connection to performance. But we showed that for quarterbacks, once you condition on some minimal amount of play, the correlation between aptitude and performance was as high as r = .5…which is quite strong. It’s the common case of regression gone bad when people don’t recognize that the predictor has a complex relationship to the outcome. There are many reasons why a quarterback doesn’t play much; so at the low end of the outcome, the prediction is poor and the variance widely dispersed. But there are fewer reasons for success, and if the predictor is one of them, then it will show a better association at the high end.
Here’s their blog, and here’s Eric’s football graph:
P.S. The graph would look better with the following simple fixes:
1. Have the y-axis start at 0. “-2000.00” passing yards is just silly.
2. Label the y-axis 0, 5000, 10000. “10000.00” is just silly. Who counts hundredths of yards?
3. Label the x-axis at 10, 20, 30, 40. Again, who cares about “10.00”?
I’ve complained about R defaults, but the defaults on whatever program created the above plot are much worse! (I do like the color scheme, though. Setting the graph off in gray is a nice touch.)