Regular readers of this blog are familiar with the pinch-hitter syndrome: People whose job it is to do just one thing are not always so good at that one thing. I first encountered this when noting the many silly errors introduced into my books by well-meaning copy-editors with too much time on their hands. As I wrote a few years ago:
This is a funny thing. A copy editor is a professional editor. All they do (or, at least, much of what they do) is edit, so how is it that they do such a bad job compared to a statistician, for whom writing is only a small part of the job description?
The answer certainly isn’t that I’m so wonderful. Non-copy-editor colleagues can go through anything I write and find lots of typos, grammatical errors, confusing passages, and flat-out mistakes.
No, the problem comes with the copy editor, and I think it’s an example of the pinch-hitter syndrome. The pinch-hitter is the guy who sits on the bench and then comes up to bat, often in a key moment of a close game. When I was a kid, I always thought that pinch hitters must be the best sluggers in baseball, because all they do (well, almost all) is hit. But of course this isn’t the case–the best hitters play outfield, or first base, or third base, or whatever. If the pinch hitter were really good, he’d be a starter. So, Kirk Gibson in the 1988 World Series notwithstanding, pinch hitters are generally not the best hitters.
There must be some general social-science principle here, about generalists and specialists, roles in an organization, etc?
This idea was recently picked up by a real-life baseball statistician–Eric Seidman of Baseball Prospectus–who writes:
I wanted to talk to you about the pinch-hitter theory you presented, as I’ve noticed it in an abundance of situations as well.
When I read your theory it made perfect sense, although a slight modification is needed, namely in that it makes more sense as a relief-pitcher theory. In sabermetrics, we have found that pitchers perform better as relievers than they do as starters. In fact, if a starter becomes a reliever, you can expect him to lop about 1.4 runs off of his ERA and vice-versa, simply by virtue of facing batters more often. When you get to facing the batting order the 2nd and 3rd time through, relievers are almost always better options because they are fresh. Their talent levels are nowhere near those of the starters–otherwise, they would BE starters–but in that particular situation, their fresh “eyes” as it pertains to this metaphor are much more effective.
For another example, when working on my book Bridging the Statistical Gap, I found that my editor would make great changes but would miss a lot of ancillary things that I would notice upon delving back in after a week away from it. Applying that to the relief pitcher idea, the editor was still more talented when it came to editing, but his being “in too deep”, the equivalent of facing the opposing batting order a few times, made my fresh eyes a bit more accurate.
I’m wondering if you have seen this written about in other areas, as it really intrigues me as a line of study, applying psychological concepts as well as those in statistics.
These are interesting thoughts–first, the idea of applying to relief pitchers, and, second, the “fresh eyes” idea, which is more adds some subtlety to the concept. I’m still not quite sure what he’s saying about the pitchers, though: Is he saying that because relief pitchers come in with fresh arms, they can throw harder, or is he saying that, because hitters see starters over and over again, they can improve their swing as the game goes on, whereas when the reliever comes in, the hitters are starting afresh?
Beyond this, I’m interested in Seidman’s larger question, about whether this is a more general psychological/sociological phenomenon. Do any social scientists out there have any thoughts?
P.S. I seem to recall Bill James disparaging the ERA statistic–he felt that “unearned” runs count too, and they don’t happen by accident. So I’m surprised that the Baseball Prospectus people use ERA rather than RA. Is it just because ERA is what we’re all familiar with, so the professional baseball statisticians want to talk our language? Or is ERA actually more useful than I thought?
Perhaps there is endogenous entry as well. Maybe people who aren't good at anything are asked to do only one thing.
Prof. Gelman, I'm amazed that you would apply your experiences to the entire field of copy editing.
Let me suggest that there may well be other factors to consider:
Yes, too much time on their hands may be one.
Too little could be another.
Pressure to show they are doing something, anything, to justify their jobs.
Style requirements about use of Latin that perhaps you aren't aware of.
Lousy pay, attracting poor quality workers.
Poor management that allows people to run amok.
I haven't been reading your blog long enough to know what errors you've found, other than the mention of "e.g." instead of "for example." I'll poke through your archives to see what else you've cited in the past.
I enjoy your writing and look forward to reading you in the future.
Style requirements about use of Latin that perhaps you aren't aware of.
Style requirements about use of Latin of which you are perhaps not aware?
@shallnotput,
Ironic? Yes, but Pam was writing a blog comment, and I actually prefer her version, all things considered, because it sounds more natural.
Prof. Gelman, I don't think you're giving due consideration to the fact that copy editing is one of the many jobs you would be far better at than most people.
@shallnotput
That's the sort of nonsense up with which I will not put.
— attributed to Sir Winston Churchill
ERA isn't so bad, unless you're naively comparing starters to relievers or pitchers on good defending teams to pitchers on poorly defending ones. Really all of those faults are really shared by RA.
There are better measures, but few that don't require a lot of explanation.
Dr. Gelman, as a former psych (and stats) professor, perhaps I can be of some help.
There are a number of psychological factors in play on this one. To start on the proofreading issue:
1) Simple attention. Proofreading is hard because it takes sustained attention and it's hard to sustain attention for extended periods of time. The brain gets tired. The more tired you are, the worse you are at it. It has nothing to do with ADHD. People have a much shorter attention span than they give themselves credit for.
2) Functional Fixedness. Here's a simple brain teaser. You are put into a room with a box of 1/2" nails, a hammer, and a candle with a 1" diameter base (and nothing else). Using only those items, your job is to hang the candle on the wall. Note that because the nails are so small, you won't be able to use the "pound a row of them into the wall and set the candle on them… the nails won't be able to get under the candle's center of gravity. How to do it… answer in a moment. ;-)
3) The Stroop effect. Think of the word "red" written in blue ink. If I asked you to read the word, you'd have no trouble. If I asked you to tell me the color of the ink, it would be hard.
Why? Because reading is much more automatic than color naming. You know your colors just fine, but it's so tempting to say "red" to the first one. When proofreading, you get caught up in the message of the text itslef (a very automatic process) and you don't even notice that I mispelled the word "itslef" half a line ago.
The answer to the brain teaser in #2 is to empty out the box of nails, take one of the nails and use that to affix the box to the wall and use that as a candle holder. You don't stop to think about the box as anything other than "something that holds nails." You suffer from functional fixedness.
Relating it all back to baseball:
There might be something to be said for starters having tired eyes (or more to the point, a tired pre-frontal cortex.) Of course, starters also have tired arms going into their third trip around the lineup, but here's an interesting study that could be done empirically. The brain, when it isn't able to process information fully has a fail-safe. It plugs in information based on past experiences. It's a shortcut that saves energy. If the pre-frontal cortex is tired (the area of the brain that deals with planning… like planning what I'm gonna throw this batter in this at-bat), it may simply say "Hey, you got him out with a fastball down and away last time. Try that again." Pitch F/X can tell us whether pitchers begin repeating pitch sequences, particularly when they've been out there a while. (A good hitter will quickly pick up on sequencing… with bad results for the pitcher…)
Functional fixedness and automatic processing abound in baseball. It's the reason why teams spend so much time trying to find "a good closer candidate" when they draft, but never stop to question whether having a "closer" is a good idea. Again, this probably plays out in pitch sequencing on the pitcher's part. A starter who has gotten two grounders from the guy at the plate with outside curves will focus on locating the curve ball again to get the same result rather than thinking "if I went in on his hands with the fastball, he'll probably pop it up." The point is to get outs, not get ground balls. The reliver coming in with a fresh outlook is more likely to try stuff like that.
It probably also has something to do with the pinch-hitter effect. Once a guy is on the roster, he has to have a role. Right? So, you give him the role of PH, often without a question of why he's on the roster to begin with. I have to wonder how many MLB players have jobs right now "because they're here."
I can answer your two baseball questions. I'm not a huge sabergeek, but I've been around enough of them that these are obvious to me.
Both are true, but he's talking about the latter.
A bit of both, but mostly the first. There are better stats than ERA for measuring pitcher skill. If he were seriously examining pitchers, he would use something like FIP or tRA. But he's just making a simple general point here, so the familiar ERA is good enough.
Although it easy to judge for pinch hitters as the statistics are readily available, can you be really sure it's appilcable in your situation?
I mean, does it make sense for someone less than an expert to judge an expert? That's essentially what you're doing. Perhaps the copy editor knows some more obscure rule of style that means using i.e. and e.g. is more consistant with how you have written the rest of the document.
And anyway, presumably the pich hitter only goes in when the stakes are high and if s/he was randomly swapped out with the usual player then we may find that the pinch hitter *was* performing better in that situation.
Megan: Pinch hitters are called in to replace the #8 hitter when there's some kind of lefty/righty pitcher shift. They're not called in to replace the #3 batter, which is how I think of myself!