Amy Cohen points me to this blog by Jim Manzi, who writes:
Ezra Klein and a variety of other thoughtful liberal bloggers have been pointing to an Economic Policy Institute analysis that they claim demonstrates that Wisconsin’s public employees, even after adjusting for benefits and hours worked, face a ” compensation penalty of 5% for choosing to work in the public sector.” Unfortunately, when you get under the hood, the study shows no such thing. . . . reading the actual paper by Jeffrey H. Keefe is instructive. Keefe took a representative sample of Wisconsin workers, and built a regression model that relates “fundamental personal characteristics and labor market skills” to compensation, and then compared public to private sector employees, after “controlling” for these factors. As far as I can see, the factors adjusted for were: years of education; years of experience; gender; race; ethnicity; disability; size of organization where the employee works; and, hours worked per year. Stripped of jargon, what Keefe asserts is that, on average, any two individuals with identical scores on each of these listed characteristics “should” be paid the same amount. . . .
I’m having difficulty extracting the main points from Manzi’s blog without just pasting it all in, so let me say that Manzi’s key idea is that if private sector employees in Wisconsin are being paid on average 5% more per hour than comparable public sector employees there, it’s not at all clear that this is a “compensation penalty” rather than simply that the private sector workers are worth 5% more, perhaps because they’re doing something more valuable or perhaps because they’re more qualified in ways other than measured by Keefe’s control variables.
Manzi concludes:
The whole question – as is obvious even to untrained observers – is whether or not there are material systematic differences between the public and private employee that are not captured by the list of coefficients in his regression model. His statistical tests simply assume that there are not.
I don’t know if Wisconsin’s public employees are underpaid, overpaid, or paid just right. But this study sure doesn’t answer the question.
That sounds about right to me. But I don’t think this sort of study is completely useless either. (Just to be clear: I haven’t actually followed the link to read Keefe’s report, so in writing about this study, I’m really writing about this study as described by Manzi.)
From one perspective, sure, I agree that a statistical analysis of the sort described above based on observational data can never be a true direct comparison. (Not to mention the difficulty of classifying people like me who work in the quasi-public sector.) But if you take things from the other direction, this sort of study can be valuable.
What do I mean by “the other direction,” you might ask? I mean, suppose you start, as people do, with raw numbers: Salary plus benefits = X% of the state budget. The state has Y number of employees. Average income of all Wisconsinites is Z. Then you start adjusting for hours worked, ages of the employees, etc etc, and . . . you end up with Keefe’s analysis.
My point is, people are going to make some comparisons. Comparisons aren’t so dumb as long as you realize their limitations. And once you start to compare, it makes sense to try to compare comparable cases. Taking Manzi’s criticism too strongly would leave us in the position of allowing raw numbers, and allowing pure unblemished randomized experiments, but nothing in between.
In summary:
1. Manzi’s right to emphasize that a simplistic interpretation of regression results can be misleading.
2. Regressions of observational data can be a good way of going beyond raw comparisons and averages.
Some of this discussion reminds me of the literature on the wage premium for risk, where people run regressions on salaries for comparable jobs in order to estimate how much people need to be paid to risk death or injury.. Based on my reading is that these studies can’t be trusted: if you’re not careful, you can easily estimate the value of life to be negative–after all, the riskiest jobs (lumberjack, etc.) tend to pay poorly, while the best-paying jobs (being Bill Gates, etc.) are pretty safe gigs. With care, you can get those regressions to give reasonable coefficients in the range of $1 million per life, but I don’t really see these numbers as meaning anything at all; they’re just the results of fiddling with the models until something reasonable comes out. I’m not saying that the people who do these analyses are cheating, just that they want reasonable results but the models seem too open-ended to be a good measure of risk premiums.
P.S. Ezra Klein replies, agreeing with Manzi’s statistical critique but writing that “the burden of proof is on those who say Wisconsin’s public employees make too much money.” I’m sure people can disagree about where the burden of proof should fall, but I think Klein’s point is similar to mine, that if you want to claim that public employees are overpaid, that claim will start with a comparison of some sort, and then you have to go from there.
The statistical analysis deserves to be panned. There is a much simpler way to determine if public employees are over or underpaid. Steve Landsburg addressed a similar question about public sector employees, replying to a Krugman post:
The idea is if a public sector employee is really underpaid, that implies that he or she has better opportunities elsewhere. "Better opportunities" can be in terms of whatever it is that the individual cares about in a job (hours, wages, benefits, sense of purpose, etc). Quit rates reveal preferences, which can encapsulate many more factors than a regression ever could.
I don't know what the quit rates show in Wisconsin, but this is the most sensible thing to look at.
You do tend to dismiss a lot I've learned in my econ classes.
The statistical value of life is quite a lot higher than 1 million USD. More in the range of 4 – 5 million.
The methods do make sense how I look at them. They ask the ceteris paribus question: given all kinds of characteristics that usually define wage, what does taking life risk adds to your income potential. Then they extrapolate. That last step is risky, totally true, but it is one of the few ways to get to a guestimate of what a life is worth.
Those methods are related to how actuaries make risk premiums, only the logic has been reversed. An insurance firm wants people to share risks. In the value of life perspective, people share the payout of death in advance in higher c.p. wages.
You need a value of life as a starter for all kinds of policy analysis. Without it, well, just look at the state of your healthcare debate. No point in telling which treatments are efficient. So lets just try all treatments, ey?
It's a Morton's fork. if public employees are paid more, they're obviously featherbedded; if they're paid less, they're obviously worth less. Every piece of data always supports the required conclusion.
Dave:
I agree that it makes sense to look at quit rates but that won't tell the whole story either. Nothing will. There's no magic number here.
Willem:
I'm quoting studies from the 1970s, when the value of a life was commonly assessed in the $200,000 range. It's gone up faster than inflation since then (perhaps because the workforce is older than it used to be, and risks are presumably more salient to older people).
In any case, I think you might have misinterpreted my comment. Follow the link. I agree that the idea of value-for-life is important and useful. I've used the concept in my own articles and books. The problem comes in the implementation of those regressions, which you have to tune pretty carefully to get numbers that seem right based on other criteria. Liking a concept (in this case, value of a life, qalys, etc.) does not require that I like a particular method of conceptualizing it.
Derek:
I'd expect public employees to be paid more in some sectors and less in others. The jobs are not completely substitutable.
RE quit rates:
It seems to me that using quit rates to assess compensation accounts for more than it ought to: i.e. some sense of duty/altruism.
Presumably, many public sector employees take and keep their jobs because they find them meaningful in other-directed ways. The sort of emotional compensation provided by fulfilling some sense of duty seems to me to be quite distinct from the sort of compensation that is meant to be accounted for in these studies (i.e. monetary compensation from the employing organization). In other words, the case that public employees are more satisfied with their jobs than "comparable" private employees seems distinct from the case that public employees are better compensated.
Trevor, if they are working out of altruism, that is a compensating differential and means the public can afford to pay them less without much negative repercussion.
I realize this is a stat's blog, but sometimes it makes sense to go with theory over empirics.
What would it mean if they were not compensated at least as well as they could in the private sector? The union is providing them no or negative value or the government drives such a hard bargain that even the power of the public union can't make up the difference. I find both very unlikely.
OTOH, if you are going to do the study I think I would have used medians rather than means.
Quit rates are tricky, as well.
First, the deicison to quit (as opposed to being declared redundant) may have implications on Unemployment Insurance. So the "quit rate" may measure the ability of employees to find a job they prefer. So there may be considerable incentive on the part of employees to not quit but rather be "laid off".
Second, in a state educational system, all of the teaching jobs (or most of them, anyway) may be supplied by the state. So few people would quit for an equivalent position at a competing firm.
Finally, one might consider the way that benefits are paid out. Some pension plans may penalize quitting (i.e. by only vesting after a number of years) whereas 401k plans may be very portable. These factors could create huge incentives to stay put that are hard to capture in the crude rate.
It's a simple metric and possibly a useful metric, but there really isn't a magic bullet here.
Quit rates would still have to be interpreted, so I don't see how they are the most sensible.
If public sector quit rates are lower than in the private sector, does it mean that public sector employees are overpaid ? Maybe they are all just Milton's from Office Space and incapable of getting another job, maybe many public sector jobs require skills sets specific to the public sector that make transferring to a private sector career more difficult.
If public sector quit rates are higher is it because the public sector employees are underpaid ? Maybe working in the giant govt. bureaucracy is less satisfying.
No matter what the data shows about quit rates, it will be spun.
Talk about a dog chasing his own tail: maybe the quit rate is directly related to the health insurance bargaining power of the state compared to that of any company.
Also, re: quit rates…most often people take public employment because they're looking for better benefits in exchange for a lower pay rate; that is, they're trading earning potential for security. This alone would produce lower quit rates.
"I agree that it makes sense to look at quit rates but that won't tell the whole story either. Nothing will. There's no magic number here."
I would agree if the magnitude of the difference were 5 or 10% but its a factor of 3 which is pretty staggering. Obviously I could be missing something but barring an explanation that doesn't involve Milton from Office Space I find Landsburg pretty convincing.
All I know is that if I lived in Wisconsin, I'd be very pissed off that my tax dollars are contributing to state employees viagra needs. That is clearly NOT a necessity and its certainly not my responsibility to pay for anyone's sex life. My God. We waste so much money in this nation that its pathetic. Also, I have zero sympathy for these state employees with cushy jobs and all. Maybe its time to start outsourcing their jobs to India and China for them to realize how fragile the job market is these days. I am sure we can find some people in India to do their desk jobs for 65% less than Wisconsin tax payers pay them.