Matthew Yglesias links approvingly to the following statement by Michael Mandel:
Homeland Security accounts for roughly 90% of the increase in federal regulatory employment over the past ten years.
Roughly 90%, huh? That sounds pretty impressive. But wait a minute . . . what if total federal regulatory employment had increased a bit less. Then Homeland Security could’ve accounted for 105% of the increase, or 500% of the increase, or whatever. The point is the change in total employment is the sum of a bunch of pluses and minuses. It happens that, if you don’t count Homeland Security, the total hasn’t changed much–I’m assuming Mandel’s numbers are correct here–and that could be interesting.
The “roughly 90%” figure is misleading because, when written as a percent of the total increase, it’s natural to quickly envision it as a percentage that is bounded by 100%. There is a total increase in regulatory employment that the individual agencies sum to, but some margins are positive and some are negative. If the total happens to be near zero, then the individual pieces can appear to be large fractions of the total, even possibly over 100%.
I’m not saying that Mandel made any mistakes, just that, in general, ratios can be tricky when the denominator is the sum of positive and negative parts. In this particular case, the margins were large but not quite over 100%, which somehow gives the comparison more punch than it deserves, I think.
We discussed a mathematically identical case a few years ago involving the 2008 Democratic primary election campaign.
What should we call this?
There should be a name for this sort of statistical slip-up. The Fallacy of the Misplaced Denominator, perhaps? The funny thing is that the denominator has to be small (so that the numerator seems like a lot, “90%” or whatever) but not too small (because if the ratio is over 100%, the jig is up).
P.S. Mandel replies that, yes, he agrees with me in general about the problems of ratios where the denominator is a sum of positive and negative components, but that in this particular case, “all the major components of regulatory employment change are either positive or a very tiny negative.” So it sounds like I was choosing a bad example to make my point!
It sounds like the 90% isn't accurate. You may not go as far as saying "Mandel made a mistake," but I would. "90% of the increase" means that the denominator should only include the increase, not the decrease.
So, if there was a net increase of 10 jobs, 9 from Homeland Security, but the net increase in jobs is adding 90 jobs and losing 80, then HLS is only 9/90, or 10% of the "increase."
I'd call it "Fallacy of the Meaningless Net Denominator" since "net" implies it includes positive and negative denominators.
I would call your post missing the forest for the trees.
You can reword the statement the following: Only 10% of the increase in federal regulatory employment came from sources other than Homeland Security.
The article begins with the sentence "As Republicans go on the attack about excessive regulation under the Obama Administration, it’s worth noting two things."
If his a proper understanding of the numbers suggests that federal regulatory employment didn't increase significantly than there's no new excessive regulation and the author of the post won his argument.
Seems like a better way to state this is, "Without the increase from Homeland Security, the total increase in federal regulatory employment over the past ten years would have been 90% lower."
Another reason why it is ridiculous is if he had choosen a department that had decreased. "Environmental regulation contributed -5% to the increase…"
I see this type of thing a lot. The notion of a factor "explaining" growth leads pretty quickly to odd results. For example, a factor that "explains" 93% of the growth yet is actually the 2nd or 3rd largest-growing factor (with the others exceeding 100%, offset by big declines in other factors). Also if the overall growth is small, then the percentage "explained" by a factor can be huge, for example 795%.. Finally, a factor that declines substantially might "explain" more than -100% of the growth.
I'd call it the "I'm Brian and so's my wife" phenomenon, since you can have departments A & B each increasing by x, department C decreasing by x, so that department A accounts for 100% of the overall increase – and so does department B.
Good point. Your observation is elementary and obvious in hindsight but it didn't occur to me.
How would you rephrase the statement?
Hi
You are barking up the wrong tree here, I think. I'm quite familiar with the problem you mention. In fact, I regularly attack the phrase "consumer spending is 70% of economic activity" as misleading precisely because there's a big negative in GDP, imports. (You should take that one on–it's far too pervasive, and has real consequences).
In this case, however, all the major components of regulatory employment change are either positive or a very tiny negative (take a look at the chart in the post in question). Therefore it's appropriate to talk about homeland security's growth as a share of the total change.
You see similar misleading language in the business press, in several forms, all of which involve a concealed or shifting denominator. Puff piece: XYZ Corp’s profits rose 300% last year! Well, actually they’d just barely broken even the year before, so in absolute terms their profit rose from $3 to $12. The change isn’t remotely material, but if you’re writing for XYZ’s public relations department, it sure sounds good.
In general we need a way to distinguish percentage points from percentages of another value. If your net profit was 10% last year, and it increases by twenty percent, what’s your new profit rate, 12% or 30%? In the sloppy formulations of many business articles, there’s no way to know. It ought to be clear—percent versus percentage points—but readers simply can’t rely on that today. Sadly, I think a lot of journalists don’t understand the distinction, much less make it clear in their articles.
A distinct problem, nonetheless related, is the tendency to choose between absolute values and percentages according to which sounds more dramatic. So the same change in unemployment can be described as a 0.01% shift, if you want to minimize it, or as ‘sixteen thousand workers’ if you want to dramatize it. Worst of all, many articles will mix the two, possibly just to vary the language a bit, more likely to advance an agenda, but in either case to the detriment of understanding.
Mike:
Good point!
This sounds like a version of the Lake Wobegone Conundrum:
Suppose we have a budget comprised of seven departments, each of the same number of employees. Further suppose that in the next budget three of the departments lost 10 employees each and four added 10 employees each, for a net organizational increase of 10 employees. In this case, using the Mandel approach, four departments could say that they account for 100% of the organizations total employment increase.
As you suggest, Mandel's "90%" finding may not be strictly wrong, but that approach is certainly susceptible to misleading and inaccurate interpretation.