Michael Wiebe writes:
China has had double-digit economic growth for nearly three decades. How can we explain this? In my dissertation, I [Wiebe] studied one explanation that is backed up by a large literature: meritocratic promotion. The idea is that politicians compete in promotion tournaments, where the politician with the highest GDP growth rate in their jurisdiction is rewarded by being promoted. By tying promotion to economic growth, meritocratic promotion creates strong incentives to boost GDP, and hence helps explain China’s rapid growth.
When I collected data on prefecture politicians, however, I found no evidence for meritocracy: there was no correlation between GDP growth and promotion, despite trying many different models. How is this null result consistent with the positive findings in the rest of the literature? To find out, I replicated the main papers claiming evidence for prefecture-level meritocracy. Short answer: the literature is wrong.
This post summarizes my replications. I find that the results in the literature are not robust to reasonable specification changes, or are due to data errors. You can find the full details, and a few more replications, in the paper here. . . .
I have not read the paper so I cannot comment one way or another on Wiebe’s conclusions. I’ll just say that I like the idea of looking at data carefully and understanding the links between data, statistical model, and substantive theory. If Wiebe is doing what he says he’s doing, then this is a great example of the sort of social science workflow we should be aiming for.
It appears to be part of a dissertation, so the fact that data is not provided may be reasonable. At least he discusses a number of data errors made in prior work. However, there are very very many p values – and very little in the way of graphs of the data. It leaves me wanting to see something more descriptive than ‘this was miscoded and when the coding is corrected the p value goes from x to y.’
What’s the conclusion? What’s the reason(s) given in the paper that’s responsible for China’s consistent and large economic growth? (I don’t think it’s reasonable to read a 60 page paper to find the answer.)
Here’s my conclusion from Ch. 1:
> The original motivation for the meritocracy literature was to provide an explanation for China’s incredible economic growth. But is meritocracy necessary for understanding how China could sustain double-digit growth for three decades? After all, China had favorable conditions for growth: high levels of human capital, high state capacity, and political stability, to name a few. Perhaps standard growth theory gives a sufficient explanation, without needing to appeal to the incentives of politicians. Alternatively, it is possible that politicians were indeed incentivized to boost growth, but for the purpose of raising tax revenues rather than winning a promotion tournament. As argued by Su et al. (2012), local governments targeted economic growth in order to make up for the revenue shortfall caused by the 1994 tax reform, which shifted revenues to the central government. I am not taking a stand on whether these alternative explanations are correct. Instead, I merely want to demonstrate that, as economists, our explanations for China’s growth remain strong even if we discard the meritocracy hypothesis.
https://michaelwiebe.com/assets/ch1.pdf
I also have a short follow-up post on county-level meritocratic promotion, where county leaders are incentives to boost GDP growth.
https://michaelwiebe.com/blog/2021/02/meritocracy
Thanks.
Michael, I think one statistical challenge for running this type of regression is that, if one “capable leader” did receive many promotions, they would also have been placed into many cities, and thus their average GDP growth would be close to the national mean. This is some sort of selection bias too.
Another related issue is “regression to the mean”. The GDP growth is random at its best: It is not clear if a “capable leader” is rewarded based on the merit, or a random realization of their merit. Maybe a measurement error model can help separate these two factors.
This is actually the basis of Yao and Zhang (2015)’s approach, which uses AKM to estimate leader effects: if leaders move across cities, you can separate the leader fixed effect from the city fixed effect (and from the year effect). You’re suggesting that leader effects are too small to detect; it would’ve been nice to see a power calculation in the original paper.
By the way, your introduction reads: “I conclude by proposing a model of meritocracy where county-level promotion tournaments provide a causal explanation for China’s economic growth…” but it appears that the model is only using regression to explain promotion by GDP growth. It is not a causal explanation for GDP growth.
Also, remember that merit-based promotion is not the same as meritocracy: the problem’s with the “ocracy”.
The idea is that county-level meritocratic promotion could affect GDP growth (contrasting with no meritocratic promotion at *prefecture/province* levels). But you’re right, we don’t have strong evidence for that, just a correlation.
You can find the conclusion at pp 15-16. He doesn’t claim to have an explanation.
One point:
In my experience, the problem of binary variables and the restriction, therefore, of using a probit model is common. One way to expand the model is using latent variables:
https://www.statmodel.com/download/Muthen1979.pdf
The paper is a step-by-step guide of how to incorporate latent variables in a probit model.
Note: Regarding the conclusion, it’s opaque and I don’t have the time to read the other 14 pages to understand it.
“The best-published papers studying meritocracy at the prefecture level do not provide robust evidence that prefecture leaders are promoted based on their performance in growing GDP. My null result in Chapter 1 is not contradicted by the literature, since the results in the literature are not robust to reasonable specification changes. Overall, I conclude that meritocratic promotion, at least at the prefecture level, does not explain China’s rapid economic growth.”
The literature suggested that China had somehow managed to create bureaucracies that were not primarily self-serving, quite an astonishing claim which unsurprisingly turned out to be false. The unwritten conclusion: modern Chinese bureaucracy is indistinguishable from other bureaucracies throughout history, at least as far as meritocratic promotion is concerned.
I only have a comment about workflows in general, nothing specific to the scientific work done here. Since the analysis seem to be based on previously published work, I would have expected to see fully reproducible code and data, so the original authors can at least check where things went wrong, or so that others can critique the present analysis.
It seems that the prior probability of reproducing an analysis is roughly 30%, and I assume that this would include the present work. This depressing fact kind of makes it imperative to release reproducible code and data.
We recently wrote a paper on this important issue (area: psycholinguistics, not economics): https://psyarxiv.com/hf297/
Good point, I should do that. I got busy with job searching.
Part of the problem is the lack of incentives to publish code and data. I suppose comments like yours are the incentive!
You can give your code and data a DOI and license on osf.io, and provide citation information. Then any future user will be able to cite and reuse your code, which will give about as much credit as one gets with a paper. It’s true that one can’t increase one’s h-index with code citations, although who knows? With google scholar automatically tracking citations, maybe code-as-scientific-output will also start appearing under the google scholar counts. Someone should send google a feature request.
In my experience, releasing code and data has some intangible rewards. Other people use one’s code and data, which advances knowledge.
For me it’s also personally embarrassing if others can’t reproduce my analyses (although I have to admit that it has happened more than once to me that I updated Xcode and my Rmd file stopping working from one minute to the next–had to reinstall Stan and brms, LOL. I can’t even reproduce my own code sometimes).