Someone who would prefer to remain anonymous writes:
Before anything else and like many of the people who write to you, as an early career academic in the area of economics, I feel constrained in the criticisms that I can make publicly. So I have to kindly request that, if you do publish any of what follows, that identifying information about myself be not made public.
My correspondent continues:
The recent disclosed leaks at Uber revealed some, well, distressing behaviour by some of my peers. As the Guardian recently wrote (https://www.theguardian.com/news/2022/jul/12/uber-paid-academics-six-figure-sums-for-research-to-feed-to-the-media), several noted economists collaborated with Uber when writing some academic papers. In short, Uber shared data with a selected group of economists, paid them and had Uber economists collaborating with the authors.
The act of collaboration is not, in itself, necessarily a bad thing, as the potential access to proprietary data allows research to be done that otherwise would not be possible. But the way things were done raises several issues, many of which you have commented on before. I’d like to focus on one in particular: how do we deal with studies done with closed data shared by interested parties?
In the leaked emails, Uber staffers wrote that, concerning one economist who wanted to do a separate unpaid study using Uber’s shared data: “We see low risk here because we can work with Landier on framing the study and we also decide what data we share with him.” I.e., the issue here isn’t just of replication, already a serious concern, but the risk that a company may omit data so as to influence academics doing a study, so as to frame things in the best light possible. It is distressing to read that executives wanted to work on report’s message to “to ensure it’s not presented in a potentially negative light”.
Perhaps I’m being a bit too naive about all this, in that the obvious question when seeing a study like this is to ask why would Uber be ready to collaborate unless they are going to get what they wanted? Indeed, I recall being a little bit sceptical about Hall and Krueger’s initial NBER paper when I read it. But the excerpts produced by The Guardian are so much worse than I’d have feared, even if we don’t know the exact extent that Uber acted as these excerpts describe, it’s hard not to fear the worst. Where does that leave us with these studies? Should we dismiss them altogether or can we salvage some their analyses?
There is one small bright spot in all of this. Because I am not a Labour economist, I only ever read Hall and Krueger’s initial NBER paper, so missed Berg and Johnston’s later critique (https://journals.sagepub.com/doi/full/10.1177/0019793918798593?journalCode=ilra), where the issue of inadequate data was already being raised, among others even stronger criticisms. So even before the emails, there were some people tackling these issues head on. And, to the credit of ILR Review, the journal that published Hall and Krueger’s paper, the critique was published by themselves, unlike what happens so often.
Anyway, that’s all. I guess I’m shocked at how much worse things seem to be, at how willing Uber was to manipulate and try to use well regarded academic’s reputations to, it seems, launder their own reputation…
Some googling revealed this exchange between financial journalist Felix Salmon and economists Michael Strain and Justin Wolfers:
This followup wins the thread:
Lots to chew on here, so let me go through some issues one at a time:
1. Conflicts of interest
It’s easy to get all “moralistic” about this. It’s also easy to get all “It’s easy to get all ‘moralistic’ about this” about this.
So let’s be clear: the conflict of interest here is real; indeed, it’s hard to get much more “conflict of interest” than “Company pays you $100,000 to write a paper, then you write a paper favorable to that company.” At the same time, there’s nothing necessarily morally wrong about having a conflict of interest. It is what it is. Every year I fill out my conflict of interest form with Columbia. “Conflict of interest” is a description, not a pejorative.
With regard to the Hall and Krueger paper, the dispute was no whether there was a conflict of interest but rather (a) whether the conflict was sufficiently disclosed, and (b) how this conflict should affect the trust that policymakers would hold in its conclusions.
I don’t have a strong feeling about the disclosure issue—Salmon holds that the statement, “Jonathan Hall was an employee and shareholder of Uber Technologies before, during, and after the writing of this paper. Krueger acknowledges working as a consultant for Uber in December 2014 and January 2015 when the initial draft of this paper was written,” is not “an adequate disclosure that Uber paid $100,000 for Krueger to write this paper.” I dunno. I’ve written lots of acknowledgments and I don’t recall ever mentioning the dollar value. It seems to me pretty clear that if you have one author who worked at the company and another author who was paid by the company, the conflict is there, whether it’s $1000 or $100,000.
Regarding trust: yeah, with this level of conflict, you’d want to see some data analyses by an outside team, like with that Google chip-layout paper.
News reports should be more clear on this. A headline, “Ride-hailing apps have created jobs for Paris’s poorer youth, but a regulatory clampdown looms,” should be replaced by something like “Uber-paid study reports that ride-hailing apps have created jobs for Paris’s poorer youth, but a regulatory clampdown looms.” Just add “Uber-paid study” to the beginning of every headline!
I don’t get this reaction:
I mean, sure, I don’t like Uber either. Some people think a company like Uber is cool, some people think it’s creepy. Views differ. But “sell their souls . . . destroy their lives . . . especially distressing”? That seems a bit strong. Consider: back in 2015, economists absolutely loved Uber, which is no surprise given that economists loved to talk about the problems with the market for taxicabs, the famous medallion system, etc. Economists hated taxi regulation about as much as they hated rent control and other blatant restrictions on free enterprise. Economists on the center-left, economists on the center-right, they all hated those regulations, so it’s no surprise that they loved Uber. The company was an economist’s dream, along with being super convenient for users.
3. Interesting data
I get Wolfers’s point that Krueger would find the Uber data interesting. I would have too! Indeed, had Uber offered me $100,000, or even $50,000, I probably would’ve worked for them too. I can’t be sure—they never approached me, and it’s possible that I would’ve said no—, but, if I had said no, it wouldn’t have been because their data were not sufficiently interesting. The point Wolfers seems to be missing here is that God is in every leaf of every tree. Yes, Uber data are interesting, but lots of other data are interesting too. Ford’s data are interesting, GM’s data are interesting, Bureau of Labor Statistics data are interesting. Lots of interesting data out there, and often people will choose what to look at based on who is paying them. I think one missing piece in the public discussions of this case is how much economists looooved Uber back then: it was a symbol of all that was good about the free market! So they found these data to be particularly interesting.
4. What can you get for $100,000?
A funny thing about the discussion is how little an amount of money $100,000 seems to be to commenters, and that includes people on both sides of the issue! Wolfers thinks that $100,000 is so small that it is “extremely unlikely” that Krueger would write a paper for that paltry sum. From the other direction, Dubal thinks it’s “pathetic” that he would “violate basic rules of research for just 100,000.”
I only met Krueger once, so I can’t speak to his motivations, but I will say that being the second author on a paper can sometimes be pretty easy, and $100,000 is real money! For example, suppose Krueger’s consulting rate was $2000/hour. He should be able to do the work required to be second author on a paper in less than 50 hours. The disclosure in that article says he was working as a consultant during the 2-month period when the initial draft of the paper was written. Spending 50 hours on a project during a 2-month period, that’s plausible. So I can’t really see why Wolfers thinks this is “extremely unlikely.”
There is an alternative story, though, consistent with what Wolfers hypothesizes, which is that Krueger would’ve coauthored the paper anyway but took the $100,000 because it was offered to him, and who wouldn’t take free money? I’m willing to believe that story, actually. This also works as a motivation of Uber: they offered free money to Krueger for something he would’ve done anyway, just to give him an excuse to clear his schedule to do the work. So, he didn’t coauthor a paper for $100,000; he coauthored a paper for free and then accepted the $ to motivate himself to do it. Meanwhile, from Uber’s perspective, the money is well spent, in the same way that the National Science Foundation is motivated to pay me to free up my time to do research that’s they think will be valuable to society.
Regarding Dubal’s comment: I don’t see what “basic rules of research” were being violated? Not sharing your data? If working with private data is violating a basic rule of research, fine, but then scientists are doing that for free every day. If you set your time machine back to 2015, and you consider Krueger as an economist who thinks that Uber is cool, then getting paid by them and working with their data, that’s even cooler, right? I imagine that for an economist in 2015, working with Uber is as cool as working with a pro sports team would be for a statistician. Getting paid makes it even cooler: then you’re an insider! Sure, Krueger was a big-name academic, he’d served at the highest levels of government, and according to Wolfers he was doing well enough that $100,000 meant nothing to him. Still, working with Uber, as an economist I’ll bet he thought that would be something special. Again, Uber in 2015 had a different aura than Uber today.
5. “Laundering” and the role of academia and the news media in a world that’s mostly run by business and government
What exactly was it about the Hall and Krueger paper that bothered people so much? I don’t think it was simply that these guys were working for Uber or that, given that Uber was paying them, they’d write a report with a pro-Uber spin. I think what is really bugging the critics is the sense that academia—in this case, then NBER (National Bureau of Economic Research), an influential club or organization of academic economists—was being used to launder this money.
If Hall and Krueger were to publish a book, The Case for Uber, published by Uber Press, arguing that Uber is great, then it’s hard to see the problem. These guys chose to take the job and they did it. But when published as an NBER preprint, and one of the authors is a respected academic, it seems different—even with the disclosure statement.
Again, it’s a problem with the news media too, to the extent that they reported this study in the way they’d report straight-up academic research, without the “Uber-funded study claims . . .” preamble to every headline and sentence describing the published findings.
This all kinda makes me think of another well-known economist, John Kenneth Galbraith, who wrote about “countervailing power.” Galbraith was talking about economic and political power, but something similar arises in the world of ideas. Government and business are the 800-pound gorillas, and we often like to think of academia and advocacy organizations as representing countervailing power. When industry or government inserts propaganda into academic channels, this is bothersome in the same way that “astroturf” lobbying seems wrong. It’s bad for its own sake—fooling people and potentially affecting policy through disinformation—and also bad in that it degrades the credibility of un-conflicted scientific research or genuine grassroots organizing.
In saying this, I recognize that there’s no pure stance in science just as there’s no such thing as pure grassroots politics: Scientists have prior beliefs and political attitudes, and they also have to be paid from somewhere, and the same goes for grassroots organizers. Those mimeograph machines don’t run themselves. So there’s a continuous range here. But getting $100,000 for two months of work to coauthor a paper, that’s near the extreme end of the range.
What I’m getting at here is that, while there is indignation aimed at Krueger here, I think what’s really going on is indignation at perceived manipulation of the system. One way to see this is that nobody seems particularly angry at the Uber executives or even at Hall, the first author of that paper. If it’s bad science, you should be mad at the people who promoted it and the person who did the work, no? I think there’s this attitude that the full-time Uber employees were just doing their jobs, whereas Krueger, who was just a consultant, was supposed to have had a higher loyalty to academia.
There’s one other thing I wanted to get to, which was Wolfers’s attitude that Krueger needed to be defended. (Again, nobody seemed to feel the need to defend Hall for his role in the project.)
One part of the defense was the silly claim that he wouldn’t have done it for the money, but I think underlying there were two implicit defenses:
First, conflict of interest sounds like a bad thing, Krueger was a good person, and therefore he couldn’t’ve had a conflict of interest. I don’t think this argument makes sense—I see conflict of interest not as an aspect of character but as an aspect of the situation. When I write about Columbia University or any organization that is paying me or my family, I have a conflict of interest. I can still try to be objective, but even if I have pure objectivity, the conflict of interest is still there. It’s inherent in the interaction, not in me.
Second, Krueger is a political liberal so therefore he couldn’t have issued a report unduly favorable to Uber, because liberals are skeptical of corporations. I don’t think this argument works either, first because, as noted above, back in 2015 economists of a wide range of political stripes considered Uber to be awesome, and second because Krueger, while political, was not known as a hack. He works with Uber, they tell him good things about the company, he coauthors a positive report.
I always wondered if something similar was going on when the economist James Heckman hypes early childhood intervention programs. Heckman is a political conservative, and one would expect him to be skeptical of utopian social spending programs. So when he and his collaborators found (or, to be precise, thought they found) strong evidence of huge effects of these programs, it was natural for him to think that his new stance was correct—after all, he came to it despite his political convictions.
But it doesn’t work that way. Yes, you can be biased to come out with a result that confirms your preconceptions. But when you come out with a result that rocks your world, that could be a mistake too.
7. Who to credit or blame
I agree with my correspondent, who focused the blame (or, depending on your perspective, the credit) for this episode on Uber management. The online discussion seemed to be all about the economist who consulted for Uber and was the second author of the paper, but really it seems that we should think of Uber, the organization, as the leader of this endeavor.
Full disclosure: I’ve been paid real money by lots of organizations that have done bad things, including pharmaceutical companies, tech companies, and the U.S. Department of Defense.
P.S. Interesting comment here from economist Peter Dorman.
P.P.S. More here.