Last month we discussed the controversy regarding the recent revelation that, back in 2015, economist Alan Krueger had been paid $100,000 by Uber to coauthor a research article that was published at the NBER (National Bureau of Economic Research, a sort of club of influential academic economists) and seems possibly to have been influential in policy.

Krueger was a prominent political liberal—he worked in the Clinton and Obama administrations and was most famous for research that reported positive effects of the minimum wage—so there was some dissonance about him working for a company that has a reputation for employing drivers as contractors to avoid compensating them more. But I think the key point of the controversy was the idea that academia (or, more specifically, academic economics (or, more specifically, the NBER)) was being used to launder this conflict of interest. The concern was that Uber paid Krueger, the payment went into a small note in the paper, and then the paper was mainlined into the academic literature and the news media. From that perspective, Uber wasn’t just paying for Krueger’s work or even for his reputation; they were also paying for access to NBER and the economics literature.

In this new post I’d like to address a few issues:

1. Why now?

The controversial article by Hall and Krueger came out in 2015, and right there it had the statement, “Jonathan Hall was an employee and shareholder of Uber Technologies before, during, and after the writing of this paper. Krueger acknowledges working as a consultant for Uber in December 2014 and January 2015 when the initial draft of this paper was written,” is not “an adequate disclosure that Uber paid $100,000 for Krueger to write this paper.”

So why the controversy now? Why are we getting comments like this now:

rather than in 2015?

The new information is that Krueger was paid $100,000 by Uber. But we already knew he was being paid by Uber—it’s right on that paper. Sure, $100,000 is a lot, but what were people expecting? If Uber was paying Krueger at all, you can be pretty sure it was more than $10,000.

I continue to think that the reason the controversy happened now rather than in 2015 is that, back in 2015, there was a general consensus among economists that Uber was cool. They weren’t considered to be the bad guy, so it was fine for him to get paid.

2. The Necker cube of conflict of interest

Conflict of interest goes in two directions. On one hand, as discussed above, there’s a concern that being paid will warp your perspective. And, even beyond this sort of bias, being paid will definitely affect what you do. Hall and Krueger could be the most ethical researchers in the universe—but there’s no way they would’ve done that particular study had Uber not paid them to do it. This is the usual way that we think about conflict of interest.

But, from the other direction, there’s also the responsibility, if someone pays you do to a job, to try to do your best. If Hall and Krueger were to take hundreds of thousands of dollars from Uber and then turn around and stab Uber in the back, that wouldn’t be cool. I’m not talking here about whistleblowing. If Hall and Krueger started working for Uber and then saw new things they hadn’t seen before, indicating illegal or immoral conduct by the company, then, sure, we can all agree that it would be appropriate for them to blow the whistle. What I’m saying is that it, if Hall and Krueger come to work for Uber, and Uber is what they were expecting when they took the job, then it’s their job to do right by Uber.

My point here is that these two issues of conflict of interest go in opposite directions. As with that Necker cube, it’s hard to find a stable intermediate position. Is Krueger an impartial academic who just happened to take some free money? Or does he have a duty to the company that hired him? I think it kind of oscillates between the two. And I wonder if that’s one reason for the strong reaction that people had, that they’re seeing the Necker cube switch back and forth.

3. A spectrum of conflicts of interests

Another way to think about this is to consider a range of actions in response to taking someone’s money.

– At one extreme are the pure hacks, the public relations officers who will repeat any lie that the boss tells them, the Mob lawyers who help in the planning of crimes, the expert witnesses who will come to whatever conclusion they’re paid to reach. Sometimes the hack factor overlaps with ideology, so you might get a statistician taking cash from a cigarette company and convincing himself that cigarettes aren’t really addictive, or a lawyer helping implement an illegal business plan and convincing himself that the underlying law is unjust.

– Somewhat more moderate are the employees or consultants who do things that might otherwise make them queasy, because they benefit the company that’s paying them. It’s hard to draw a sharp line here. “Writing a paper for Uber” is, presumably, not something that Krueger would’ve done without getting paid by Uber, but that doesn’t mean that the contents of the paper cross any ethical line.

– Complete neutrality. It’s hard for me to be completely neutral on a consulting project—people are paying me and I want to give them their money’s worth—but I can be neutral when evaluating a report. Someone pays me to read a report and I can give my comments without trying to shade them to make the client happy. Similarly, when I’ve been an expert witness, I’ve just written it like I see it. I understand that this will help the client—if they don’t like my report, they can just bury it!—so I’m not saying that I’m being neutral in my actions. But the content of my work is neutral.

– Bending over backward. An example here might be the admirable work of Columbia math professor Michael Thaddeus in criticizing the university’s apparently bogus numbers. Rather than going easy on his employer, Thaddeus is holding Columbia to a high standard. I guess it’s a tough call, when to do this. Hall and Krueger could’ve taken Uber’s money and then turned around and said something like, This project is kinda ok, but given the $ involved, we should really be careful and point out the red flags in the interpretation of the results. That would’ve been fine; I’ll just say again that this conflicts with the general attitude that, if someone pays you, you should give them your money’s worth. The Michael Thaddeus situation is different because Columbia pays him to teach and do research; it was never his job to pump up its U.S. News ranking.

– The bust-out operation. Remember this from Goodfellas? The mob guys take over someone’s business and strip it of all its assets, then torch the place for the insurance. This is the opposite of the hack who will do anything for the boss. An example here would be something like taking $100,000 from Uber and then using the knowledge gained from the inside to help a competitor.

In a horseshoe-like situation, the two extremes of the above spectrum seem the least ethical.

4. NBER

This is just a minor thing, but I was corresponding with someone who pointed out that NBER working papers are not equivalent to peer-reviewed publications.

I agree, but NBER ain’t nothing. It’s an exclusive club, and it’s my impression that an NBER paper gets attention in a way that an Arxiv paper or a Columbia University working paper or an Uber working paper would not. NBER papers may not count for academic CV’s, but I think they are noticed by journalists, researchers, and even policymakers. An NBER paper is considered legit by default, and that does have to do with one of the authors being a member of the club.

5. Feedback

In an email discussion following up on my post, economics journalist Felix Salmon wrote:

My main point the whole time has just been that if Uber pays for a paper, Uber should publish the paper. I don’t like the thing where they add Krueger as a second author, pay him $100k for his troubles, and thusly get their paper into NBER. Really what I’m worried about here is not that Krueger is being paid to come to a certain conclusion (the conflict-of-interest problem), so much as Krueger is being paid to get the paper into venues that would otherwise be inaccessible to Uber. I do think that’s problematic, and I don’t think it’s a problem that lends itself to solution via disclosure. On the other hand, it’s a problem with a simple solution, which is just that NBER shouldn’t publish such papers, and that they should live the way that god intended, which is as white papers (or even just blog posts) published by the company in question.

As for the idea that conflict is binary and “the conflict is there, whether it’s $1000 or $100,000” — I think that’s a little naive. At some point ($1 million? $10 million? more?) the sheer quantity of money becomes germane. . . .

Salmon also asked:

Is there a way of fixing the disclosure system to encompass this issue? It seems pretty clear to me that when the lead author is literally an Uber employee, Uber has de facto control over whether the paper gets published. Again, I think the solution to this problem is to have Uber publish the paper. But if there’s a disclosure solution then I’m interested in what it would look like.

I replied:

You say that Uber should just publish the paper. That’s fine with me. I will put a paper on my website (that’s kinda like a university working paper series, just for me), I will also put it on Arxiv and NBER (if I happen to have a coauthor who’s a member of that club), and I’ll publish in APSR or JASA or some lower-ranking journal. No reason a paper can’t be published in more than one “working paper” series. I also think it’s ok for the authors to publish in NBER (with a disclosure, which that paper had) and in a scholarly journal (again, with a disclosure).

You might be right that the published disclosure wasn’t enough, but in that case I think your problem is with academic standards in general, not with Krueger or even with the economics profession. For example, I was the second author on this paper: where the first and last authors worked at Novartis. I was paid by Novartis while working on the project—I consulted for them for several years, with most of the money going to my research collaborators at Columbia and elsewhere, but I pocketed some $ too. I did not disclose that conflict of interest in that paper—I just didn’t think about it! Or maybe it seemed unnecessary given that two of the authors were Novartis employees. In retrospect, yeah, I think I should’ve disclosed. But the point is, even if I had disclosed, I wouldn’t have given the dollar value, not because it’s a big secret (actually, I don’t remember the total, as it was over several years, and most of it went to others) but just because I’ve never seen anyone disclose the dollar amount in an acknowledgements or conflict-of-interest statement for a paper. It’s just not how things are done. Also I don’t think we shared our data. And our paper won an award! On the plus side, it’s only been cited 17 times so I guess there’s some justice in the world and vice is not always rewarded.

I agree that conflict is not binary. But I think that even with $1000, there is conflict. One reason I say this is that I think people often have very vague senses of conflict of interest. Is it conflict of interest to referee a paper written by a personal friend? If the friend is not a family member and there’s no shared employment or $ changing hands, I’d say no. That’s just my take, also my approximation to what I think the rules are, based on what Columbia University tells me. Again, though, I think it would’ve been really unusual had that paper had a statement saying that Krueger had been paid $100,000. I’ve just never seen that sort of thing in an academic paper.

Did Uber have the right to review the paper? I have no idea; that’s an interesting question. But I think there’s a conflict of interest, whether or not Uber had the right to review: these are Uber consultants and Uber employees working with Uber data, so no surprise they’ll come to an Uber-favorable conclusion. I still think the existing conflict of interest statement was just fine, and that the solution is for all journalists to report this as, “Uber study claims . . . ” The first author of the paper was an Uber employee and the second author was an Uber consultant—it’s right there on the first page of the paper!

As a separate issue, if the data and meta-data (the data-collection protocol) are not made available, then that should cause an immediate decline of trust, even setting aside any conflict of interest. I guess that would be a problem with my above-linked Novartis paper too; the difference is that our punch line was a methods conclusion, not a substantive conclusion, and you can check the methods conclusion with fake data.

Salmon replied:

If you want the media to put “Uber study claims” before reporting such results, then the way you get there is by putting the Uber logo on the top of the study. You don’t get there by adding a disclosure footnote and expecting front-line workers in the content mines to put two and two together.

As discussed in my original post on the topic, I have lots of conflicts of interest myself. So I’m not claiming to be writing about all this from some sort of ethically superior position.

6. Should we trust that published paper?

The economist who first pointed me to this story (and who wished to remain anonymous) followed up by email:

I’m surprised that the focus of your thoughts and of the others who commented on the blog were on whether Krueger acted correctly, not on what to do with the results they (and others) found. I guess my one big takeaway from now on is to move my priors more strongly towards being sceptical of papers using non-shared, proprietary data…

That’s a good point! I don’t think anyone in all these discussions is suggesting we should actually trust that published paper. The data were supplied by Uber; the conclusions were restricted to what Uber wanted to get; the whole thing was done in an environment where economists loved Uber and wanted to hear good things about it; etc.

I guess the reason the conversation was all about Krueger and the role of academic economics was that everyone was already starting from the position that the paper could not be trusted. If it were considered to be a trustworthy piece of research, I think there’d be a lot less objection to it. It’s not like the defenders of Krueger were saying they believed the claims in the paper. Again, part of that is political. Krueger was a prominent liberal economist, and now in 2022, liberal economists are not such Uber fans, so it makes sense that they’d defend his actions on procedural rather than substantive grounds.