Albert-Laszlo Barabasi is underpaid. By a lot!

David Sholl writes:

I thought your readers might be interested in this excerpt from the relatively new book in the Malcolm Gladwell tradition by Albert-Laszlo Barabasi, The Formula: The Universal Laws of Success:

It’s possible to put actual monetary value on each citation a paper receives. We can, in other words calculate exactly how much a single citation is worth. Any guesses? Shockingly, in the United States each citation is worth a whopping $100,000. We know this by looking at the amount of money the nation spends on research . . . If we then divide that figure by the total number of citations collectively generated by the papers these funds paid for, we can estimate the cost of a single citation.

A very conservative estimate for my US academic department [the School of Chemical & Biomolecular Engineering at Georgia Tech] is that my colleagues receive more than 20,000 citations a year. Using Barabasi’s estimate is means that our department’s research is worth $2 billion per year. My colleagues do great work, but that estimate seems nonsensical, to put it charitably.

The author of that book has come up on this blog before, even with a Gladwell comparison! He has interesting ideas but seems pretty sloppy when it comes to numbers. Or maybe I should say, he’s pretty sloppy with his numbers but he seems to have interesting ideas. I’m a statistician so of course I focus on the numbers.

I looked him up on Google Scholar. He has 250,000 citations—that’s twice as many as me!. 250,000 citations multiplied by $100,000 per citation . . . that comes to 25 billion dollars!

Or we could do this on an annual basis. The author of that book had 18,000 citations last year, so that would be $1.8 billion. No way his employer is paying him anything like that. Indeed, you could ramp up his salary to college-football-coach level and he’d still be an amazing bargain.

P.S. Sholl adds:

On a milder note, it is worrying how the text goes from calculating “exactly” to giving an “estimate” for the same quantity over the space of just a few sentences.

Indeed. Saying that he can “calculate exactly how much a single citation is worth,” that’s just tacky. Too bad they can’t take this guy and Dr. Oz and the divorce prediction guy and the disgraced primatologist and the pizzagate guy and the voodoo guy and the sleep guy and all the rest of them in a single university where they could offer some sort of interdisciplinary degree in exaggeration and hype. It shouldn’t be hard to set this up with that $25 billion endowment.

P.P.S. Sholl also writes:

I am hoping you might prompt some more energetic people to fact check other parts of his book.

OK, fine. But I’m not gonna be the one to do this!

P.P.P.S. More here. Too much more, actually.

24 thoughts on “Albert-Laszlo Barabasi is underpaid. By a lot!

  1. Even ignoring the “actual” thing, I really can’t understand the logic as a proxy measurement. Would he also argue that every word in the research article is worth millions for the same reason?

    I wonder if he *actually* believes what he’s written there – that citations are worth hundreds of thousands? I find it hard to believe that he does, but then why would he throw it into his book?

    • Anonny:

      My guess is that he’s fundamentally a non-quantitative thinker. Remember last year when two leading print and TV commentators promoted the following ridiculously innumerate claim: “Bloomberg spent $500 million on ads. The U.S. population is 327 million. He could have given each American $1 million and still have money left over”? Lots of people don’t understand numbers. Or they’re really busy and spend too much time on TV or writing books and they say things that don’t make sense without checking them.

      • With these kinds of mistakes, when it comes to journalists or scientists in non-quant focused fields then sure, it’s probably a lack of understanding. But according to Google, this guy’s a physicist. My undergrad was in Physics (made the leap to stats for my graduate studies), and at least in my experience that required quite a lot of quantitative shrewdness to understand!

        It reminds me a lot of that economist who claimed that a significance test provided strong evidence of fraud in the 2020 US Presidential elections… even an economist should have a good enough grasp of basic stats to know better!

  2. Dude. What?

    I bought a car for thirty thousand dollars. The car has four wheels. Does that mean each wheel is worth $7500?

    Or might it be that the $30k is also going towards other parts of the car?

    I feel like research often produces things besides citations!

    This is the dumbest logic I’ve heard since, well, the Brian Williams / Mara Gay thing.

    • James:

      I don’t think his argument is quite that stupid. The claim is not that each citation has an inherent value of $100,000. The claim is that the total value of research is X, and research produces Y citations, and that X/Y = $100,000, so if we divvy up X to research papers in proportion to their citations, you’ll get a value of $100,000 per citation. That is, it’s not that the citations have that value, it’s that the existence of the citations are a measure of its inherent value.

      In your example, it would be like taking the total amount spent on cars and dividing it by the total number of wheels and saying that the cost of each wheel is whatever. This would not literally represent the cost of the wheel; rather it would represent the entire cost of the vehicle.

      • Ok, if the price tag is just meant to convey the amount of funds the research will get, then sure.

        Still, like the Williams/Gay thing, it seems off by orders of magnitude. The budget of the NSF is on the order of $10B/y. This one guy probably hasn’t gotten 2.5y worth of NSF funding for himself across his career!

        • I think the $25B is meant to represent value (to society) created, not value captured. So, by his logic, he’s a huge bargain, with whatever funding he’s getting.

          I still think the $100k/citation number is silly for reasons elsewhere in this thread.

        • If it’s value created to society then we’re back to the car example: associating the value of a whole with only a tiny part of the whole, and being amazed that such a tiny part would be so valuable.

          Citations don’t create value, the other outputs of research do. If I add four more wheels to my car that doesn’t double its value to society.

          Hey, why stop at things with prices. A human life is priceless. Each human has two nostrils. Therefore a human nostril is priceless.

        • “Citations don’t create value, the other outputs of research do. If I add four more wheels to my car that doesn’t double its value to society.”

          +1. “It cost a ton of money and therefore this is valuable” sounds like Bitcoin thinking. Or maybe it explains why the US healthcare system is the best in the world.

          “A human life is priceless. Each human has two nostrils. Therefore a human nostril is priceless.”

          Of course, you won’t be quite as dismissive when you wake up one morning without nostrils.

      • If that’s Barabasi’s argument it’s badly written, since it really does sound like he thinks X/Y is an actual assessment of the monetary worth of citations, and not a marker of the proportion of cited/uncited research. He writes, “Citations, then, serve as currency in the scientific world. I don’t mean that as a figure of speech.” … and then the quotes in your post above.
        (Google Books is great…)

        In addition, I’m pretty sure his numbers are too high, even if one buys the inane logic, but I’ll try to sketch this later.

  3. In a nutshell, this represent all of the bad practices that have been highlighted on this blog. He clearly wrote his headline story first – then supporting it with “analysis” was simple. The idea that research is valuable can make sense, and the idea that you can try to quantify it is also reasonable. But, from that point on, his statements are indefensible. I won’t detail the reasons, since most readers of this blog know well how many things are wrong with his inferences (or, more technically, not necessarily wrong but unsupported). What really bothers me is that he does understand the assumptions and limitations – I don’t for a minute believe he is ignorant of these. It is just that it doesn’t matter to him. He gets attention and citations, and after all, research is a good cause and probably undervalued. However, I have to add that I believe his research is highly overvalued.

  4. This kind of indiscriminate averaging drives me crazy. I used to work with a military training pipeline at one point in my career and would frequently hear that the cost per trainee was a simple average from which the additional cost of adding trainees could be deduced. It’s a ridiculous statement that ignores the distinction between fixed and variable costs and fully ignores the idea of incremental cost. This simple calculation was used to justify all sorts of funding decisions and as a result, even very smart people seemed unwilling to acknowledge the flaw in reasoning.

    • This kind of indiscriminate averaging is how we get the “cost per mile driven” of cars. This is generally a terrible way to think about car maintenance costs. Even an affine function total = fixedperyear + costpermile * miles is better.

      But more to the point, a thing not mentioned here is that he implicitly assumes all citations are positive. That’s far from true. Citations to LaCour’s work or the STAP cell people are clearly negative, and relatively large negative. In fact you could imagine entire fields where the average is negative (maybe some kinds of psychology, some kinds of medicine, some kinds of education research or criminology or whatever)

      When a certain field is filled with lots of NHST and noise-mining it’s probably a net loss for society

      • Daniel:

        I think the cost-per-mile-driven thing is much much much better. For one thing, it’s in the right order of magnitude. For another, it’s useful because lots and lots of people seem to act as if the cost to operating the car is gas and nothing else. They really don’t seem to think the other costs are there at all!

        • Agreed it’s much better, but it’s still dumb. Suppose you typically drive say $10k miles a year. Cost per mile for those 10k miles is $0.8236/mile (current AAA estimate), so $8236 to operate your car. Now suppose you want to go for an extra vacation, it’ll be 1000 miles. The estimated cost is $824. But suppose you’re getting 25MPG and $4/gal gas costs, so gas cost is $160. What’s the extra $664? It doesn’t exist. Oh sure, wear and tear wise there’s some marginal cost, and there’s maybe a marginal increase for your insurance, but all those put together marginally for the extra 1000 miles will be at most another $160, and probably more like half that. So let’s say half that 664/80 = 8.3, you’re overestimating the extra costs by almost a factor of 10.

        • You can see where this goes wrong if you do some algebra. Suppose that total = fixed + c * miles, then you do a “per mile” calculation, you get:

          published statistic = total/mile = fixed/mile + c

          asymptotically for very large number of miles, you get close to c the real marginal cost per mile because fixed/mile goes to zero, but for real world driving when fixed/mile >> c then the real marginal cost c is much less than the apparent value estimate you get by doing this division.

          The same thing is going on with “Citations”. It’s clear that most citations are of nearly negligible value. We could change the citation norm among journals and require people to do a broad literature search and cite anything that has similar keywords to their article. This might triple the number of citations we get this year. But the “value of research” would go up by basically zero (or even go down actually). This is like c ~ 0 in my model.

          Almost all of the “value of research” comes from something *other* than citation. Citation itself is nearly valueless, kinda like playing a pop song as a ringtone doesn’t add significantly to the value that the artist creates for society each time someone calls you (or maybe xeroxing some worksheets for 5th graders to practice algebra doesn’t add to the dollar value that the curriculum people are creating by writing the worksheets)

          Research is a public good, ideally it creates “knowledge” which is valuable in the sense that people can do a better job of living their lives (by making better decisions about things) but once you have that “knowledge” out there, the marginal cost of teaching one more person that F=ma or that doing more than about 5 practice problems per night is unhelpful for 5th grade math students, is more or less zero. You can’t sell it to the “consumers” because they can just “give it away to others”.

          Anyway, it costs money, let’s say $100k to do a research study on 10000 fifth grade kids to figure out what the optimal amount of time per night is to study basic algebra, but applying that knowledge to all 50M kids doesn’t cost anything extra, so it’s not $10/kid implies $500M/yr in extra cost to do a good job teaching math.

  5. I find it useful to flip the frame around. He uses his (silly) estimate to argue that a citation is worth $100,000. How valuable! But it is also consistent with the cost of a citation being $100,000. How wasteful!

  6. The inverse of this algorithm must necessarily exist. That is, the value of a company, based solely on original research, must be equivalent to the number of citations (plus an amount that’s not directly derived from the research). In other words, the algorithm must be homomorphic. (Also, this is a way of quantifying how wrong, as everyone on this forum agrees, this algorithm is.)

    Consider Zap Energy. It’s a company that’s building a fusion reactor. It’s based solely on original research. And, it only has one product, is not commercially viable but has a value that’s publicly available (it just went through a series B round). The public value of Zap Energy should be equivalent to the number of citations used for its fusion reactor.

  7. Wow, now the average research article being behind a $50 paywall seems like a bargain! It’s shocking that noone ever actually pays for these, for a mere $500 everyone could get a millions-worth of scientific research.

  8. It is honestly insane that he believes this, and I truly hope that this is not something that he actually believes, and instead something he is just saying for his book.

  9. Will someone hire me as their research budget administrator? Because I’ve found a way to ace the Barabasi test! Fund a lot of researchers to write papers that are not read or cited. Now take my total budget divided by the total citations, and I have generated monetary value of infinity!
    (Has he heard of survivorship bias?)

Leave a Reply

Your email address will not be published. Required fields are marked *