How an academic urban legend can spread because of the difficulty of clear citation

Allan Dafoe writes:

I just came across this article about academic urban legends spreading because of sloppy citation practices. I found it fascinating and relevant to the conversations on your blog.

The article is by Ole Bjørn Rekdal and it is indeed fascinating. It begins as follows:

Many of the messages presented in respectable scientific publications are, in fact, based on various forms of rumors. Some of these rumors appear so frequently, and in such complex, colorful, and entertaining ways that we can think of them as academic urban legends. . . .

To illustrate this phenomenon, I draw upon a remarkable case in which a decimal point error appears to have misled millions into believing that spinach is a good nutritional source of iron. Through this example, I demonstrate how an academic urban legend can be conceived and born, and can continue to grow and reproduce within academia and beyond.

The story begins:

The following quote, including the reference, is taken from an article published by K. Sune Larsson in the Journal of Internal Medicine:

The myth from the 1930s that spinach is a rich source of iron was due to misleading information in the original publication: a malpositioned decimal point gave a 10-fold overestimate of iron content [Hamblin, 1981]. (Larsson, 1995: 448–449)

The quote caught my [Rekdal’s] attention for two reasons. First, it falsified an idea that I had carried with me since I was a child, that spinach is an excellent source of iron. The most striking thing, however, was that a single decimal point, misplaced 80 years ago, had affected not just myself and my now deceased parents, but also a large number of others in what we place on our table.

But you have to read Rekdal’s article for the full story. There are no bad guys here—no Weggy or Weick-style plagiarism. Rather, it’s a twisty tale indicating the challenges of conveying information that we hear about second- or third-hand.

18 thoughts on “How an academic urban legend can spread because of the difficulty of clear citation

  1. I think this is also the sort of thing that results in people saying that physical laws are fine-tuned to the level of 10^60 or whatever.

    If you simply pause and think about that for a few seconds, it is a pretty good guess that no physical measurement of anything, ever, has been made to that level of precision, so that simply cannot be true, or at least it cannot be something we know to be true.

    • Physical quantities are often measured relative to other physical quantities. In this case (the question of why the cosmological constant is so small), what is measured is the ratio between the energy density of the cosmological constant and the energy density of dark matter. This is about 2.

      However, the 10^{-60} refers to the ratio between the measured energy density of the cosmological constant and the *theoretically expected* energy density of the cosmological constant. Measurement error is not a problem for a theoretical expectation.

      Hope that clarifies things for you; the point of the argument is simply that our theoretical expectations are inadequate in some way.

  2. I’m sure this kind of thing is common in many fields.

    My favorite example in psychology is probably Lhermitte (1983), who studied patients with frontal lobe injury. He devised an elaborate experimental procedure in which he showed that his patients would sometimes, if you provoked them a little bit and got them in the habit of doing it, grab and use objects at inappropriate times (he called this “utilization bhaviour”). Three decades later, you’ll find comments all over the literature about how “patients with frontal lobe injury just can’t help using any object in front of them”, but they don’t cite the original study, they just cite other people who have cited other people who have cited the original study, and no one seems to understand what Lhermitte actually did or found.

  3. >> “How an academic urban legend can spread because of the difficulty of clear citation”

    “clear citation” solves nothing if the clearly-cited source provides false incomplete, or unverifiable information

    difficult detective work often involved in verifying complex citation trails is akin to the problem of replicating research studies

    too much faith placed in academic citation rituals

  4. In my high school chemistry class we did an analysis of the available iron in spinach based on the level of raw iron and calcium and the level of oxalic acid, as measured by some USDA lab or something (it wasn’t a chemistry experiment, it was just a paper calculation).

    If I remember correctly we found that spinach had a negative iron and calcium content (that is, the oxalate binds all the iron and calcium in the leaves, with some oxalate left over to bind additional iron and calcium from your body). Of course, biochemistry is a lot more complex than what would happen in a beaker or whatever, so your mileage may vary, however I do know someone who suffers from kidney stones, and research I did years ago suggested that people with hard water or who take calcium supplements after meals have LOWER kidney stone incidence, and the theory is that having calcium in your stomach to bind oxalate before it enters your bloodstream is the suspected causal mechanism.

    All that is to say that things are more complicated than a beaker based analysis with or without decimal point errors might suggest.

  5. Is it limited to citation? Take the gun discussion and ask yourself how many times you’ve heard the phrase “semi-automatic”. As far as I can tell, there’s no effort to recognize that almost any weapon is semi-automatic because it only mean you pull the trigger, it fires and the next cartridge loads and then you need to pull the trigger again. Every pistol is semi-automatic, as are most shotguns: they have magazines with multiple rounds and when you pull the trigger the next cartridge moves into firing position. This level of understanding isn’t even at the Wikipedia level but I never hear it. I don’t even expect them to get that an “assault rifle” is a fully automatic weapon, a machine gun, that is highly regulated and essentially banned for nearly all private ownership (and is rarely used in crime).

    To go slightly deeper, the “assault weapons ban” was passed in 1994 and expired in 2004 but no reporter seems able to look up what it actually said. That is, it didn’t ban anything but instead limited the number of features weapons could have, so a rifle could have a pistol grip but not a folding stock (or vice versa) or a flash suppressor but not a pistol grip. It did the same for pistols and shotguns: limited the features they could have. It “banned” magazines over 10 rounds but allowed existing magazines to stay. And a few states – 8? – still have these rules in effect.

    My point is a) on perhaps the issue of the last several years, b) the press can’t manage to check Wikipedia and c) can’t convey even the most basic salient facts about such an important issue … and anyone expects them to get other, more obscure or complicated stuff right?

    • Not to beat on this but here’s today’s Boston Globe about gun control:

      “Q: Would a renewed ban on assault weapons have stopped the Orlando gunman from buying the weapon he used in the attack?

      A: Possibly. The gunman, Omar Mateen, who had been on a terror watch list until 2014, bought a Sig Sauer MCX rifle — a spinoff of the military-style AR-15 — as well as a 9-millimeter handgun at a Florida gun shop about a week before the attack. The original assault weapons ban passed by Congress in 1994 prohibited the sale of 19 kinds of semi-automatic weapons, including AR-15s, AK-47s and Uzis, as well as dozens of types of handguns, ammunition and other rifles that met certain military-style conditions. It is unclear how many of these military-style features were on the AR-15-style rifle that Mateen used, so it may or may not have fallen under the ban.”

      This is nearly gibberish. The law listed 19 specific weapons, but of course you could then make a similar weapon with fewer features but still semi-automatic because nearly all weapons are semi-automatic. In other words, since he bought a weapon in the week or so before killing people, then it would have been legal to buy that weapon and it still would have been a semi-automatic, probably with a pistol grip, and certainly capable of using detachable magazines. If he had access to pre-existing and thus legal large capacity magazines then he would have been completely legal. And while it’s easier to get large capacity magazines for rifles, you can buy 50 round magazines for pistols – looks like a drum sitting below the pistol – and 28 or 30+ round magazines for pistols are fairly common.

    • Sorry if I was off topic but my point I thought was fairly basic: huge issue, not something tiny like the amount of iron in spinach, that is current, not rooted in a misunderstanding of decades ago, and the media can’t even get the basics correct. I might have used BMI as an example, but that would be just coming up with examples of the same thing: that an idea or measure is rooted in nonsense layered over in time, which lack of initiative and energy keeps in place. If I could solve the issue by referring to a wiki page, then this problem wouldn’t exist. As in, here’s the wiki link for spinach, which discusses the urban legend: Did that make the problem go away?

  6. Britta Stordahl once wrote a paper complaining that her article which explicitly debunking a ‘myth’ was cited in support of that ‘myth’:

    I’ve had the some problem with one of my own papers, in which I strongly advise against using a certain equation…only to see my paper cited in support of using that equation.

Leave a Reply

Your email address will not be published. Required fields are marked *