The Case for More False Positives in Anti-doping Testing

Kaiser Fung was ahead of the curve on Lance Armstrong:

The media has gotten the statistics totally backwards.

On the one hand, they faithfully report the colorful stories of athletes who fail drug tests pleading their innocence. (I have written about the Spanish cyclist Alberto Contador here.) On the other hand, they unquestioningly report athletes who claim “hundreds of negative tests” prove their honesty. Putting these two together implies that the media believes that negative test results are highly reliable while positive test results are unreliable.

The reality is just the opposite. When an athlete tests positive, it’s almost sure that he/she has doped. Sure, most of the clean athletes will test negative but what is often missed is that the majority of dopers will also test negative.

We don’t need to do any computation to see that this is true. In most major sports competitions, the proportion of tests declared positive is typically below 1%. If you believe that the proportion of dopers is higher than 1%, then it is 100% certain that some dopers got away. If you believe 10% are dopers, then at least 9 out of 10 dopers will test negative!

As Kaiser points out in the case of Lance Armstrong, passing 500 tests is not as impressive as it might sound:

The independence assumption is the key here. If I were a doper, and I pass the test, this tells me that my doping regimen is pretty good; if I pass two tests, it increases my confidence that my doping regimen is good; the more tests I pass, the more I feel good about the expertise of my doping advisors.

Another way to think about this is the fact that every athlete who have confessed and/or failed a positive test will have had a long string of negative tests prior to failing. Unless one believes these athletes (like Andy Pettite) who claim that the only time they took steroids was the time they got caught, it is very difficult to make the case that a string of negatives means much.

Also here:

The anti-doping agencies are so concerned about not falsely accusing anyone that they leave a gigantic hole for dopers to walk through. . . . While we think about Armstrong’s plight, let’s not forget about this fact: every one of those who now confessed passed hundreds of tests in their careers, just like Armstrong did. In fact, fallen stars like Tyler Hamilton and Floyd Landis also passed lots of tests before they got caught. In effect, dopers face a lottery with high odds of winning and low odds of losing. . . .

Another myth shattered by this scandal is the idea that stars don’t need to cheat. It is most likely the opposite. At the very top of any sport, especially a sport that pays, the difference between the number 1 and the number 2 is vast in terms of financial reward but infinitestimal in terms of physics. Every little advantage counts. Placebos count.

It’s hard to imagine why someone who has no chance of winning anything would take drugs that might kill them. So, when they say everyone was cheating, I [Kaiser] wonder if they meant everyone who was competitive was cheating.

13 thoughts on “The Case for More False Positives in Anti-doping Testing

  1. OK – disclaimer first: I was a major Lance fan.

    To some degree I don’t think this is a question of the stats. Doping rules are essentially a specification on the sportspersons bodily contents, and the point of a specification is to either pass or fail, accepting that there will be results either side of the line that are not ideal. We accept this on a daily basis when we buy pharmaceuticals – when we buy them, they have passed a specification, we don’t ask the pharmacist about the false negative rate on ibuprofen. I think that areas such as false positives on driving tests are probably more dangerous to more people than false negatives on PEDs but the principle of passing a specification is essentially the same.

    My other thought is a little more philosophical – I think the reason I get upset about sportspeople taking illegal substances is more about the fact that this destroys the ‘awe’ about the achievement rather than the fact that someone else should have won. Without PEDs we can appreciate how much better these people are than us; with PEDs that link is lost. To my mind this justifies a reasonable false negative rate – I don’t want to destroy people’s careers wrongly just to preserve the ‘awe’.

  2. Not everyone competitive is cheating. As Taleb says, to prove the existence of black swan I only need produce one. The year before I started my PhD – back in 1984 – I won the world judo championships. Even then, people who doped swore you couldn’t win without it. I won. I knew people who doped back then. Some won, many did not. My daughter won an Olympic medal in judo and is now UFC champion in mixed martial arts. She doesn’t do PEDs. Every time an athlete is caught and I hear everyone does it used as an excuse it infuriates me. Not every world champion cheats.

  3. The other thing to point out here is that when people have tried to actually verify that “500 tests” number, the counts they come up with are far, far lower. I recall someone trying to count all the times Lance was tested, and with very generous counting could only get to 250 or so. (By generous, one example is that they assumed that every time Lance finished in the top 3 in a race in the 90’s, they counted that as a time when he would have been tested, but we know for a fact that the testing during that era was sporadic and of course was not testing for many of the drugs being used at the time.)

  4. “Putting these two together implies that the media believes that negative test results are highly reliable while positive test results are unreliable.”

    Not at all. It just means that the media thinks 500 tests are more reliable than 1.

  5. “If you believe that the proportion of dopers is higher than 1%, then it is 100% certain that some dopers got away. If you believe 10% are dopers, then at least 9 out of 10 dopers will test negative!”

    And what if I believe 0.5% are dopers? What then?

    • Rahul:

      I believe that less than 0.5% of cyclists are dopers. I personally have no problem biking the 4 blocks to my office without the aid of any artificial stimulants. I don’t even drink coffee! I do, however, believe that more than 0.5% of members of the U.S. Postal Service Pro Cycling Team were dopers.

  6. I don’t think there is a case for more false positives because the Western justice system is based on a presumption of innocence. We’d rather let a thief go free than jail the wrong man. That by definition needs a test with few false positives.

  7. Pingback: Success » Diigo Links (weekly)

  8. There are a couple of substantive issues important to this discussion:

    1) Many of the “false negatives” are true negatives in the sense that there is no drug present in the blood stream, but false negatives in the sense that they don’t detect people who have, at some time, doped. EPO is transient in the human body. A larger problem than false negative tests are true negative tests that are only negative due to the time at which the tests are given. Increasing the number of false positives by increasing the sensitivity of the test likely would not catch any more dopers – giving the testers unlimited access to urine/blood samples throughout the training cycle would do much better.

    2) Since, on any cycling team, there will be only 1 or 2 cyclists with a chance of winning any particular race, the motivation for many cyclists to dope is to keep a job. If the difference between the winner and second place is small, so is the difference between those who are scraping by on a pro team and those that barely missed the cut. The monetary advantages exist at any level among the pro ranks.

    • This sentence struck me:

      >>”The decline in steroid use has allowed the natural order to reassert itself: before steroids overwhelmed women’s track in the Seventies, black women like Wilma Rudolph and Wyomia Tyus dominated sprinting. Today, lead by young Marian Jones, who is potentially the Carl Lewis of women’s track, black women rule once more.”<<

  9. This “statistical” analysis seems to have a glaring flaw by assuming that all test and all samples are the same. But are the tests and the samples the same across the statistics.

    How does this analysis account for the concept of a) dope detection improved so after many initial tests what was undetectable was detectable, or b) that the doper/handler/body screwed up and the post-doping levels fluctuated to a higher than “normal” level and were caught?

Comments are closed.