Skip to content
 

This is where they publish the stuff that can’t make it into Psychological Science

N=22 (link from here and here).

I was unhappy to see that the two news articles took the study at face value.

Photoplethysmograph, indeed.

As Daniel Kahneman might say, “You have no choice but to accept that the major conclusions of these studies are true.”

Not.

8 Comments

  1. EJ Wagenmakers says:

    Ah, but not just N=22. At least three key p-values are only significant with a one-sided test. A nice example to discuss in class though :-)

    • Andrew says:

      Yes, a sad sad comment that a journalistic outlet which would view the statements of various public figures with extreme skepticism, would just accept these N=22 cases because they come from a scientist.

    • Martha says:

      And there were (at least) 19 significance tests performed.

      But the last lines of the Jezebel link gave me a laugh that was worth the price of admission:
      “(I know that there is no analogous study for men, but I want to make it clear that my first sexual encounter with my current partner ended with him needing to be bandaged because I have no spatial reasoning abilities and accidentally pushed him off the bed.) (That has set the tone for our relationship.)”

  2. Stuart Buck says:

    Another horrible one: cbsnews.com/news/can-lots-of-sex-protect-the-prostate/

    “Don Juans of the world, take note: Men who sleep with lots of women may be less likely to develop prostate cancer than men who don’t play the field, a new Canadian study suggests. Researchers said they found that Montreal-area men who’d had more than 20 female sex partners in their lifetime had a 28 percent reduced risk of prostate cancer, compared with men who only ever slept with one woman. Previous studies have suggested that frequent ejaculation can protect against prostate cancer, said senior study author Marie-Elise Parent, an associate professor at the University of Montreal School of Public Health. One possible explanation: the beneficial effect might be due to a reduction in the concentration of cancer-causing substances in prostatic fluid, the researchers said.”

    Hmm, seems problematic, even just to assume that number of female partners equals number of ejaculations. More possible complications, though:

    “For instance, men who said they’d never had sexual intercourse were almost twice as likely to be diagnosed with prostate cancer as those who said they had intercourse. The researchers also found that a man’s risk of prostate cancer increased if he only slept with men. Having more than 20 male partners in one’s lifetime doubled the risk of prostate cancer, compared to men who never slept with another man.”

    If frequency is the main thing, there’s no reason for people who are gay or who have zero partners to see risk go in precisely the wrong direction.

    So what could have gone wrong? Multiple comparisons, perhaps?

    “We’re asking questions about everything you can think of under the sun,” [the researcher] said. “This is only one segment out of many, many others that we are investigating.’

    • Martha says:

      A brief glance at the paper suggests that indeed there was an abundance of multiple comparisons: Table 2 (“Odds ratios (OR) and 95% confidence intervals (CI) for the association between sexual activity, number of female sexual partners, and prostate cancer in the Prostate Cancer & Environment Study”) has around sixty CI’s. Tables 3 and 4 have still more.

  3. David Condon says:

    You’re being too generous. To quote from the abstract:

    “during the fertile (follicular) and non-fertile (luteal) phases of their menstrual cycle, with the order of cycle phase at the first testing session counter-balanced. Women tested first in the fertile phase showed significantly greater genital arousal to female-male penetrative versus oral sex in both testing sessions”

    So half are being tested in the fertile phase first, and half are non-fertile first. The n is 11.

  4. John says:

    Oh my lord that’s bad. Even if you ignore the multiple testing, N, experimenter degrees of freedom, etc. one the main conclusions isn’t justified. The difference between their significant physiological and non-significant subjective report isn’t significant.

    That said, the researchers were more circumspect than the news outlets. It is not just that the news outlets believed the researchers, it is that they drew a new conclusion that is both unwarranted, and not what the researchers concluded. The researchers said that women who run in the follicular phase of the study first have the preference toward penetrative sex. They did not say that women in the follicular phase prefer penetrative sex, which is how the news outlets reported it.

Leave a Reply