Washington Post falls for that horrible air-rage study, and what gets me really angry about this

Someone just pointed me to this news article entitled, “Air rage incidents are on the rise. First-class sections aren’t helping,” which falls hook, line, and sinker for a notorious discredited study that appeared in PPNAS last year.

I can hardly blame the Washington Post reporter for getting this one wrong, given that NPR swallowed the bait last May. Still, I hate to see it happen. What next—himmicanes??

I look forward to the day when news reporters won’t automatically think that things are true, just cos they’re published in tabloid science journals.

What gets me angry here

I can hardly blame the Washington Post reporter for, on deadline, making a mistake that was also made by reporters from so many other news organizations. And it’s hard even for me to get angry at the authors of the air-rage paper, those statistical naifs who are just doing research the way their teachers taught them, gathering data, looking for statistically significant patterns, and then telling the kind of catchy causal stories that will get their work published in top journals.

No, what really annoys me are those sections of the scientific community who promote junk science and who acquiesce in its promotion, scientists who seem to have a say-no-evil, all-publicity-is-good-publicity attitude. I’m angry at scientists who have lifetime job contracts and who have received the laurels of their professions, who promote junk science.

Also I’m angry at the National Academy of Sciences, which is burning a little bit of its prestige—and the prestige of science in general—every time it publishes something like the himmicanes and air rage papers. No, NAS, its not always good news when you’re mentioned in the Washington Post. Not this time. Not at all.

11 thoughts on “Washington Post falls for that horrible air-rage study, and what gets me really angry about this

  1. It appears that reporters are under pressure to mention “scientific” research no matter what. The article was off to a decent start before the author started treating the PPNAS study as something grave and authoritative.

    This seems rather reasonable:

    “‘The conditions on board just lead to more potential for air rage,’ says Sara Nelson, international president of the Association of Flight Attendants-CWA. Nelson ticks off the reasons: tight seating, fuller flights, alcohol consumption, fewer flight attendants, human nature.”

    But apparently such observations weren’t enough. There’s a strange belief out there that if you refer to research of some kind–no matter what its quality–you’ve done *your* research and can speak with authority.

    • Diana:

      I agree. The qualitative evidence is just fine here. Also interesting that “first class sections” were not mentioned by the flight attendants’ spokesperson. The first-class-seating thing got into the headline of the article solely through the junk-science study.

  2. >”According to data from the International Air Transport Association, a trade association that represents 265 airlines and 83 percent of global air traffic, unruly passengers on planes worldwide increased 14 percent in 2015 over the previous year. In 2015, there were 10,854 reported cases of such incidents, or one in every 1,205 flights. In 2014, there were 9,316 incidents, or one per 1,282 flights.”

    https://www.washingtonpost.com/lifestyle/travel/air-rage-incidents-are-on-the-rise-first-class-sections-arent-helping/2017/01/23/4e3e6752-dd99-11e6-918c-99ede3c8cafa_story.html

    I don’t think this means there is necessarily a rise in incidents, as opposed to reporting practices. It looks like the definition of unruly behaviour was was clarified in 2014, incentives to report such incidents (right of recourse) have been increased, and they don’t think anywhere near all the applicable incidents are being reported. I would also wonder if the list of participating airlines has stayed constant.

    >”In April 2014, States recognized the significance of this issue and the need to strengthen
    international air law when they attended a Diplomatic Conference and approved the Montreal Protocol
    2014 (MP14).

    […]

    1.3 Unruly behaviour includes assault of other passengers or crew, sexual abuse or
    harassment, illegal consumption of narcotics, refusal to comply with safety instructions, making threats
    that could affect the safety of the crew, passengers and aircraft, and other types of disorderly behaviour
    that impact good order and discipline on board.

    1.4 Since 2007, IATA has collected statistics on unruly passengers from the Safety Trend
    Evaluation Analysis and Data Exchange System (STEADES), a database owned and managed by IATA
    to which 190 airlines submit periodic reports on a non-mandatory basis. It is likely that the statistics
    significantly under-estimate the extent of the problem.

    1.5 Statistics from STEADES on unruly passenger incidents in 2014 are shown in Appendix
    A. In 2014, there was an average of 1 unruly passenger incident per 1,289 flights, an increase when
    compared to 1 incident for every 1,362 flights in 2013. Together with the statistics from individual civil
    aviation authorities and feedback from member airlines, this data suggests that unruly passenger incidents
    have become more prevalent.”

    […]

    2.5 Right of Recourse – airlines usually have to bear the costs incurred as a result of unruly
    passenger incidents and these can be substantial, in some instances over US$200,000. MP14 recognizes
    that airlines may have a right to seek compensation for costs incurred as a

    http://www.icao.int/Meetings/a39/Documents/WP/wp_139_en.pdf

    I think the uncertainties and faulty analysis surrounding this topic pretty much render it “fake news”.

  3. Aren’t you being a little too nice to DeCelles here? It looks like she was interviewed very recently for this article and didn’t seem to have learned anything since publishing it. She’s at Harvard (albeit as a lecturer/visiting professor) and her co-author is a Harvard professor established enough to have another guy’s name in front of his job title.

    Given what the professors have read about their study since they published, when a reporter calls, aren’t they obligated to call attention to the problems? Second, shouldn’t there be a “Fiske” rule of some kind, where the WaPo, NYT and other top ten general publications have to say ‘wait, I cannot write a story about anything she was involved in without checking for debunkings”?

    • Pk:

      I agree that DeCelles et al. haven’t behaved in an ideal way: as scientists, they should learn from expert criticism rather than sloughing it aside. But I recognize that it can be tough if you “work hard and play by the rules” (as Bill Clinton put it) and then get get a big splashy success, and then the rug is pulled out from under you. It seems that a lot of people have difficulty maintaining scientific objectivity in such a situation. The cost to objectivity is admitting that (a) your big success was no success at all, (b) perhaps a big chunk of your research direction is misguided, (c) you might have to relearn research methods. Meantime, at Harvard there are various tenured cheerleaders who will pat you on the head and assure you that “the replication rate in psychology is quite high—indeed, it is statistically indistinguishable from 100%.”

      So, lots of incentives in the wrong direction.

      There is one big incentive in the right direction: if you clean up your research methods you have a chance of really learning something about the world. But to even accept that this is an incentive, the researcher needs to let go of his or her conception of science.

      • I agree in that it would be hard for me to want to take some grand action to undermine such a prestige-enhancing publication as an early-career scholar whose potential tenure-track career destinations may hinge on the publication seeming legit to hiring committees.

        One would hope that peer reviewers would snuff these things out before the Andrew Gelmans and many others notice the serious problems. TBH, it is surprising to me that the issue with confounding flights with people per flight wasn’t caught prior to publication. The archetypal adversarial peer reviewer is normally looking for those things even when they don’t exist.

        • Jacob:

          It’s my impression that PPNAS reviewers are much more concerned about excitement level than correctness. That is, they’ll reject a paper without batting an eye if they don’t deem it’s important enough. But if they think it’s important, then they’re much more interested the paper’s message than in the details of how it got there. Also, lots of scientists nowadays are suckers for “big data” and that was one of the gimmicks of this particular paper, that they had this unique dataset from an airline. Finally, the editor of this paper was Susan T. Fiske, who just loves this sort of stuff. First, this means that she can send the paper to sympathetic reviewers. Second, even if one of the reviewers points out flaws, she can still accept the paper. Remember, she accepted the himmicanes paper too.

        • It’s not just that Fiske loves this kind of stuff but the paper was also co-authored by one of her former students (Norton).

  4. But the article (in the Washington Post) is dated January 24. If that was 2016, then never mind. But if it is 2017 then shouldn’t we expect reporters to be a bit more thorough? Like googling enough to find that criticism has been levied against the research? (given the time lags for Andrew’s posts, I suspect this was 2016 – in which case my comment is irrelevant).

  5. I agree, the _real_ problems driving this are embedded in senior levels of academia, University presidents and VPs, academic societies and citation/funding successful acdemic super cliques.

    Not sure what advice I would have to give folks working in those positions and I doubt there are simple solutions – these organisations are super-tankers that can’t turn on a dime. For instance, it can take most of a University president’s energy for a year or so to successfully convince a (undesirable) Dean that they would be better off at someone’s else’s university.

    Here is my for instance – I had to review a claim based on a published epidemiology study. Some of the authors of that paper had written a previous methodology paper on the methodology they used in that paper. It was clearly one of the better methodology papers in the area and concluded with only method A should be used as method B is not conservative enough. Strangely in their later published epidemiology study they used only method B and there was not mention of why B instead of A (I doubt any of the reviewers actually read the methodology paper even though it was referenced.)

    Why would this happen? I did notice that a few of the coauthors of published epidemiology study were nearing the end of their first five years of their first academic appointments and that if method A had been used there would be no p_values below the magic .05 (with B a couple were just below .05). Hard to be sure in any given case but one could certainly appreciate the temptation to relax ones’ standard to keep one’s academic career or a few junior colleagues careers going.

Leave a Reply to Marcus Cancel reply

Your email address will not be published. Required fields are marked *