Researcher offers ridiculous reasons for refusing to reassess work in light of serious criticism

Jordan Anaya writes:

This response from Barbara Fredrickson to one of Nick’s articles got posted the other day.

Alex Holcombe has a screenshot of the article on Twitter.

The issue that I have with the response is that she says she stands by the peer review process that led to her article getting published. But Nick’s critique also underwent peer review! (I assume letters to the journal undergo peer review but see Nick’s tweet here.)

So what’s different between the peer review process that makes her article infallible and Nick’s peer review process that makes his article so bad it’s not worth responding to?

I dunno, but Fredrickson’s reasons for not responding to the criticism (which, by the way, is full of specifics) are absolutely bizarre. She writes:

Readers should be made aware that the current criticisms continue a long line of misleading commentaries and reanalyses by this set of authors that (a) repeatedly target me and my collaborators, (b) dates back to 2013, and (c) spans multiple topic areas. I [Fredrickson] take this history to undermine the professional credibility of these authors’ opinions and approaches.

OK, let’s go through this carefully:

– “a long line …”: These authors have found many problems in the published work by Fredrickson and collaborators.
– “misleading”: That’s cheap talk given that Fredrickson provides zero examples of anything misleading that was written.
– “repeatedly”: Again, the authors found many problems, also the articles in question are still in the published record. Given that this work continues to be cited, if it has errors it should continue to be criticized.
– “target”: Is it “targeting” to point out errors in published papers? If so, why is this a bad thing.
– “dates back to 2013”: OK, so some errors by Fredrickson et al. have been pointed out for several years, so they’re old news. But the articles in question are still in the published record. Given that this work continues to be cited, if it has errors it should continue to be criticized.
– “spans multiple topic areas”: Huh? Is there a rule that people are only supposed to write about one topic area?

This is weird, weird stuff, and as always it makes me sad, more than anything else. Fredrickson is a prominent figure in her field and has a secure job. She could admit her errors and move on. But no, it’s never back down, never admit error. It’s so so sad, to think that someone is in a position to learn from error but refuses to do so. I’ve seen this enough that it hardly surprises me. But I still find it upsetting.

Again, even setting all substance aside, it’s bizarre that Fredrickson proposes that the critics be discounted because they been making these criticisms for several years. The published work is part of a stream of work that is many years old, hence it’s no surprise that the criticism is many years old too.

P.S. Above I wrote that Fredrickson didn’t “respond to” the criticism. Let me clarify that in this case I think a reasonable response would be for her to:

1. Retract the published claims, and
2. Thank the critics for tracking down the problems in those published papers.

That would put her in position to do:

3. Correct the supplementary record (not just in journal articles but also in books, lectures, etc., explain what went wrong so that people don’t mistakenly believe those erroneous published claims), and
4. Do some new research trying to figure out what’s really going on, without getting fooled by statistical noise or fake math.

Step 4 is what it’s all about. The ultimate goal is to help people, right?

37 thoughts on “Researcher offers ridiculous reasons for refusing to reassess work in light of serious criticism

        • Should Fredrickson’s response ever have been published?

          If I’m the editor of that journal and I receive a screed on personal grudges for a response to a criticism of the author’s work, the appropriate course of action is to advise the author of the screed that they have two weeks to provide a substantive response, if they choose not to do so, the critical letter goes to press with a “decline to respond” notice from the author of the original paper.

          It should be understood by everyone in science that personal grievances aren’t published in scholarly journals.

        • It may not be entirely coincidental that Carol Nickerson and I had made a COPE complaint about the editor-in-chief’s handling of a commentary that we had written article about an almost-unrelated article in the same journal (Psychoneuroendocrinology) a few years ago.

        • +1

          Publishing it was definitely a mistake. It should have been acknowledged as such and not explained away with a vague “at the request of the author”. (As far as I can tell there’s no policy to allow such a withdrawal.) The reasoning behind its original publication and its subsequent un-publication should be made explicit.

        • Well its publication certainly reflected poorly on its author. But it reflects poorly on all of science, too, that such petty tripe would be set in type at all. I think withdrawing it was the best outcome.

  1. So once again we see that it is too much to ask authors to honestly respond to criticism. So once again, the obvious next best solution is to let both sides speak. The corollary is that when we shut down one side of a debate, we can expect the worst elements of the other side to be unleashed undermining the credibility of the entire field.

  2. Reading between the lines, Fredrickson’s response is basically “fuck those guys, they don’t like me personally, meanwhile I have these other people who say I’m right”. I don’t think there’s much value to going through this reply in detail beyond that – it’s way more of a grudge than an argument.

    • Zhou:

      The scary thing is that this sort of strategy can work. Fredrickson has powerful allies such as Roy Baumeister and Steven Pinker who seem pretty much willing to ignore devastating criticism by focusing on the purported flaws in the tone of the criticism. And, as is often the case in such settings, the poor tone of the defenders of the status quo is ignored; all that matters it that it is possible, some way or another, to not look head-on at criticism of published and celebrated research in their field.

      • To be fair, Steven Pinker did refer to the Losada article as “a silly bit of physics envy in psychology” in a tweet (sapinker/status/359721149170593792) when our critique was published in American Psychologist.

        • Nick:

          But Pinker also endorsed the newspaper article defending Fredrickson’s work. He had to balance between the principle of supporting the academic establishment, and the principle of describing published research accurately, and he chose the former. Maybe that’s the right position to take—to publicly support the promotion of bad work, in order to support the general edifice of science. It’s not the position that I take, but maybe he’s right. The point here is that Fredrickson can count on people such as Baumeister to continue puffing her work and minimizing her errors, and she can count on people such as Pinker to cheer on when critics of her work are attacked. The active support of the Baumeisters of academia, and the tacit of support the Pinkers, that can count for a lot.

        • Pardis Sabeti January 21, 2018, Boston Globe, regarding the “power pose”

          “but the effect of specific poses on people’s feelings of power has been replicated in 17 independent studies, with promising evidence of effects on many other psychological states in many more. ”

          https://www3.bostonglobe.com/ideas/2018/01/21/for-better-science-call-off-revolutionaries/8FFEmBAPCDW3IWYJwKF31L/story.html?event=event25&arc404=true

          Who here agrees with this claim? I simply don’t believe it.

          There is a thread of possibility that there is an effect “of specific poses on people’s FEELINGS [my caps] of power”. I could accept a small but measurable effect. But the claim in the power pose paper was dramatically farther reaching: “As predicted, high power posers performed better and were more likely to be chosen for hire” – that’s a pretty far stretch from feelings.

        • Defending other work, Sabeti continues:

          “These novel ideas are important because they awaken new thinking”

          It’s great to awaken new thinking with new ideas! Publish an essay or write a blog and flesh them out all you want, but don’t make false claims about them.

  3. Ah, the wonderful wacky world of academia. I must confess, I steered clear of a teaching profession after watching 40 years of Dad’sgonized academic career. He had some great periods during his tenure. But he was too effete for his competitive colleagues. I will say the most creative intellectuals have a very tough go at it as some scientists at MIT had shared with me when I was in Boston.

    Actually Rawls also referred to the changes in university life that contributed to a very unpleasoant atmosphere for many academics.

    We saw an end of an era in so far as academic prestige.

    • Actually Rawls also referred to the changes in university life that contributed to a very unpleasoant atmosphere for many academics.

      Specifics? Anything we can read?

  4. I will say little irks me. I do get a bit annoyed when someone can avoid being taken to task for is a patently ridiculous thesis/argument. If I don’t have a good rebuttal, I will admit to it. I think it is a rebellion against the arrogance that some display.

  5. Anyone should love for their work to be “targeted”. That means someone read it and payed attention. I guess the critique could be worthless though (“where is your calculation of statistical significance”, etc).

  6. ‘Repeatedly’, ‘long line’, ‘targeted’, are words designed to characterize the criticism as bullying or harrassment. Reminds me of your recent post about a female author incorrectly explaining NHST. She took to twitter to claim she had been ‘villified’.
    It’s simple cry-bullying.

  7. FWIW, I attempted to replicate Fredrickson et al’ analysis using rstanarm (via stan_glm). Using model stacking across the various combinations of predictors (and transformations of predictors), I landed on a median estimated effect of about .05, but it was pretty noisy (sd = .13). Average treatment effect estimates of the stacked models was an effect size of .3, but only about 60% of posterior predicted estimates were greater than zero. In short, I agree with Friedman et al…assuming the theoretical model was correct, the original paper made big claims based on inconclusive data.

  8. Boxers often use the “power pose” or the ring equivalent of the “stare down” because, in Evander Holyfield’s summary “it works”(to intimidate the opponent and make him psychologically vulnerable. Boxers tend to believe (or say they do) that boxing is a battle of “willpower”, the man who wants to win more will tend to win more. They are right that it works—sometimes. It also fails to work sometimes. As it often happens that both boxers try to stare down the other, the interpretation is that one man wanted to win more than the other, so he won. His stare down was more potent. Hence power poses work, maybe, sometimes, at least in the mind of the power-poser. Meanwhile, despite loss of tenure at Harvard bid, Amy is laughing all the way to the bank, as Liberace put it.

  9. This acting by Barbara Fredrickson reminds me to a publication from 4 April 2016 in ‘Zoology in the Middle East’, see https://www.tandfonline.com/doi/full/10.1080/09397140.2016.1172405

    This publication about Virginia Barbour does not refer to our requests from the end of June 2015 to publisher TF and to EiC Max Kasparek for access to the raw research data of Al-Sheikhly et al. (2013). This publication about Virginia Barbour does not refer as well to the acting of COPE when COPE was asked to help us with getting access to these raw research data.

    TF responded on our request on 16 June 2016. This response was received after I had started to send TF daily reminders. I reacted in these daily reminders on the contents of auto-replies from TF which I had received on earlier reminders. That’s quite a weird way of communicating about our request for access to the raw data. Anyway, it turned out to be a way to get a response, although only after almost one year.

    TF wrote in this response from 16 June 2016: “Like the majority of scientific journals, this one does not compel the author to provide the raw data of the research to anyone. We will not be responding to your request to provide you with this.”. TF also wrote in the same response: “no-one in this organisation will respond further to your emails” and “we will not be engaging further with you”. EiC Max Kasparek had until today never communicated with me about this request to get access to the raw research data.

    A manuscript which is mainly based on the findings of two reports at https://osf.io/5pnk7/ is at the moment at a peer-reviewed journal. Responses from journal editors reveal that submitting a manuscript about this fraudulent Basra Reed-warbler study is challenging. The by far most remarkable statement was received from Alan Lee, EiC of the peer-reviewed journal Ostrich. Alan Lee wrote to me on 31 May 2019:

    “My current position of the MEZ article is neutral, and any opinion I have would not withstand rigorous interrogation, as I have to the best of my knowledge never seen the species in question, nor do I know anyone connected to the journal MEZ in which this research is published, or anyone associated with the MEZ article, and I have never been to either Iran or Iraq. My unqualified opinion on the matter is thus based only on people I do not know, have never met, and thus whose professional integrity I cannot judge either way.” [the ‘MEZ article’ is the fraudulent Basra Reed-warbler study].

    This statement implies that Dr Alan Lee, http://www.fitzpatrick.uct.ac.za/fitz/staff/research/lee , a Research Associate of the premium ornithological institute of a whole continent, and EiC of the premium ornithological journal of a whole continent, is towards my opinion arguing:

    (1): he is unable to judge manuscripts for Ostrich when he has not observed the study species by himself;
    (2): he is unable to judge manuscripts for Ostrich when he has not visited the country where the field data have been collected;
    (3): he is unable to judge manuscripts for Ostrich when he has not visited any of the surrounding countries of the country where the field data have been collected;
    (4): he is unable to judge manuscripts for Ostrich when he does not know the authors and/or others who are involved in the study.

    This is towards my opinion a remarkable point of view of an Editor-in-Chief (of an ornithological journal).

  10. I’ve worked in life science research (cell biology and regenerative medicine) for over a decade and I had no idea that such studies were even conducted in other fields, let alone be taken seriously. It’s terrifying.

    I’m not qualified to judge the statistical analysis methods used in the paper, but I’d love to read the methodology, particularly how telomere length was assessed, which tissues were sampled, and over what time period.

    Perhaps I’ve been missing out on a whole exciting world of psychophysioloogical study!

Leave a Reply

Your email address will not be published. Required fields are marked *