From the Archives of Psychological Science

Jay Livingston pointed me to PostSecret, which I’d never heard of before, and the above image, which apparently first appeared in 2011.

P.S. The image and the title of this post do not quite align. My concern with the journal Psychological Science is about incompetent work rather than made-up data.

11 thoughts on “From the Archives of Psychological Science

  1. There are (some) bad apples in every occupational and any other group. Procedures are in place to minimize their harm. They can’t be completely and permanently eliminated. What else is new?

    • Renzo:

      The problem is not just “bad apples”; it’s also people who are trying their best but don’t understand the research process. That’s why we speak of cargo-cult science. Honesty and transparency are not enough. The “I made up the data” story is funny/horrifying, but if that’s all it was, I wouldn’t always be going on about Psychological Science. The larger problem is people who didn’t make up the data but are still doing bad science, and the leaders in academia who promote and defend such work.

      And, no, none of this is “new.” But I think it’s important to write about problems, even when they are not new.

      • Seems like much of it comes down to researchers not understanding the statistical methods they use. Seems like much of the problem (but not all) could be solved if professional statisticians were consulted anytime a statistical test is used, or to have some certification process for non statistician researchers so that the chances of bad experimental design or overly flawed interpretation are less.

        • Maybe “professional statisticians” are a more homogeneously competent group than other professionals, but requiring “professional economists” to be consulted in economic decisions (not to mention research) would hardly make a dent in bad research. Your comment appears to make good research a matter of “understanding the statistical methods they use.” That is certainly necessary, but far from sufficient to ensure good research practice. Sound critical thinking is required, and no discipline has a lock on that. Nor does most training of professionals really ensure these traits – even without the problematic incentives that magnify the problems greatly. Certifying non statistician researchers sounds like a pipe dream to me. Who would do the certification? If it’s Andrew, I’m fine with that. But, more realistically, it will end up being based on credentials, number of publications or citations, etc. Back to the same set of problems.

        • It’s not that good research is a matter of understanding the statistics used, but a deeper understanding certainly would improve quality. There is a bunch of completely inconsequential research out there that uses statistics in flawed ways. Making their use of statistics perfect wouldn’t make it “good research,” per se, but it would improve its “quality.” Lots of research doesn’t even need much critical thinking.

  2. I certainly agree that leaders in psychology should not promote, deny, or defend dishonest or incompetent work. I can’t recall in my decades in the field of personality and social psychology, any leader promoting or defending poor or dishonest work. If you can cite some concrete examples of a psychology leader promoting or defending incompetent work I would surely disagree with them. To be clear I am aware of examples of dishonest and/or incompetent work emanating from some bad apples and a few overambitious grad students but not so aware of leaders promoting or defending it. Most offenders who get caught are soundly castigated. Which leaders were you thinking of specifically? I would like to disapprove of them if they are promoting or defending dishonest or incompetent work.

  3. Renzo Alves and others, I have an example of a psychologist who is regarded as a leader in the field and who was “promoting or defending dishonest or incompetent work.” This leader in the field was not amused when I confronted her with this behaviour. She therefore decided to send me serious legal threats. This implies that I will not disclose her name. Excuse me very much, but that’s how it works when you are a mr. nobody.

Leave a Reply

Your email address will not be published. Required fields are marked *