Skip to content
 

He’s annoyed that PNAS desk-rejected his article.

Baruch Eitam writes:

This may be a rant I don’t think so and so I am sharing it with you but one can never be sure.

Just had a paper desk rejected from PNAS. You may not appreciate the journal but it is one of the most important journals a psychologist can publish in. So if I think work out of my lab is important enough to interest “the general scientific community” I send it there.

The paper in question builds on 10 years of my group’s research on what we termed reinforcement/reward from being effective. In an elaborate program of research (published in psychology journals), we found, using the key measures of cognitive psychology—response time and response frequency—that “pure effectiveness” is rewarding (it’s more theory-laden but I think this is sufficient).

In the relevant study, we measured more than 100 clinically depressed individuals and showed that their responsiveness to effectiveness is identical to that of the general population.

Why is this important? Because major depression is thought to be caused by/to involve muted responsiveness to reinforcers so our study suggests that either this is incorrect or that multiple reward systems exist.

This is the editorial board’s evaluation which is seemingly the basis for desk rejection (my emphasis):

Editorial Board Member Comments:

Thank you for sending PNAS your paper on this important topic. Unfortunately, our general-science audience would be unlikely to find this research programmatic enough to be fully convinced—nor does it have a distinct innovation in other ways (sample, methods). A specialized journal might be more receptive.

So why did I think this would interest you? I find this to a manifestation of both the confusion/crises that scientific psychology is in and a strong example of how this biases the work done by experimental psychologists looking for recognition.

Why? To buy into our interpretation of the data you need to accept the fact that my behavioural measures index reinforcement. As I wrote above we laboured to make this case for years. But experimental psychologists themselves are not sufficiently sure of this nor would a group of them agree on what would count as a “significant (basic) discovery”.

This means that we (whoever we are) are F. As the gates to the “general” journals open only by hype or curiosities. Or even worse, by the status of the authors.

Note: As some of my best friends and respected colleagues published fantastic papers in such journals I would like to stress that this is not always the case: only that as a general rule I find it to be correct.

What do you think?

My reply: Oh, PNAS is terrible! I’ve published there myself, but they publish lots of crap. But any journal is pretty random.

Eitam’s paper (with Shirel Bakbani-Elkayam, Tohar Dolev-Amit, Eitan Hemed, and Sigal Zilcha-Mano) is here. I have no view on it; you can form your own judgment on where it should be published. As noted above, publication in any particular journal is random, so I don’t think you can hold it against PNAS for rejecting this particular submission.

14 Comments

  1. Shravan says:

    PNAS is a brand-name scientist journal. If you are not eating at the top table, or don’t have an author who is, and/or don’t have friends on the inside, just publish elsewhere. Why do you care? I don’t submit there. I used to get rejected by Cognition when i was young, and people told me i could only count as a psycholinguist if i managed to publish there. But then istarted reading Cognition papers and realized it was a white old boys’ club. The articles were often a joke and no serious reviewer could have both read and understood these papers and accepted them. The only rational explanation was that the big name of the author or authors got people through. I review for Cognition sometimes but i never submit there, although i may do it soon just for fun. My students say i am a big name now, so maybe my garbage research can appear there too.

    PS ppl also told me a real psycholinguist gets divorced at least once, but that didn’t happen either, and i am more real than many. Ppl come up with all kinds of metrics of success, pulled out of their nether regions.

    • Keith O’Rourke says:

      > maybe my garbage research can appear there too.
      Good idea – just disguise thoughtful research as being garbage ;-)

      • Shravan says:

        You’re joking but that is what it takes. Null result? Claim you found evidence for no difference. Large absolute t-value from a power 0.10 study? Make big claims that you have solved all of mankind’s problems.

    • Andrew says:

      Shravan:

      PNAS isn’t as bad as Perspectives on Psychological Science. They ran a lying hit piece on me, failed to even consider correcting it—and I’m about as connected as can be! The team at the Association for Psychological Science doesn’t just publish junk science, they also go on the offensive to attack dissenters. PNAS follows a more passive, flood-the-zone, approach.

      • Shravan says:

        Yes, OK, I don’t know much about these journals but I can believe that they treat outsiders (outside psych) badly. They wouldn’t treat Meehl or Cohen the same way as they treated you if these people had brought up statistical objections (which is what I assume you did).

        I am pissed with PNAS for a different reason: they published a study on a particular topic from MIT researchers, then some (not well known) Indian guys wrote a paper on the exact same topic. Desk-rejected. Reason: not of broad interest to the community. I learnt about this from complaints I heard from the Indian authors, who wrote to me for advice. This incident was strong evidence for me (of course there are many others, from different journas) that the branding-level of the researcher matters much more than anything else. Level 4 or above only. The rest can go look at cat photos.

        • Rahul says:

          Although I understand where you are coming from, let me play devils advocate for a bit:

          I read this sort of argument a lot and it puzzles me: “Forum X does not judge work on it’s inherent merit but on the credentials of the guys who wrote it or the journal that published it”

          But is this a bug or a feature? In a scarcity of time and resources, isn’t some sort of triage required? Is there some other way of prioritizing, if not based on credentials. e.g. If I am browsing through news and I percieve “Andrew Gelman” (or MIT or some such signal) then obviously I pay more attention. Isn’t this sort of credential based prioritization integral to how we work?

          Of course, I see the downsides of overdoing this (eg legacy biases etc. ) but all I’m saying is credentials have a role. There has to be a hierarchy of credibility of researchers, journals & institutions. To judge an idea purely on its merits (agnostic to who it comes from) is an idealistic fantasy. Branding is ubiquitous for a reason!

          • Shravan says:

            Sorry for not responding to this interesting point earlier. Ihad anexciting weekend, high fever, corona test (5 days till the result arrives), isolationin hospital, then they realized i had a sepsis. This might sound like good news but it probablymeans my dialysis shunt is infected and will have to be completely removed. This is probably worse than getting the virus.

            Sure. I think of it in Bayesian terms. What you propose is using very informative priors, maybe to the extent that the data have absolutely no influence on the posterior, the data here is the paper itself. This will work well in many cases. After all, many of the brand-name people have done and do excellent work.

            At least some ofthese brandnames develop a i-can-never-be-wrong attitude and produce substandard work that is unquestioningly published in major journals. Soon they also develop expert opinions on stuff they know literally nothing about (pinker, gigerenzer, on corona). Because of their wide ranging influence, their opinions are considered credible. Feature or bug?

            Women who are not brand name will tend to be rejected because informative priors (there have been cases of reviewers advising all female authors to get a man involved). Same thing with obscure researchers in places like Iran, China, India, etc. Such people can publish in second tier journals. Feature or bug?

            The interesting question for me is: does it hurt to publish in journals that do not have names like Nature or Science or PNAS. My conclusion after 20 years of doing research is no. My advice to someone like Eitan is to know his place in the food chain, and do the best he can and keep his head down. Ignore the branding culture prevalent in science today, it comes from a hunger for celebrity and fame. Don’t aspire to having a media link on your home page, don’t aspire to give a ted talk. Leave allthat to the people hungry for public attention and hungry to show off their elephant boots, made from legally culled elephants.

            • Martha (Smith) says:

              Shravan,
              Best wishes for a speedy recovery!

            • Nick Adams says:

              I agree with the last point. I read 25 or so medical journals regularly and am no more influenced by something published in the New England Journal of Medicine than a more obscure journal – it’s the research quality that counts, not the label.

            • Shravan says:

              Sorry, Eitam. Typing on an ipad, hard. I keep missing the right key.

              • Baruch Eitam says:

                Sharavan,
                First of all, I hope your medical complication is not (too) severe.
                Second, thanks for the wise advise.
                As I wrote, this all feels somewhat silly given the mayhem all around us (I myself having completely immersed in trying to help for the last 10 days), my work, and more so my grievances seem of so little significance.
                And yet, I don’t think i am averse to fame but not of the NYT or Noah Harari type, I would like recognition for the discoveries I think I have made (so many I’s).
                My key point was not being rejected from PNAS (i get rejected from many outlets, very often) but rather, current psychology’s inability to agree on what ‘a discovery’ is (or even what would the evidence for it be).
                More harshly, it seems that psychologists themselves do not even believe that response times (potentially The tool of cognitive psychology) can serve as (sufficient) evidence for any significant discovery.
                So it was the *reason* for the desk rejection that jolted me rather than the desk rejection (from PNAS or whatever) itself. Keep healthy.

    • Martha (Smith) says:

      “pulled out of their nether regions” — My poor old departed Dad is probably slapping his head in his grave, thinking, “Why didn’t I ever think of that one!”

  2. Baruch Eitam says:

    I must admit that I feel sorry for sending this (it was quite a few months ago), a matter of timing I guess. Hope everyone is safe and sound.

  3. Baruch Eitam says:

    Now I bungled it — apologies Shravan.

Leave a Reply to Shravan