Fernando Martel Garcia points me to this news article by Ben Goldacre:
For anyone with medical training, mainstream media coverage of science can be an uncomfortable read. It is common to find correlational findings misrepresented as denoting causation, for example, or findings in animal studies confidently exaggerated to make claims about treatment for humans. But who is responsible for these misrepresentations?
In the linked paper (doi:10.1136/bmj.g7015) Sumner and colleagues found that much of the exaggeration in mainstream media coverage of health research—statements that went beyond findings in the academic paper—was already present in the press release sent out to journalists by the academic institution itself.
Sumner and colleagues identified all 462 press releases on health research from 20 leading UK universities over one year. They traced 668 associated news stories . . .
The story is pretty much as you’d predict: a lot of the exaggeration comes in the press release.
I remarked that this makes sense. I agree. Of course, this is just a start, as I’m sure a lot of academics would be happy to put their names on various exaggerated claims! See, for example, here, where the researchers in question were very active with the publicity, and in which they dramatically overstated the implications on individual-level behavior that could be drawn from their state-level analysis. The lead research in this case was just a law professor, but still, we’d like to see better.
As this example illustrates, the problem is not necessarily any sort of conscious exaggeration or hype: I assume that the researchers in question really believe that their claims are supported by their data. For that matter, I assume that disgraced primatologist Mark Hauser really believes his theories.
To put it another reason: be skeptical of press releases, not because they’re written by sleazy public relations people, but because they’re written by, or with the collaboration, of researchers who know enough to make a superficially convincing case but not enough to recognize the flaws in their reasoning.
Along with (medical doctor and epidemiologist)Ben Goldacre, when it comes to analyzing health and medical pronouncements, the best journalist websites belong to Gary Schwitzer (Healthnewsreview.org)and Susan Perry (Minnpost.com). Many others in the media are often merely uninformed stenographers.
Read this untangling of how good science led to a disastrous press release before you hold scientists accountable for press releases written by others.
http://www.boulderweekly.com/article-13656-fact-to-fiction.html
Even if the PR guys wrote the release the Professors get to see it before it goes out. Right?
So why did they not change or object to whatever was misleading?
No, they didn’t get to see it. Did you read the article in the link? Do you know how communications offices work?
I cannot claim I know how communication offices work universally, but in the few news releases I was involved in, my boss (the PI on the project) sure insisted that the Media Relations guys run the final copy by him before it was released. In fact, I remember he annoyed some of the PR guys by making nit-picky changes, but at least we never had this we-never-read-what-they-released excuse.
PS. Yes, I read the article. But it was rather long so I might have missed the relevant part. If so, apologies.
Can you point out where it explicitly says that the authors never got to see the PR release before release. And if they didn’t why is it so hard for Professors to insist that they are shown the draft?
It looks like the lady who wrote the press note is standing by what she wrote. In fact, even the PI on the study says later in the article that, “What she [the PR lady] wrote was not incorrect”
Basically it looks to me that they all were OK with the wording till the uproar & what we are seeing now is post hoc firefighting & damage mitigation.
Its senior university folks (the VPs) fighting for funding and prestige (students and more funding). Most of them have little choice other than to resign.
Sadly, Professors who exaggerate & hype get away with zero repercussions.
Is there a good reason why we should be softer on them than the plagiarists & data fakers?
Rahul:
Professors who don’t exaggerate & hype get threatened by the VP of Research’s office :-(
(Well at least at one university.)
Rahul:
The plagiarists and data fakers are often the same people who exaggerate and hype. Perhaps the hype motivates them to cheat. I dunno, but it’s all evilicious to me.
@Andrew:
I disagree. I don’t think the plagiarists necessarily overlap with hype guys. For one, there’s a lot more of the exaggerators than plagiarists or data fakers (I think). Obviously, since we punish one set of crimes far far harshly than the other.
Plagiarism or data faking (if caught) often (though not always) leads to a lost job. When was the last time an academic was fired for hype?
I think we are dealing with two orthogonal, low correlation offenses.
Do you have any data / evidence to show that “plagiarists and data fakers are often the same people who exaggerate and hype”?
Rahul:
The plagiarists and fakers I discuss on this blog have typically been involved in hype. Without the hype, I probably wouldn’t have heard of these people in the first place.
Ok, maybe plagiarists also tend to by hypers. But not the other way around.
Lots of hypers don’t indulge in any plagiarism. I’d say a very very small proportion of hypers are plagiarists also.
By going exclusively after the plagiarists we are missing the bigger chunk of the problem.
But I don’t go exclusively after the plagiarists! I’m hard on lots of hypers, including Satoshi Kanazawa, David Brooks, the himmicane guys, and that law professor mentioned in the link above, just for a few examples.
No not you, but the penalties. Just compare the average outcome for a plagiarizer you covered on the blog versus a hyper.
I’m saying that we, as a system, let the hypers get away with it. We are hard on plagiarizers but forgiving of hypers.
Rahul:
You write, “Plagiarism or data faking (if caught) often (though not always) leads to a lost job.” Data faking if another story, but, for better or worse, the plagiarists in academia who get caught don’t seem to lose their jobs. Ed Wegman, Frank Fischer, Laurence Tribe, Alan Dershowitz, Ian Ayres, Matthew Whitaker: I don’t think any of these guys lost their jobs.
Journalists who are caught plagiarizing often lose their jobs, and that’s another story—indeed, I’ve discussed this on the blog—but in academia, not so much.
@Andrew:
Academic plagiarists who did lose jobs:
http://www.dailymail.co.uk/news/article-2026121/Academic-Philip-Baker-quits-plagiarising-graduation-speech.html
http://www.post-gazette.com/local/city/2011/10/08/Pitt-professor-loses-job-after-admitting-to-plagiarism/stories/201110080148
http://ethiopiaforums.com/professor-lost-job-twice-after-accusation-of-plagiarism/6461/
http://amherststudent.amherst.edu/?q=article/2012/09/25/basler-resigns-after-admitting-plagiarism
http://www.cu-aaup.org/documents-3/report-on-the-termination-of-ward-churchill-2/
Admittedly, some indulged in both plagiarism & fabrication so it is hard to tell what exactly got them fired.
But now, can we list Professors who got fired for hype? Maybe there’s some but I don’t know.
Rahul:
I don’t know of any professors (or, for that matter, journalists) who lost their jobs based on hype alone. It’s tough—there’s less of a sharp line for hype than there is for plaig or fraud—but I see your point. For example, when James Watson hyped that cancer cure, in some sense he was just doing his job, as typically construed. Similarly with all the letters of recommendations we write, the abstracts to many of our research papers, etc. We’re basically told to hype.
@Andrew:
Yes, that’s exactly my point. We behave like the proverbial drunk looking for his lost car keys under the streetlight.
We go after plagiarism & fraud because those are so easy to prove. But the damage done by hype or exaggerated press releases is more insidious & far more commonplace.
I think it would do the system a lot of good to focus on the hypers for a while. And looking at the impact section of DoE / NSF / NHS funding proposals might be a good start. Those are often a goldmine of hype & fantasy.
Publish and perish at Imperial College London: the death of Stefan Grimm:
http://goo.gl/857BWY
Let’s not forget that funding from research come from organizations who push for “social impact,” applications, benefits to the public, and so on. Very few seem to have much interest in funding research that is “just science.” NIH is a case in point. Just try to get something funded on the grounds that it will advance scientific understanding but probably with no implications at all for any other benefit. (Maybe math is an exception, but my impression is that not a lot of them are funded well if at all.
Yep.
That’s why we see funding proposals for Quantum Computing talk about making internet searches faster.
https://pressroom.usc.edu/quantum-computers-could-help-search-engines-keep-up-with-the-internets-growth/
I see that the USC PR office is doing great with the hype machine.
Sad. Very sad.
And, frankly, this vast over-hyping is about to get much, much worse for NIH-funded research. This spring, NIH applications will start using a new biosketch format (http://drugmonkey.scientopia.org/2014/12/02/the-new-nih-biosketch-is-here/)… for those who may not know what a biosketch is, it’s basically a very boiled down and highly standardized version of a CV that must be submitted for every named person on a grant proposal.
The new section C in the biosketch, which replaces a simple listing of up to 15 key publications, requests the applicant to “briefly describe up to five of your most significant contributions to science.” This is an invitation for mega-hype… it’s more about selling yourself and your “science”. Nonsense.
Is NIH being taken over by MBAs?
The Rabett variation is that all press releases within the last five years need to be attached to any grant proposal.