Andrew Perrin nails it:
Twice a year, like clockwork, the ethics cops at the IRB [institutional review board, the group on campus that has to approve research involving human subjects] take a break from deciding whether or not radioactive isotopes can be administered to prison populations to cure restless-leg syndrome to dream up some fancy new way in which participating in an automated telephone poll might cause harm.
Perrin adds:
The list of exemptions to IRB review is too short and, more importantly, contains no guiding principle as to what makes exempt. . . . [and] Even exemptions require approval by the IRB.
He also voices a thought I’ve had many times, which is that there are all sorts of things you or I or anyone else can do on the street (for example, go up to people and ask them personal questions, drop objects and see if people pick them up, stage fights with our friends to see the reactions of bystanders, etc etc etc) but for which we have to go through an IRB in our roles as researchers, teachers, or students.
Recently a high school student contacted me about a research project he has, involving online surveys. He had some interesting ideas and it looked like a great project. But it might never get done. Why? His high school has no IRB. But he can’t do it through the Columbia IRB because he’s not a Columbia student (and it’s his project, not mine, so it doesn’t help that I work here.)
Bottom line: Everybody’s afraid of getting sued. That’s the problem with living in a country that’s run by lawyers.
Two other things:
1. At the university, endless hours are wasted on getting permissions to do innocuous surveys. Meanwhile, what about dangerous medical experiments, the kind of study where a drug company crams some illegal aliens into a bunch of Miami hotel rooms? Don’t worry, those guys use commercial IRB’s that approve everything.
2. I do have problems with the ethics of surveys that don’t pay their participants. But that has nothing to do with the questions that are being asked. Instead, the IRB has all this “minimal risk” B.S.
Of course this is not the world’s most important problem, but it really is pretty bogus.
Hi Andrew, You’re right, that IRBs don’t “get” some of the worst players in medical shenanigans. But from my point of view as a patient who’s done research, they don’t go far enough. You, or at least I, wouldn’t want a high school student, or a grad student, or a professor who’s confident he knows what’s best, surveying people about anything. You don’t know what they’ll do with the information. It’s a hassle, sure, but worth it and necessary.
Elaine:
I know what you mean, but it’s a free country. If a student wants to survey people, that might be a rude thing to do, and you and others should be able to freely criticize the activity, but I don’t think the student should need the permission of a committee to do so. I could see that the committee could be required if the student wants to use the school’s institutional affiliation (for example, “I’m calling from Columbia University and . . .,” but if the student wants to just do the survey, I don’t think it makes sense to require approval.
Why is the IRB required for the high school project? Does the high school student have federal funding, like an NSF grant? Is the student planning on publishing in an external venue? If neither of those cases are true, they shouldn’t need IRB approval for this, unless their specific high school has an IRB policy about this (and, since the school doesn’t have an IRB, I’m guessing they have no policy). What rule would the student be breaking?
Lorin:
There’s no funding involved. I have no idea why the student was told he needs an IRB (which the school doesn’t have), but this is what he was told, and indeed he canceled his project because of this.
I have a funny feeling that some idiot teacher had heard that if you want to do a survey, you need IRB approval and gave the student incorrect information.
This is my intuition as well. IRB approval is only required in the narrow venue of academic and pharma research. I’m not even aware if the IRB has any legal mandate at all.
Actually, high school doing research for competitions (Intel STS–the former “Westinghouse” competition)DO need IRB approval.
For what it’s worth, APA publications require that authors sign a statement that they have behaved ethically. But the section about institutional approval is conditional: it says “when institutional approval is required, psychologists… obtain prior approval” (see here: http://www.apa.org/pubs/authors/ethics02.pdf) This seems to leave an opening for research activities that are not required by law to have IRB approval.
I completely agree with the post and disagree with this comment. IRBs are not at all a protection against a professor or student misinterpreting their results. They don’t even seem concerned with interpretation. Andrew is right. They are only concerned about the risk of getting sued, and they seem incredibly paranoid, especially because they only have incentives in one direction. That is, they never get a pat on the back when a great study gets approved, but they probably get fired or chastised if a study gets approved and then results in a lawsuit or protest. They seem especially paranoid when it comes to the internet. I had a similar experience to Andrew’s with a colleague’s proposed online study of a rare pregnancy condition. Our IRB said it was too risky and did not approve it. I’m still unclear what the risk was: maybe that someone would get carpel tunnel syndrome while filling out the survey?
I do a lot of work with secondary analysis of already collected data. Some of this data is from medical claims. We freely send this information (along with every possible piece of identifying information) to insurance companies in order to process bills. But IRBs are concerned with researchers seeing data that has been stripped of identifying information.
I recognize and support the idea that patients are allowed to have privacy. But I remain unclear why it is in anybody’s best interest for information on how effective or economical a therapy is to be supressed. Certainly, in aggregate, making this information harder to get is hurting patients.
And this is before we even consider the possibility of patient contact. It is a very puzzling system.
I don’t see why this point is so obviously an argument against IRB: “…there are all sorts of things you or I or anyone else can do on the street (for example, go up to people and ask them personal questions, drop objects and see if people pick them up, stage fights with our friends to see the reactions of bystanders, etc etc etc) but for which we have to go through an IRB in our roles as researchers, teachers, or students.”
I think the reason to make research more difficult (by requiring various ethical reflections to be made explicitly) is that it is driven by more than simple curiosity. Sometimes research will be carried out without any real curiosity on the part of the researcher or student (in both cases, there may be talk of an “assignment”). This also means that there is no end to the amount inconvenience that the research community could cause to broader society.
By contrast, the little everyday experiments mentioned here have a dynamic that is directly connected to the “researcher’s” (i.e., the amateur’s) natural curiosity. The experiment would never have been thought of without actual curiosity, and it ends when it is satisfied.
I think we have right to inconvenience our fellow humans to satisfy our curiosity. But I don’t think we an unlimited right to bother them in our pursuit of “the truth” in a more universal sense. That would simply get out of hand.
Thomas:
I like the ideal of ethical standards of the don’t-bother-people-unless-you-have-a-good-reason variety, and I agree that there is a potential problem if thousands or millions of students are running around bugging people to do experiments that the students are only doing because they need course credits. I just don’t think the IRB is a sensible way to address this.
Yes, I was not defending the IRB (I’ve never experienced one so I don’t know). I was just puzzled about that particular argument. The mere fact that you can do something for personal reasons without approval does not make it absurd (or even questionable) that you would have to seek approval to do it for professional reasons.
BTW, I don’t think the problem arises just because students need credits. Researchers (and their institutions) need publications. Companies need “studies that show”. To think all, or even most, of this is driven by basic curiosity is naive. So we need something to complicate the drive for ever greater datasets.
The role of the IRB is not to put a stop to pointless or bothersome surveys, though. Lack of resources already does a pretty good job of placing a limit on them.
I suppose that’s not it’s primary aim, if you put it like that. But consider Andrew’s example of “stag[ing] fights with our friends to see the reactions of bystanders”. Anyone is entitled to do that for fun, i.e., to satisfy their curiosity on a given day (although there are no doubt limits to how far you are allowed to take it, especially in regards to involving the police). But surely an IRB would raise the question of whether the public space of the NY subway should be invaded by, say, 100 runs of this “experiment” during rush hour on chosen week, using public funding to hire actors? That’s an important “ethical” question.
But again, I’m fine with it not being part of the IRB’s remit. I was just puzzled by the way Andrew invoked the argument as if it were obvious.
IRBs’ remit is to protect research subjects. In today’s world, that has significant secondary benefit in decreasing the risk of successful lawsuits. Even with IRBs, Duke got their federally-funded research shut down in 1999, and much of Hopkins’ federally-funded research was suspended in 2001. While the regulators may well have acted with too heavy a hand, there’s quite a bit of agreement that the systems in place at these well-respected institutions (granted, neither is Columbia) didn’t adequately protect their subjects.
It’s easy to understand the stakes with experimental, potentially toxic drugs. Surveys are less clear, but under modern standards of concern for privacy even pieces of paper can pose “dangers” to subjects. I’d be glad to support a federal law waiving such concerns for everyone who has commented on this post, but the problem is that virtually every researcher is sure they’re not the problem — so where do we draw the line? I won’t defend the current standards, but it’s not easy to invent better alternatives.
At Shaping Tomorrow’s World, U of Western Australia cognitive sciences prof Stephan Lewandowsky mentioned a frothcoming paper in Psychological Science. Search for “NASA and the blogosphere.” This involved an Internet survey, which generated a firestorm in part of the blogosphere, which (as recounted in a dozen following posts there, especially the 1,000 or so comments) occasioned a plethora of intense accusations, including bad faith, bad statistics, violations of ethics rules as the goalposts kept moving. Obviously certain people did not like the results of the paper at all, for reasons perhaps outside normal scientific arguments.
He noted:
People were irate enough (about a self-selected, voluntary Internet poll) to do a FOI request to U WA, in an effort to claim wrongdoing. IRBs may be paranoid, but some may have reason to be.
There are some serious discussions of statistics in this sequence, but it also turns out that it may well have generated lots of data for more papers about conspiracist ideation.
Related to topcis oft discussed here, in a comment following one of the more recent posts, Lewandowsky writes:
Wow, there is a lot of misinformation on here. First, read the Belmont Report to understand why IRBs exist and what their mandate is. (IRB’s legal mandate is restricted to FDA-regulated research and research funded by federal agencies that adhere to the Common Rule; institutions can dictate their own policy for what the IRB oversees). Then read OHRP’s Advanced Notice of Proposed Rulemaking to see that much consideration is being given to expanding what can be exempted from IRB. Also, please provide the data that says “commercial IRBs approve everything”. Have you read up on the GAO investigation in which 2 commercial IRBs disapproved an unethical study? Furthermore, did you know that many commercial IRBs will review high school projects to determine if they IRB exempt – pro bono? Lastly, for the investigator whose on-line survey wasn’t approved, did you talk to the IRB to understand how the study could be done in an approvable way?
Rose:
I will ignore your instructions to me to read things. In response to your other questions:
1. Of course I was exaggerating when I wrote that commercial IRB’s approve everything. My point is that they can be lax on real issues of safety, whereas the university IRB’s tend to raise issues on everything.
2. For the high school student mentioned above, I got the impression that there was no IRB to talk to, that was the problem.
“That’s the problem with living in a country that’s run by lawyers.”
I thought it was run by fat-cat banker/Wall St. types.
I understand the frustration with IRB boards. Indeed, back when I was a lowly RA a VA Hospital, we had to wait 6 mo. for the board to meet so that we could change the wording of a recruitment flier. That said, I think that we ethical scientists miss the benefits that come from community oversight.
Any regulation, almost by definition, will cause inconvenience for those following the rules. Time and time again though, we are faced with the fact that there are some researchers/organizations who (if allowed) will behave unethically and put patients at risk. IRBs are a check against dangerous curiosity (imperfect though they may be).
It is certainly reasonable to ask if there is a better way to police research.
Ian:
My suggestion: Less policing of surveys and recruitment flyers, more enforcement of the law (as in the Miami hotel example).
Andrew, did you comment on the ANPRM that suggests changing regulation to have less IRB oversight of surveys? It’s one thing to complain about things, but usually much more effective to participate in the change process.
As a former member of the support staff for a University IRB whose human subject research is primarily in the social and behavioral sciences (and humanities), I can say that I am ecstatic to have landed a new job allowing me to quit the IRB business once and for all. I used to be the person who communicated the IRB’s ridiculous, asinine “concerns” such as those described in the link in the original post above. For many years, I just accepted it all. In time I was able to see the inconsistency, the over-reaching, the stifling of real research, and the utter failure of so many IRB’s to apply the flexibility in the Regulations. Being the middle-man while the IRB members remained anonymous eventually began to make me feel like an idiot and liar. I am not researcher, but I believe in Research; I believe in education. I could not in good conscience continue to perpetuate this horrible, horrible system.