I sent Howard Wainer the causal quartets paper and he wrote that it reminded him of a theory he had about placebos:
I have always believed (without supporting evidence) that often a substantial amount of what is called a placebo effect is merely the result of nonresponse.
That is, there is a treatment and a control—the effect of the treatment is, say, on average positive, whereas the effect in the control condition is, on average, zero, but with a distribution around zero. Those in the control group who have a positive effect may believe they are getting the treatment and stay in the study, whereas those who feel no change or are feeling worse, are more likely to drop out. Thus when you average over just those who stay in the experiment there is a positive placebo effect.
I assume this idea is not original with me. Do you know of some source that goes into it in more detail with perhaps some supporting data?
I have no idea. I’ve always struggled to understand the placebo effect; here are some old posts:
– Placebos Have Side Effects Too
– The placebo effect in pharma
– A potential big problem with placebo tests in econometrics: they’re subject to the “difference between significant and non-significant is not itself statistically significant” issue
– Self-experimentation, placebo testing, and the great Linus Pauling conspiracy
– Lady in the Mirror
– Acupuncture paradox update
Anyway, there’s something about this topic that always gets me confused. So I won’t try to answer Howard’s question; I’ll just post it here as it might interest some of you.
Could placebo effects be a result of dosing or randomization errors? Has anyone looked into how often clinical trials screw up the administration?
Regression to the mean is the explanation that explains a lot of what people mis-interpret as patients benefitting from getting a placebo in a causal sense. I.e. in many studies patients only get included if they are “bad enough” on the variable under analysis, which then leads to regression to the mean due to this selection effect. Additionally, the setting of a clinical trial might get a patient more attention than usual (i.e. it can sometime be the setting rather than the placebo intervention). That’s not to say that patients do not sometimes benefit from getting a placebo, but people often get confused and assume that this is very clearly the case, when the other two explanations above tend to occur, too (esp. regression to the mean can have a huge effect).
http://www.dcscience.net/2015/12/11/placebo-effects-are-weak-regression-to-the-mean-is-the-main-reason-ineffective-treatments-appear-to-work/
This is an interesting read about placebo as a statistical artefact and regression to the mean.
One needs to be careful in terms of what you mean by ‘placebo effect’. In most psychiatric drug trials, the placebo is administered on top of standard of care that is provided to both placebo and drug groups. Some people define ‘placebo effect’ as the effect including standard of care and other people define it as the difference (not usually observed in the trial) in the placebo over and above the standard of care. Standard of care in a psychiatric drug trial would likely include periodic visits to the clinic, patient diaries, etc. Keeping track of your condition and seeing supportive clinical staff are both considered positive in terms of psych conditions.
Outside of standard of care effects, there is probably a contribution to placebo effect due to both regression to mean and also drop out of people not responding well. Sadly, I have seen a case where a placebo controlled clinical trial completely screwed up and everyone received drug.
Was there still a difference between the groups?
LOL. No.
Here is a relevant Cochrane Review on placebos: “We studied the effect of placebo treatments by reviewing 202 trials comparing placebo treatment with no treatment covering 60 healthcare problems. In general, placebo treatments produced no major health benefits, although on average they had a modest effect on outcomes reported by patients, such as pain. ” (https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003974.pub3/full). If I remember right, they attribute the positive outcomes more to regression to the mean and self-limiting diseases than Wainer’s to proposed mechanism.
However, my paper on amazon reviews of medical products (which skew v positive relative to RCTs) argues for similar mechanism to what Wainer proposes: people with good outcomes persist with the treatment and go on to write reviews (https://www.sciencedirect.com/science/article/pii/S0277953617300400).
Checking for bias in the dropouts is standard in well-designed clinical trials. Search for “censoring at random”.
Like others said, regression to the mean. Most people only seek out healthcare when their problem becomes exceptionally bad. Then doing almost anything can seem like a miracle cure.
Apparently this is a huge issue with allergy drugs.* The placebos are ~95% as effective as the drugs, and the placebos are *very* effective. The researchers only care about “significance”, the patients feel better, and some company gets paid so everyone is happy.
* This was discussed on this blog before but a quick search didn’t work. If anyone cares I’ll go find it.
Ditto Dieter. Also, it would be easy to study the placebo effect with 0 attrition. While fidelity is difficult to achieve for most useful interventions, it’s easy to achieve for fake ones. Given that there have been studies focused strictly on placebos, I have to believe at least some would use the latter. Just give a one-time sugar pill as treatment. Choose subjects entering the hospital. Measure outcomes when patients leave. Easy.
Some medical studies have three arms (treatment, placebo, nothing); perhaps all of them should, I dunno. Anyway you can compare the ‘no treatment’ group to the placebo group to quantify the placebo effect, and compare the treatment effect to the placebo effect to quantify the efficacy of the treatment.
Yes. The Placebo Effect in medical research is a multi-headed hydra. Some parts are biological, some are psychological and some are statistical. You have regression to the mean, the natural course of diseases, bias in responses from patients (they want to say and believe they improved), changes in other aspects of their lives due to being a subject in a medical trial (more monitoring, for example), differential dropping out.
Here’s a handy article summarising a lot of what the placebo effect actually is: https://sciencebasedmedicine.org/placebo-myths-debunked/
That’s a great essay, thanks Elio. Very concise. I really liked this summary:
“Placebo effects break down into several categories. One category is illusory – the misperception of improvement through regression to the mean or biased reporting. The second category is non-specific effects, such as emotional comfort from a practitioner, relaxation, or improved self-care or compliance. This third category is comprised of effects which can plausibly result from psychological interventions only. These relate mainly to stress, depression, anxiety, and the perception of pain and similar subjective symptoms. There is a mind-body connection – it’s called the brain.”
-Steven Novella from Elio’s link
I am half way through the causal quartets paper – and I’m so excited this is written down. Have been frustrated with the widespread notion that a linear regression model without interaction effects is somehow sufficient in observational studies to adjust for biases. “We realize that ‘the treatment effect’ in any given study will depend entirely on the patient mix”. Bingo! What patient mix is assumed when these models estimate the “average” treatment effect? More often than not, it’s not what the modeler believes. And if the modeler wants to use a patient mix that isn’t what’s found in the observational data, that needs to be made explicit, and supported by evidence.