Colonoscopy corner: Misleading reporting of intent-to-treat analysis

Dale Lehman writes:

I’m probably the 100th person that has sent this to you: here is the NEJM editorial and here is the study.

The underlying issue, which has been a concern of mine for some time now, is the usual practice of basing analysis on “intention to treat” rather than on “treatment per protocol.” In the present case, randomized assignment into groups “invited” to have a colonoscopy and those not, resulted in a low percentage actually following the advice. Based on intention to treat, the benefits of colonoscopy appear to be small (or potentially none). Based on those actually receiving the colonoscopy, the effectiveness appears quite large. While the editorial accurately describes the results, it seems far less clear than it could/should be. The other reasons why this latest study may differ from prior ones are valid (effectiveness of the physicians, long-term follow up, etc.) but pale in importance with the obvious conclusion that when adherence is low, effectiveness is thwarted. As the editorial states “screening can be effective only if it is performed.” I think that should be the headline and that is what the media reports should have focused on. Instead, the message is mixed at best – leading some headlines to suggest that the new study raises questions about whether or not colonoscopies are effective (or cost-effective).

The correct story does come though if you read all the stories but I think the message is far more ambiguous than it should be. Intention to treat is supposed to reflect real world practice whereas treatment per protocol is more of a best-case analysis. But when the difference (and the adherence rate here, less than 50%) is so low, then the most glaring result of this study should be that increasing adherence is of primary importance (in my opinion). Instead, there is a mixed message. I don’t even think the difference can be ascribed to the difference in audiences. Intention to treat may be appropriate for public health practitioners whereas the treatment per protocol might be viewed as appropriate for individual patients. However, in this case it would seem relatively costless to invite everyone in the target group to have a colonoscopy, even if less than half will do so. Actually, I think the results indicate that much more should be done to improve adherence, but at a minimum I see little justification for not inviting everyone in the target group to get a colonoscopy. I don’t see how this study casts much doubt on those conclusions, yet the NEJM and the media seem intent on mixing the message.

In fact, Dale was not the 100th person who had sent this to me or even the 10th person. He was the only one, and I had not heard about this story. I’m actually not sure how I would’ve heard about it . . .

Anyway, I quickly looked at everything and I agree completely with Dale’s point. For example, the editorial says:

In the intention-to-screen analysis, colonoscopy was found to reduce the risk of colorectal cancer over a period of 10 years by 18% (risk ratio, 0.82; 95% confidence interval [CI], 0.70 to 0.93). However, the reduction in the risk of death from colorectal cancer was not sig- nificant (risk ratio, 0.90; 95% CI, 0.64 to 1.16).

I added the boldface above. What it should say there is not “colonoscopy” but “encouragement to colonoscopy.” Just two words, but a big deal. There’s nothing wrong with an intent-to-treat analysis, but then let’s be clear: it’s measuring the intent to treat, not the treatment itself.

P.S. Relatedly, I received this email from Gerald Weinstein:

Abuse of the Intention to Treat Principle in RCTs has led to some serious errors in interpreting such studies. The most absurd, and possibly deadly example is a recent colonoscopy study which was widely reported as “Screening Procedure Fails to Prevent Colon Cancer Deaths in a Gold-standard Study,” despite the fact that only 42% of the colonoscopy group actually underwent the procedure. My concern is far too many people will interpret this study as meaning “colonoscopy doesn’t work.”

It seems some things don’t change, as I had addressed this issue in a paper written with your colleague, Bruce Levin, in 1985 (Weinstein GS and Levin B: The coronary artery surgery study (CASS): a critical appraisal. J. Thorac. Cardiovasc. Surg. 1985;90:541-548). I am a retired cardiac surgeon who has had to deal with similar misguided studies during my long career.

The recent NEJM article “Effect of Colonoscopy Screening on Risks of Colorectal Cancer and Related Death” showed only an 18% reduction in death in the colonoscopy group which was not statistically significant and was widely publicized in the popular media with headlines such as “Screening Procedure Fails to Prevent Colon Cancer Deaths in Large Study.”

In fact, the majority of people in the study group did not undergo colonoscopy, but were only *invited* to do so, with only 42% participating. How can colonoscopy possibly prevent cancer in those who don’t under go it? Publishing such a study is deeply misguided and may discourage colonoscopy, with tragic results.

Consider this: If someone wanted to study attending a wedding as a superspreader event, but included in the denominator all those who were invited, rather than those who attended, the results would be rendered meaningless by so diluting the case incidence as to lead to the wrong conclusion.

My purpose here is not merely to bash this study, but to point out difficulties with the “Intention to Treat” principle, which has long been a problem with randomized controlled studies (RCTs). The usefulness of RCTs lies in the logic of comparing two groups, alike in every way *except* for the treatment under study, so any differences in outcome may be imputed to the treatment. Any violation of this design can invalidate the study, but too often, such studies are assumed to be valid because they have the structure of an RCT.

There are several ways a clinical study can depart from RCT design: patients in the treatment group may not actually undergo the treatment (as in the colonoscopy study) or patients in the control group may cross over into the treatment group, yet still be counted as controls, as happened in the Coronary Artery Surgery Study (CASS) of the 1980s. Some investigators refuse to accept the problematic effects of such crossover and insist they are studying a “policy” of treatment, rather than the treatment itself. This concept, followed to its logical (illogical?) conclusion, leads to highly misleading trials, like the colonoscopy study.

P.P.S. I had a colonoscopy a couple years ago and it was no big deal, not much of an inconvenience at all.

39 thoughts on “Colonoscopy corner: Misleading reporting of intent-to-treat analysis

  1. First point, NEJM is not necessarily the best place to look for sensible interpretation of RCTs.

    This issue is a familiar one to statisticians involved in trials – the critical point (as Dale and Andrew say) is that in the intention to treat analysis the intervention is the invitation not the actual screening. Which often gets lost, as in the editorial. Masks for covid-19 are another recent example.

    There’s a general move away from ITT and per-protocol analyses at the moment towards more accurate and explicit specification of the estimand of interest in the trial – which may require more complex methods to estimate it. ITT has hitherto usually been regarded as the default and does at least have the benefit of simplicity. Naive per-protocol analysis is likely to be very biased, so if that’s the thing the trial is really interested in then it is going to need to also work out what methods are needed to get at it.

    • In the intention-to-screen analysis, colonoscopy was found to reduce the risk of colorectal cancer over a period of 10 years by 18%

      Maybe someone who read it more carefully can answer. Shouldn’t the screening increase the chance you are diagnosed?

      I’m looking at figure 2 and there is a 2x higher rate of cancer in the treatment group within a year after randomization: https://www.nejm.org/na101/home/literatum/publisher/mms/journals/content/nejm/2022/nejm_2022.387.issue-17/nejmoa2208375/20221102/images/img_xlarge/nejmoa2208375_f2.jpeg

      Are those cancers that were caught at the screening?

      • I can’t figure out a way to get this outcome to make sense.

        If they are including cancers detected at the screening, then whatever protective effect is being diluted with cancers the intervention was supposed to find. And looking at the supplements, its huge. Like 70% of the cancers in the screened group were found within 2 years.

        If they aren’t including those cancers, then why is there no discussion of that huge spike right after the screening?

        P.S. This was meant to be its own comment thread, sorry Simon.

      • As far as I understand one thing that’s done in colonoscopy is during the exam, if they find pre-cancerous growths they remove them so in theory this prevents cancer from forming. Perhaps that’s part of the issue? Or perhaps I’m wrong in my understanding.

        • I think that Daniel’s point is part of the issue, but, based on my experience with colonoscopies, it’s not the whole issue. For example, I had a colonoscopy a few months ago. The reason I had the colonoscopy was that I was having sharp pains in my lower left abdomen. When the pains first started, I mentioned them to a friend who then took me to a new urgent care facility. The PA working there did what seemed like a careful examination, then told me that the problem was that I had pulled my groin. She then taught me some exercises that were intended to help heal a pulled groin. Things improved for a little while, but then started getting worse again, so I decided to go to a gastroenterologist. It took a while to get an appointment (the one who had done a colonoscopy on me before had retired), but when I got one, the physicians involved came up with convincing reasoning that I needed a colonoscopy (I had had one several years before, and they made the case to the insurance company that it would be prudent to have another colonoscopy to see whether or not the diverticulosis diagnosed at the previous colonoscopy had progressed to the more severe diverticulitis (which, in particular meant my health insurance would cover the procedure.)
          The colonoscopy went well, and the team of medical personnel involved discussed what “next steps” needed to be taken. In particular, there was a PA on the team (not the one from the urgent care facility) who said that the symptoms of diverticular disease had considerable overlap with the symptoms of being a post-menopausal woman (I am in my late 70’s), so the PA urged me to to find a gynecologist with whom to have regular appointments to try to tease out what was gynecological from what was gastroenterological.

      • Removing precancerous polyps reduces risk of cancer. It’s not clear from the article whether the increase at year 1 on this graph represents the 62 who were diagnosed with cancer at initial screening, particularly as there was some (unspecified) time lag between randomization and screening–that seems likely given that the ratio is forced to 1 at time 0.

        • Sure, but how is a screening supposed to prevent a cancer that was discovered during the screen?

          It can’t, so that is a poor measure of effectiveness. Leaving those pre-existing tumors out also wouldn’t work since presumably those exist in the control group too, they are just discovered more gradually.

          Attempting to judge the success of a screeing by the rate of disease just makes no sense.Use the grade of cancer and mortality instead.

        • I think you’re failing to make the distinction between pre-cancerous tumors and cancerous tumors (though I confess I don’t really understand how they make this distinction). If you discover a pre-cancerous tumor, remove it, and it never becomes cancerous, then you have prevented a cancer.

        • No I get it, the screening prevents cancer because the patient has pre-cancerous lesions removed.

          But it also detects cancer that already existed at the time of screening, which is a “good thing”.

      • CRC screening has the following dynamic compared to no CRC screening:

        Increase in CRC incidence in the short-term (asymptomatic CRC has an average dwell time of ~3-5 years prior to the patient becoming symptomatic) via detecting prevalent CRCs at the time of the screen event. The CRCs that would have been detected over a period of years, via signs and symptoms, are detected/diagnosed up front, asymptomatically, at the screen event.

        Corresponding decrease in CRC incidence after the screen event (there aren’t other CRCs to detect–95% of them are detected via colonoscopy).

        Decrease of *future* CRC incidence via the removal of lesions that can develop into colorectal cancer. The progression of a precancerous lesion (called an adenoma) to preclinical cancer takes, on average, about 20-25 years. The most dangerous of these types of lesions, called advanced adenomas, have been estimated to have ~5% probability of transitioning to asymptomatic CRC per year.

    • This would seem to be typical of Frequentist viewpoint problems. Instead of having a well defined generative model, and then parameters which have well defined meanings within that model, you have “estimators” which are only estimators of a particular thing conditional on the data being collected in a certain way. So for example, the observed difference in cancer incidence is an estimator of the causal change in cancer incidence **of the invitation** but it is NOT an estimator of the causal effect of **the actual colonoscopy**.

      If on the other hand you have a model where the invitation has some probability to be accepted based on perhaps the person’s health status and other things, and then the procedure itself has some probability to be preventative of future problems (ie. by removal of early polyps or whatever) then you can report something that people more reasonably care about, and include things like factors that might make people more likely to accept the invitation.

      Randomized Controlled Trials are not an invitation to ignore modeling, they’re a technique to make models more closely aligned with reality.

  2. There are now, in the last probably 5 to 10 years, a decent number of instrumental variable methods for survival data with right censoring, made for situations like this where we trust neither per protocol nor itt analyses. The downsides are some more convoluted language (treatment effects among “compliers”) and the methods are complex in some cases. There needs to be work comparing the IV methods, explaining them, and making them more easily accessible in easy to use software packages.

  3. 1) Any screening test could increase the chance of being diagnosed in the short run. This is what appears in the left side of figure 2.
    2) Colonoscopy allows to remove, not only some cancers, but also some benign polyps, which could get malign later. Therefore, colonoscopy is expected to reduce the incidence of colorectal cancer in the middle-long run.
    3) Intention-to-treat analysis is intended to preserve comparability between both groups. This does not happen in per-protocol analysis. In this study, the amount of people having no colonoscopy in the intervention group makes per-protocol analysis unreliable.
    4) As pointed-out by David and Andrew, intention-to-treat in this study does not refer to “colonoscopy”, but to “invitation to colonoscopy” instead. (I really love the example of the wedding. I’m keeping it for my own teaching.)
    5) (3)+(4) indicate that neither per-protocol nor ITT analysis in this study could answer the question: is colonoscopy useful as screening test?
    6) Then, why the New England published it? We don’t know if colonoscopy failed. We know that the study failed.

  4. Perhaps we should distinguish the never started from dropouts. In my field including the dropouts in an ITT analysis is important because treatments that are too rigorous for sizable portions of the population will look too good if we only analyze the heroic patients capable of withstanding them. The analysis of a study would be a bit more complex if we presented controls, ITT, never started, and dropped out data. However, real life is complex. Ars longa, vita brevis est as they say.

    • +1 I have read only a few of these complaints about ITT, all of which conveniently buries the problem with PP analysis, which is that drop out is biased. By subsetting to those who followed protocol, the randomization of the treatment assignment no longer holds. Then this should not be described as an RCT but an observational study. Doing PP analysis and calling it RCT is the problem. If they call it an observational study, I have no beef.

      Also, I highly doubt this controversy would even erupt if the ITT analysis agreed with the preconceived notion that this screening is beneficial.

      The situation is not one-sided as is portrayed by your correspondents. Here is an example from my own work – consider testing whether sending a marketing email to a randomly selected subset of customers to drive sales. In a per-protocol analysis, the marketing team argues that if someone did not open the email, then the person could never benefit from the marketing offer. So if we do PP, then we are comparing the people who opened emails (a fraction of the total) on the treatment arm to everyone who could have received an email but didn’t on the control arm. Do we really think that is a fair comparison? Running this test using PP is a waste of time and resources because people who open marketing emails are clearly better customers and will have higher pre-marketing average sales.
      Doing ITT is appropriate because in order for the email to be successful, it needs to first be opened then be acted on. Not measuring the former is a huge miss.

      • This is not a good example. The study clearly shows that people that underwent the screening benefited (on average). The punchline, which was hidden in the editorial when it should have been highlighted, is that screening works if it is done. Your marketing example does somewhat match what I would call the public health perspective – if many people won’t abide by the “invitation” to undergo screening, then that certainly limits the benefits of screening. However, from a patient’s point of view, I want to know how likely screening is to benefit me, not based on the fact that many people will choose not to follow the invitation.

        Actually, I (as a patient) would prefer the as-treated analysis rather than ITT or per protocol. However, any of these options begs the question of why people randomized into the invitation to undergo screening group choose not to undergo screening. If it reveals something unmeasured about their actual health status, then the ITT is more accurate than either of the other options. If it simply reflects the fact that screening is unpleasant, then the ITT is quite misleading. And, in this case, over 50% of the invitation to screening group did not undergo screening.

        I don’t think we can generalize and say that any one of the analysis options is always the best. Adherence is always an issue and there are a variety of reasons people don’t adhere to the treatment plan: personal preference, side effects, and unmeasured health conditions (in your marketing example, not opening the email is clearly relevant to the likely success of the campaign – not so clear for he colonoscopy case). I think it matters which of these affects adherence. I’d prefer to see all three analyses done and results presented – or the data provided so that anyone can perform those analyses.

        • We agree on the last point. But do you agree that anything that is analyzed per protocol does not have randomized treatment assignment? If you randomize and then exclude people based on events subsequent to treatment, it’s no longer a randomized trial, more like an observational study with selection biases. You can’t have your cake and eat it too.

          btw, a fraction of email recipients open any emails. The level of “non compliance” is much higher than 50% if you want to call it that.

        • I agree that once adherence is taken into account, it is closer to an observational study than a RCT. In fact, I tend to think of a continuum from RCT to observational studies- unless all confounders can be accounted for and/or the randomized groups are identical in all aspects, there is some degree of selection present in all RCTs.

          As for the low “adherence” to opening emails, I can attest to that fact based on how many emails I send that never get responded to. But I think the case of a relevant selection bias with unanswered emails is somewhat stronger than failure to respond to an “invitation” to have a screening colonoscopy.

        • not sure if this is going to show up below your latest reply. Agree on both counts.
          I gave the email example to show that it’s not a cut and dried issue.
          In the invitation to screening example, how do you remove those people from the holdout group who would have non-complied if they were invited to screen?

  5. I’m happy to see this issue raised here. No doubt there’s a literature on the pros and cons of ITT, and I haven’t looked into it, but it has always bothered me.

    1. If the purpose of the study is to assess a therapy/diagnostic procedure/etc., then test it directly. If the purpose is to assess a method of disseminating/encouraging/prescribing it, then test that. But why would you fold the two tests together, at least before you had evaluated them separately?

    2. I’ve seen ITT defended as a way to preserve the virtues of random assignment, and it’s true that nonadherence will be selected. That’s certainly a problem, but isn’t it a fait accompli when the data have been collected? In fact, by including nonadopters in ITT analysis, aren’t you also incorporating the selection as well as the dilution of whatever effect you’re trying to identify? In other words, the selection, if real, can’t be undone by analysis, so why not study the de facto arms themselves?

    I’m a bit hesitant to post this, because all of this must have been hashed out already, but maybe there are others out there whose response is similar to mine.

  6. > In the intention-to-screen analysis, colonoscopy […] What it should say there is not “colonoscopy” but “encouragement to colonoscopy.” Just two words, but a big deal.

    To be fair, it says three-words-hyphenated-into-one to that effect just before “colonoscopy”.

    > There’s nothing wrong with an intent-to-treat analysis, but then let’s be clear: it’s measuring the intent to treat, not the treatment itself.

    On the other hand, a per-protocol analysis wouldn’t be measuring the treatment itself either.

    • By the way, I had forgotten (the publication delay is getting out of hand) that both the article and the editorial also discuss “adjusted analyses to estimate the effect of screening if all the participants who were randomly assigned to screening had actually undergone screening”.

  7. It is well not to confuse these two questions:
    1. As a public health intervention, is inviting people to have a screening colonoscopy effective in preventing colorectal cancer?
    2. As an individual (say, over 50) will a screening colonoscopy help prevent me from getting colorectal cancer?
    This study addresses the first question.
    The second question requires a different study design. For instance, one could recruit a group of people who have already agreed to have a screening colonoscopy and randomise half to have “sham” colonoscopy. Although the screening will occasionally detect a pre-existing occult cancer, the main effect is intended to be through the removal of any benign polyps (adenomas) preventing their future malignant transformation into cancer. The sham procedure would be not to remove these, and the trial would then compare subsequent cancer incidence and mortality in the sham and real colonoscopy groups. The hypothesis that many colorectal cancers arise in benign polyps is yet to be proven so I imagine this study might pass ethical review.
    This sham procedure approach has been used to study other treatments, for example knee arthroscopy for meniscal problems.

    • Although the screening will occasionally detect a pre-existing occult cancer, the main effect is intended to be through the removal of any benign polyps (adenomas) preventing their future malignant transformation into cancer.

      It wasn’t occasional though. It looks (from per-protocol in the supplement) that cumulatively 0.5% had cancers detected during screening out of a total of 0.84% after ten years.

      In the unscreened group that was 1.2% after ten years. Presumably ~0.5% already existed before the screening, leaving new cancer forming at a rate of 0.7% per ten years.

      This matters, because 0.34% vs 0.7% appears much more effective than 0.84% vs 1.2%.

  8. In a letter to the Editor, available here

    http://www.medicine.mcgill.ca/epidemiology/hanley/screening/LettersColonoscopyScreeningColorectalCancerIncidenceMortalityNEJM.pdf

    I suggested that the journal needs to ensure that the Abstract gives the FULL story. In this case, it did not. And so, many of the science journalists, who don’t read past the Abstract, only reported the ITT results, and wrote articles suggesting that colonoscopy screening in the US isn’t all it is advertised to be.

    The authors avoided answering my point, and went sideways instead. But I think it is the Editors and referees who let us down: they need to anticipate what is made of their reports. Many lay people read (and are guided by) these lay-media digests. ITT-only results don’t speak to individuals trying to decide what to do. Many busy doctors just read the abstract.

    This reporting of ‘silly averages’ reminds me of an 1889 comment by an English statistician: “It is difficult to understand why statisticians commonly limit their inquiries to Averages, and do not revel in more comprehensive views. Their souls seem as dull to the charm of variety as that of the native of one of our flat English counties, whose retrospect of Switzerland was that, if its mountains could be thrown into its lakes, two nuisances would be got rid of at once.”

  9. This study should not be called a ‘clinical trial’ but a ‘public health intervention study’. The doctors don’t realize that not all research is a clinical trial so they misuse terms like ITT. The participants are exposed to information, not given a regimen. Choosing to do the colonoscopy makes the patient a ‘human subject’ as far as medical consent but they are not in an experimental trial. Perhaps that is why they use the terminology.

    I do something else now but for a long time I worked in clinical trials and consumer research. About 10 years ago I was approached about doing a trial for an exercise app. I spoke with a trial design/regulatory expert I knew. She said that if the participants just used the app whenever, it was consumer research. However as soon as a regimen is included, such as exercise once a day, it becomes a trial and requires all the human subject compliance. The regimen can be flexible such as exercise at least 3 times a week or take a pill for pain when a headache starts.

    There are many reasons trial subjects in any treatment groups are switched from per protocol to intent-to-treat. Non-compliance is only one–the patient might have a change in medical condition or personal situation. The statistical analysis plan is described in the protocol. Intent-to-treat classification is used for descriptive, frequentist and Bayesian analysis.

  10. Before we get too positive about the benefits of a colonoscopy, from the Mayo Clinic

    https://www.mayoclinic.org/diseases-conditions/colon-cancer/expert-answers/colon-cancer-screening/FAQ-20057826#:~:text=The%20upper%20age%20limit%20was%20set%20after%20studies,a%20previously%20diagnosed%20colon%20cancer%20or%20adenomatous%20polyps.

    “There’s no upper age limit for colon cancer screening. But most medical organizations in the United States agree that the benefits of screening decline after age 75 for most people and there’s little evidence to support continuing screening after age 85.”

    In fact, I once attended a lecture about the possible horrors of colonoscopy, which I was reminded of, is, after all and unlike some of the newest and inexpensive methods of detection, an invasive procedure. And, I bet most of the bloggers here recall this famous incident
    ——————————————————————————————————————-
    https://www.cbsnews.com/news/patient-sues-anesthesiologist-who-mocked-him-while-sedated/

    A Virginia man got an unpleasant surprise when he played back a recording taken with his smartphone to capture the instructions his doctor would give him after a colonoscopy.

    On his way home from the procedure, he hit “play” and was horrified to hear his surgical team mocking and insulting him while he was unconscious, the Washington Post reported. Portions of the recording were posted on the paper’s website.

    In the recording, made in 2013, the doctors can be heard making fun of a rash on the man’s genitals, instructing a medical assistant to lie to him, and joking about giving him a false diagnosis.

    “After five minutes of talking to you in pre-op, I wanted to punch you in the face and man you up a little bit,” the anesthesiologist, Tiffany M. Ingham, was recorded saying.
    ——————————————————————————————————————–
    Need I add that I am over 85 and therefore, possibly prejudiced?

    • Is it fair to suggest that colonoscopy screening may not be worth it if you still use it as a diagnostic tool? One of the letters to the editor available in the link above doesn’t think so:

      “An even more important reason for the lower-than-expected effectiveness of screening colonoscopy in the trial that was not addressed by the authors is that the use of diagnostic colonoscopies during the follow-up period was common. The protection provided with diagnostic colonoscopies through the detection and removal of precursors of colorectal cancer is similar to that with screening colonoscopy. It is highly likely that during follow-up, such colonoscopies were more often performed in the participants in the usual-care group, who had no offer of screening colonoscopy at baseline, than in those in the invited group, thereby leading eventually to a lower colorectal cancer incidence and mortality in the usual-care group. The estimated effectiveness of screening colonoscopy in this trial could thus be diluted, lower than expected, and lower than what existing evidence of the effects of screening colonoscopy would suggest.”

      (I’m not an oncologist but I don’t think that the correct way to evaluate the efficacy of screening is to just let people die untreated if they didn’t get screened.)

      • “Diagnostic” colonoscopy is the medical coding term applied to colonoscopies performed if an individual is presenting concerning symptoms (or as the follow-up test to a positive non-invasive CRC screen).

        This includes diagnosing symptomatic CRC (for individuals that choose not to get screened). There are many scenarios that could occur: individuals in usual-care could have had bloody bowel movements, been concerned, had a diagnostic colonoscopy performed, and found to have an advanced precancerous lesion (these bleed), had it removed, and subsequently had a CRC prevented. However, it’s also possible the bleeding could have been caused by a CRC.

        It’s better to use CRC as a screening tool and not just as a diagnostic tool for signs and symptoms. Waiting until signs and symptoms present is akin to playing Russian roulette–the symptoms can be caused by any number of issues, from bad hemorrhoids, to inflammatory bowel disease, to precancerous lesions, to CRC.

        • > It’s better to use CRC as a screening tool and not just as a diagnostic tool for signs and symptoms.

          Maybe it is or maybe it isn’t. But it seems to me that saying that a comparison of screening to a standard of care is unfair because the standard of care can produce similar benefits makes little sense and completely misses the point of the study.

      • The probability of diagnosis is going to be the probability of having cancer times the probability it is detected.

        So right away we can tell they should have monitored how often patients in each group underwent procedures that could lead to a diagnosis.

        Eg, does being in the screened group reduce the rate of getting a later diagnostic colonoscopy?

        If they had tried to model the timeseries, it would quickly be found those two parameters are degenerate so they need to collect that data.

        • I don’t think the endpoint of “does being in the screened group reduce the rate of getting a later diagnostic colonoscopy” is a relevant endpoint for the trial. There are many reasons to get a diagnostic colonoscopy, above and being having colorectal cancer. For example: hemorrhoids, chronic diarrhea, new onset Crohn’s disese, inflammatory bowel disease, infection, an intestinal vascular disorder, gallstones, perianal venous thrombosis, among others.

          Quite a few of the investigators on the trial are excellent biostatisticians and cancer models, including Ann Zauber.

  11. I was unaware that intent to treat was used widely in studies. It seems obvious that you would want to measure actual treatment as the important variable. It’s also an odd mistake as usually, researchers are looking for positive correlations to find improved treatments and intent to treat as the variable makes that a lot less likely.

    Just how common is intent to treat in the literature?

    • > It seems obvious that you would want to measure actual treatment as the important variable.

      What is not so obvious is how to measure that. In particular, ignoring the fact that some of those who you intended to treat were not treated and simply comparing those who were actually treated with the control group would be a mistake.

      > It’s also an odd mistake as usually, researchers are looking for positive correlations to find improved treatments and intent to treat as the variable makes that a lot less likely.

      Making a lot less likely that researchers looking for positive correlations find meaningless positive correlations can be a desirable thing.

    • Just to echo Carlos, at least for RCT’s in economics, this is typically the primary estimate, i.e. the treatment effect on those assigned to treatment, and as far as I know it’s the only unbiased estimate an RCT gives you without addition (and often strong) assumptions.

Leave a Reply to David Marcus Cancel reply

Your email address will not be published. Required fields are marked *