The Supreme Court meets the fallacy of the one-sided bet

Doug Hartmann writes (link from Jay Livingston):

Justice Antonin Scalia’s comment in the Supreme Court hearings on the U.S. law defining marriage that “there’s considerable disagreement among sociologists as to what the consequences of raising a child in a single-sex family, whether that is harmful to the child or not.”

Hartman argues that Scalia is factually incorrect—there is not actually “considerable disagreement among sociologists” on this issue—and quotes a recent report from the American Sociological Association to this effect. Assuming there’s no other considerable group of sociologists (Hartman knows of only one small group) arguing otherwise, it seems that Hartman has a point. Scalia would’ve been better off omitting the phrase “among sociologists”—then he’d have been on safe ground, because you can always find somebody to take a position on the issue. Jerry Falwell’s no longer around but there’s a lot more where he came from. Even among scientists, there’s a large enough minority with traditional moral values, that it shouldn’t be hard to find some who feel strongly that it’s harmful to raise a child in a single-sex family.

But what I really want to talk about here is not scientific consensus (which, after all, can be wrong) but a little-noticed (as far as I can tell) aspect of Scalia’s statement, which is that it’s an example of the fallacy of the one-sided bet, an argument that is artificially restricted to go in just one direction. Scalia’s saying that raising a child in a single-sex family might be “harmful to the child” or maybe not. But by framing it this way, he’s implicitly excluding a third possibility, which is that being raised in this way may be helpful to the child. What do I really think is happening? I think that the same-sex-parents environment will be helpful to some kids, harmful to others. Assuming the American Sociology Association report is correct and the research doesn’t find any aggregate effect, that would suggest that some kids are helped and some are hurt, with no clear evidence that the average effect is positive or negative. So, from the data, the effect could be positive, or it could be negative, or it could be small enough on average to consider as zero. And, just from prior reasoning, I could imagine an effect that is positive (gay parents try harder and they could be less likely to have unwanted children) or negative (maybe it’s better to have parents of both sexes and not to have to face prejudice from outsiders). I don’t know, apparently the data don’t know either. But by framing his statement the way he does, Scalia is excluding the possibility entirely.

In all this discussion I’m sidestepping the causal questions, how one might consider formulating hypothetical interventions, whether one would want to consider the “treatment” at the level of individual families or state-level policies, and the difficulties of statistical identification. These issues are important—indeed, central to any discussion of this issue—but here I want to focus on the one-sided argument, which is such a pervasive fallacy. I keep hoping that, by giving this error a name, I can reduce its incidence.

51 thoughts on “The Supreme Court meets the fallacy of the one-sided bet

  1. If the coinage is new enough for modification, I wonder if you can refine it to distinguish it from another one-sided aspect of his quote. Harmful compared to what? Compared to being the child of two unmarried people of the same sex? Compared to the child of an opposite sex marriage of convenience because SSM is illegal? Compared to a single-parent household? Perhaps marriage equality is terrible for the children, but less terrible than the alternatives. No doubt there is already a name for this type of one-sidedness, but I don’t have the time to search for it.

    • RJB: This is often where I find counterfactual reasoning to be extremely helpful. In a lot of my work it simply isn’t (levels of inflammatory markers are connected to a person in all sorts of complex ways). But for child rearing it is worth specifying what the reference level is. Even more interesting, the counterfactual, itself, has all sorts of quirks (does a opposite sex marriage end in divorce more or less often over the next 20 years?) that make it hard to specify.

      But this is usually the question that I like to ask. Because there are children today in foster care, with single parents, and so forth and you really have to ask (for kid X) what would the alternative have been?

    • In context, it’s pretty obvious that Scalia is wondering about raising the child by a gay couple vs. a traditional couple. This is what a mathematician would call a “partial derivative”: a change in X when one of the inputs varies and all others are artificially held constant (even if, in practice, other inputs are functions of this input.)

      The Supreme Court is ultimately interested in the net effect of gay marriage on American children, the “total derivative”. Figuring out the total derivative is a complicated task because legitimization of gay marriage changes both the family structure and the number of children in the country. It is also more debatable. For example, one of the major conservative arguments against gay marriage is that legitimization of gay marriage “devalues” traditional marriage and makes heterosexual couples less likely to marry. In addition, it increases the chance that bisexuals end up as gay couples rather than traditional families.

      However, in Scalia’s mind, that is a remote question. You can’t determine whether the total derivative is positive or negative if there’s still disagreement about the partial derivative. And Scalia is not required to decide whether gay marriage is good or bad. The situation (again, in Scalia’s mind) is that defenders of gay marriage must provide proof that DOMA is not just bad, but simply irrational, because it would be obvious to any sane person that DOMA serves no legitimate government interest. A proof that gay marriage is, on the net, neutral or beneficial to society would be a necessary but not sufficient condition. That is obviously a high barrier and (as far as he knows) sociologists don’t even have a consensus on one of the basic questions – “as to what the consequences of raising a child in a single-sex family”, which precludes them and, consequently, any sane person from concluding that DOMA is irrational.

      Scalia is not in any hurry to overturn DOMA or to declare that gay marriage is a 14th amendment protected right. He is treating this case by the book, and the book is on the side of the government. 20 years from now (when Scalia will most likely be dead) the court may revise the issue and all the sociological data accumulated in the mean time, and maybe by that time every sane person would know that gay marriage is OK, and then DOMA would be overturned.

  2. You only have to ask yourself the question: if not harmful then what? Obviously, being helpful to the child is a member of the set “or not”. You can claim that Scalia “implicitly excluded a third possibility”, but it is only your opinion.

  3. Do typical studies on stuff like smoking also include this third possibility of the habit doing a net good to the smoker’s health?

    One could argue that the smoker knows his enhanced risk but can’t stop smoking so does less of other harmful things to compensate?

    • Well, there are all sorts of things that smokers are less likely to die of, e.g., Alzheimer’s disease. Maybe that’s a benefit of smoking.

      • Have to be careful with this thinking – could simply be a competing risk scenario (e.g., die of lung cancer before they have time to get Alzheimer’s).

    • @Rahul: Epidemiologists often look at all-cause mortality for this precise reason. It is pretty clear that smoking may increase cancer mortality and CVD mortality, but it also preotects against some neurological diseases (I think Parkinson’s disease is the least controversial). Looking at all-cause mortality includes the possibility that the exposure has both positive and negative effects.

      This is a lot clearer with anti-coagulants. You dramatically reduce the risk of clotting events (to an extremely low level) but greatly increase the risk of a major bleeding episode. There is a lot of work done trying to decide which sub-populations have the potential to benefit and where the riks may outweigh the benefit.

  4. the tendentiousness over this claim is astonishing. it’s a s if the Supreme Court had only been presented with one amicus brief from sociologists. they got another one that came to the opposite conclusion. I have no idea which of the two briefs had the better argument, and Scalia doesn’t either, which amply supports his finding. Granted, it might have been helpful to have someoen with some expertise help him arbitrate this expert battle, but he punted honestly.

    • Jonathan:

      As noted above, Hartman does recognize there was an academic response on the other side. But rather than representing the entire American Sociological Association, this other brief has seven authors, of which two appear to be sociologists.

      But, in any case I agree with your implicit point that, just because an official American Sociological Association report says something, it doesn’t mean it’s correct, and Scalia could be legitimately unsure about these issues.

      My main point way that, the framing of the question as “whether that is harmful to the child or not” is implicitly biasing the issue, by excluding the possibility of benefit (or, at best, tossing all benefits into the “no harm” category).

      I doubt Scalia was doing this on purpose; I think the problem just got mistakenly framed in that way. I see this sort of thing happening all the time, which is why I gave the fallacy its own name.

      • I agree with your point in principle. But is the third option significant here? Might not there be cases where intuition leads one to discount an improbable option? Of course, one may be proved wrong but I’ll wait to see that data here.

        If one criticizes someone for implicitly ignoring an option one ought to have numbers to show that option did indeed matter in the accounting. Any study showing positive effects of single sex parents?

        • Rahul:

          In this case, my impression is that most of the research shows something like a zero net effect, which would imply a mixture of positive effects for some individuals and negative for others. To go beyond this (for example, to isolate subgroups of the population where effects are clearly positive or negative) would require a clearer definition of hypothetical treatments than I’ve seen here.

        • I guess I find this statement by Andrew confusing:

          “Assuming the American Sociology Association report is correct and the research doesn’t find any aggregate effect, that would suggest that some kids are helped and some are hurt, with no clear evidence that the average effect is positive or negative.”

          Is there a difference between the terms “aggregate effect” and “average effect”? If there is no aggregate effect shouldn’t the average effect be necessarily zero?

        • Rahul:

          I’m using “average” and “aggregate” interchangeably. The point is that an average of zero effect would correspond to a mixture of some positive and some negative, which would imply that if the effects were studied in detail, it would be possible to find positive effects in some contexts and negative effects in others.

        • From a public-policy perspective only the net effect should matter? Unless there was some way for a judge to predict which cases would swing which way. That seems a tad hard.

          e.g. say an anti-cancer drug improved survival in some patients and symmetrically hastened death in others. Unless I had a reliable way to predict individual outcomes that intervention seems useless.

      • “Nameless” above has it right. The Court is not supposed to decide whether homosexuals raising children is good or bad. That is for the legislative branch. The Court is supposed to say whether the view that it is bad for children is “arbitrary or capricious”. Thus, if they decided that there was a .1 probability that it was bad for children, .4 that it had no effect, and .5 that it helped children, they would uphold the law, because a 10% probability of harm is enough for a rational basis. The analogy is imperfect, but I hope it helps. They are supposed to rule whether the legislature could possibly have had a rational basis for passing the law.

        • But isn’t there a 0.1 probability that being raised at all by any parent is bad?! I see the point you are trying to make, but it doesn’t really work that way in practice.

        • “MR. COOPER: … The Plaintiffs’ expert acknowledged that redefining marriage will have real-world consequences, and that it is impossible for anyone to foresee the future accurately enough to know exactly what those real-world consequences would be.

          And among those real-world consequences, Your Honor, we would suggest are adverse consequences.
          But consider the California voter, in 2008, in the ballot booth, with the question before her whether or not this age-old bedrock social institution should be fundamentally redefined, and knowing that there’s no way that she or anyone else could possibly know what the long-term implications of — of profound redefinition of a bedrock social institution would be. That is reason enough, Your Honor, that would hardly be irrational for that voter to say, I believe that this experiment, which is now only fairly four years old, even in Massachusetts, the oldest State that is conducting it, to say, I think it better for California to hit the pause button and await additional information from the jurisdictions where this experiment is still maturing.


          JUSTICE SCALIA: (…)They’re arguing for a nationwide rule which applies to States other than California, that every State must allow marriage by same-sex couples. And so even though States that believe it is harmful — and I take no position on whether it’s harmful or not, but it is certainly true that — that there’s no scientific answer to that question at this point in time.

          MR. COOPER: And — and that, Your Honor, is the point I am trying to make, and it is the Respondents’ responsibility to prove, under rational basis review, not only that — that there clearly will be no harm, but that it’s beyond debate that there will be no harm.”

        • Nameless:

          I don’t like how Cooper is framing it either. He’s also placing it in the context of a one-sided bet, considering “harm” or “no harm” but not “benefit.”

          I’m speaking of course as a statistician, not as a lawyer.

        • Here’s a good analogy (I think). Suppose that voters in Texas pass a constitutional amendment that raises legal driving age from 16 to 21, on the basis that 16 to 20 year olds cause a disproportionate share of accidents. That would be a case of discrimination not unlike what we have here (discrimination by age rather than by sexual orientation.)

          In court, the defender of the amendment would argue that voters are rational because they can rationally imagine harm caused by letting 18-year-olds drive. It does not mean that he’s placing it in the context of a one-sided bet. He’s not looking at the issue as a whole and he’s not trying to debate whether letting 18-year-olds drive is good or bad for the country. He’s making a narrow observation that there is at least one specific harmful consequence.

    • Apologies, I hadn’t read the Hartmann link above when I made the previous comment, but Hatmann (and Cohen’s) response is laughable. The Regnerus brief is not based on one article. I might add that the Leon Kass brief in the same case pointed out that scientific findings by umbrella organizations are quite likely to be more politicized than scientific, and Scalia could have been reacting favorably to that point as well.

  5. I can’t believe I’m defending Scalia here, but I think you’re missing the fact that this is still a legal argument.
    In equal protection lawsuits, a law needs to pass a “rational basis” test to be constitutional. If gay marriage overall hurt children, that would constitute such a rational basis – i.e. legislators (or in this case the voters of California) would have an actual reason, rather than just prejudice, to discriminate against gays.
    This is a one-sided test: There is no constitutional obligation for lawmakers to pass optimal laws (alas) – e.g. even if gays turn out to be _better_ parents, that wouldn’t require laws giving them precedence over potential straight adoptive parents, so that possibility is irrelevant for the case before SCOTUS.

    • Thank you for putting this in better words than I could do. It’s not that Scalia is introducing some artificial restriction in his response. Rather, the very question (to which Scalia is responding) is restrictive: that children from homogomous unions fare no worse than those from heterogomous ones.

      The issue somewhat reminds me of the difference between a one-sided and two-sided t-test. Clearly, the two-sided test is more interesting (and more stringent). But if the null hypothesis reflects some sort of valence or direction, then the one-sided test is sufficient.

    • You are not entirely correct.

      Yes, generally speaking, in equal protection lawsuits, a law needs to pass a “rational basis” test to be constitutional. Even though some lower circuit judges (e.g. infamous Vaughn Walker, the closet gay judge that ruled against Prop 8 in California in 2011) did rule that anti gay marriage laws and constitutional amendments did not pass rational basis, at the Supreme Court level, pundits agree that, if the Court decides to apply rational basis, it will likely find that DOMA is constitutional. Rational basis is a very low bar and, in recent history, in cases where the Court applied rational basis review, it tended to find in favor of the government by a 4:1 margin.

      Just to illustrate this point, if, back in 1996, the Congress passed DOMA and stated explicitly that the primary purpose of this act was to provide a uniform definition of marriage in the situation where some states recognize gay marriages and others don’t, that would have been sufficient to pass rational basis review. (The Congress didn’t do that, they went into the woods by talking about “advancing the government’s interest in defending and nurturing the institution of traditional, heterosexual marriage”, “moral disapproval of heterosexuality” and so forth, which may yet prove the undoing of DOMA.)

      If the Court ends up overturning DOMA, it will probably do so by applying a higher level of review, the so-called “rational basis plus”.

  6. Interesting column in the New York Times today addresses the “evidence” that same-sex marriage would be bad for kids.

    Note in particular that the study that anti-marriage equality folks often cite, by Mark Regnerus at the University of Texas, is entirely bogus as evidence in this situation as it compares apples to oranges (as Regnerus admits). And to refute the fact that the American Academy of Pediatrics has said that marriage equality would not negatively affect kids, Ralph Reed said that the American College of Pediatricians says it would. But then the American College of Pediatricians is a self-selected, tiny group of physicians who broke away from the Academy over gay adoption a decade ago.

    Such is the quality of the evidence that Scalia seems to be relying on.

    • Bill, read the linked Regnerus (and others) brief. It’s brief, and doesn’t rely on the controversial part of Regnerus’ study in any way. As the first comment to the Chronicle article linked to by the NYT article points out “The lynch mob has come out early.”

      • The only brief I see is the one by the ASA, which states quite clearly that the Regnerus study is being misapplied by the opponents of marriage equality.

        The writer of the first comment has his opinion, but note that it engendered a lot of counter-comments, some of which state (I do not know this for a fact) that the writer of this first comment seems to have an agenda.

        • I linked to the other brief above. Believe me, I have no particular interest in this issue one way or another, but I have a pretty good sense for telling when people have a agenda getting in the way of their objectivity. Not that Regnerus doesn’t as well…. but the “error” in his study has little to do with the substance of the brief.

        • The fundamental error of Regnerus’ study is in (1) using a survey question that asked adult respondents whether their parent ever had a romantic relationship with someone of the same sex to measure being raised in a gay household, and (2) comparing respondents who said “yes” on this item to a reference group of adults raised in intact, two-parent, heterosexual households. There were better measures available in the survey, in the form of life history data of household structure, but — and I’m going on second-hand information here — they identified only 2 respondents who were actually raised in intact same-sex households since birth.

        • Yes, I get that, and I agree with it. but other studies have fundamental errors too, like drawing vast conclusions on tiny convenience samples. Now what?

  7. Andrew, your wording of the issue seems a bit off, i.e., some kids may be helped and some may be harmed. That implies that if we knew more about the kids, we could assign them appropriately, just pick out the ones who would do better in a one-sex family.
    Most likely the case is that some same-sex couples might have a good effect on the kids and some couples might have harmful effects. Like opposite sex couples who adopt children.

  8. I think you have to understand this in the context of the constitutional analysis. Under “rational basis” scrutiny, all you need to uphold the product of the political process is a one sided bet. If you have strict scrutiny, it is different. Many people argue that there should be strict scrutiny in relation to measures that discriminate against gays and lesbians, but in theory the lower courts said that wasn’t the case and rational basis scrutiny applied.

  9. Justice Kennedy seemed to be doing somewhat the reverse in suggested that same-sex-marriage bans are causing identifiable harms right now:“there are some 40,000 children in California … that live with same-sex parents, and they want their parents to have full recognition and full status.” I was wondering if there was evidence these parents would marry, if the law changed; but even so, it’s different from supposing the kids want it and are being hurt right now.

    Legal issues aside, I think it is highly questionable to base marriage (or many other) policies on what some research group purports to find about properties that help or hurt children (according to them). And how would the harms and benefits be weighed? Alcoholism, low income,single mom, etc. etc. are considered deleterious, but than there’s Bill Clinton and other success stories.

  10. Scalia is not guilty of the fallacy of the one sided bet. In the quote, Scalia says “whether that is harmful to the child or not.” “Not harmful” is the complement of “harmful” and thus includes anything that is beneficial. More to the point, it’s helpful to read the transcript rather than rely on left wing bloggers for quotes. According to the transcript, right after Scalia made the statement that has been much quoted, Justice Ginsberg said, “California–no California does.” Then Scalia directed this followup question to Cooper: “I don’t think we know that. Do you know the answer to that whether it–whether it harms or helps the child?” Scalia was very explicit in covering both cases in his followup question.

    However, you did commit the fallacy that you are accusing Scalia of when you said, “Assuming the American Sociology Association report is correct and the research doesn’t find any aggregate effect, that would suggest that some kids are helped and some are hurt, with no clear evidence that the average effect is positive or negative.” But that excludes the possibility that Scalia is raising, that despite some research finding no effect, we don’t really know what the answer is–the science isn’t really settled.

    That gets into the substance of Scalia’s claim. If you read the amicus curiae submitted by the American Sociological Association, it certainly does not read like an objective statement. The authors of the brief are unknown and we don’t know who exactly they represent. Did they take a poll of the members to decide what to write? Where there any expert dissenters? Does this represent the opinion of experts in the topic or a general consensus that includes people who know little to noting about the research? We don’t know.

    Moreover, the unknown authors cite evidence supporting their claim uncritically, often using the phrase “leading study” but attack vehemently studies that come to different conclusions. The unknown authors spend a lot of time attacking Mark Regnerus’s study which comes to a different conclusion. If you look at his study, you’ll see he is very critical of the studies that find no effect, accusing them of using small or biased samples. And yet the unknown authors of the amicus brief don’t answer this criticism of studies they seem to take as prima facie correct.

    Scalia did not commit any fallacy and he’s certainly right, based on the amicus brief, that the so-called science is unsettled.

    • Rick:

      I agree with your implicit point that, just because an official American Sociological Association report says something, it doesn’t mean it’s correct, and Scalia could be legitimately unsure about these issues.

      My main point way that, the framing of the question as “whether that is harmful to the child or not” is implicitly biasing the issue, by excluding the possibility of benefit (or, at best, tossing all benefits into the “no harm” category).

      I doubt Scalia was doing this on purpose; I think the problem just got mistakenly framed in that way. I see this sort of thing happening all the time, which is why I gave the fallacy its own name.

  11. Andrew, I would like to point out that gay couples who raise children are not depriving any child of a loving family they would otherwise have! So the entire question seems vacuous. Even if the kids happened to do worse on average – which they don’t – it’s the wrong comparison to make and has no bearing on the question of extending equal rights to same sex couples.

    • “Andrew, I would like to point out that gay couples who raise children are not depriving any child of a loving family they would otherwise have!”

      How so? You can only be adopted to one family at a time.

  12. I hate how in legal arguments “objectivity” = “playing dumb” when it suits your predisposition. True for liberal and conservative law talking guys.

  13. Andrew,

    I agree that the fallacy of the one-sided bet is important to note and watch out for. But Scalia’s statement is not a good example.

    First, as I already pointed out, the next question Scalia asked in the oral argument explicitly looked at both sides. So, he didn’t commit the fallacy. But there’s a much larger point that many people have commented on: even if Scalia did commit the fallacy, that would be irrelevant to what was at issue in the oral argument. Scalia would have been perfectly correct to focus only on one side of the bet.

    This case involves a law (prop 8) that was struck down on the basis of the equal protection clause. To determine whether the any law is in fact unconstitutional under equal protection, the court will apply one of 3 standards, depending on the circumstances: strict scrutiny, intermediate scrutiny, or the rational basis test. The facts and circumstances imply the rational basis test in the prop 8 case, and that is indeed the context of Scalia’s comment.

    The rational basis test is a very weak test and is biased toward deferring to the legislature. Under a rational basis test, the burden of proof is on the person who claims that a legal classification is unconstitutional. This person must show that the classification in the law is not rationally related to any legitimate interest of state. The state is not required to articulate what its interest is when passing the law and Justices are free to posit what that legitimate interest might be, as Scalia did when he suggested a legitimate interest might be that gay marriage is harmful to adopted children. Importantly, a Justice need not even agree with the posited legitimate interest (Scalia said he took no position on the matter). To pass the rational basis test, there has to exist at least one legitimate interest (possibly supplied by the Court) and the law can’t be irrationally related to it. And irrationality does not mean that the Court has to adjudicate scientific evidence or that the law has to be well-founded. A law can be stupid and still pass the rational basis test.

    That’s why, in the context of the rational basis test, it’s perfectly correct to focus only on one side of the issue, because that’s where the potential legitimate interests of the state are.

    I agree with Nameless though that in this case there’s a reasonable chance that the Court will apply an enhanced rational basis test, which they have done on occasion for similar cases. If they do that, they will weigh the state’s asserted interest against harm to those classified by the law. If they do get into those discussions, then both sides of the question–harm of help to adopted children–will then be relevant.

    • Rick:

      Thanks for the details. I still would prefer if Scalia had said he had no opinion on 3 options rather than 2, but I can see how his statement would make sense in this legal context.

  14. This issue is with a false dichotomy. The problem is that dichotomizing this question does not really affect his question. If there was support for the idea that having two gay/lesbian parents was superior for the children when compared to straight parents would they consider getting rid of straight marriage? I don’t think so. So the concern with are they worse off or not is the only question.

    Dealing with the fact that Mr. Hartmann discounts the findings of Mr. Regenerus. Yes, the findings were controversial. There are questions about whether the coding of the sexual orientation of parents was correct or not but I am not aware that anyone tried to replicate the findings with an alternative coding. I am sure that Mr. Gelman, like most people that have done social science research has had to make coding decisions and that he would not like someone trying to get him fired because they disagree with the decision he made. How the paper was accepted may be an important question but with the nature of the paper it would have found a journal. While someone may discount these findings, one could discount most of the findings from the early research on this subject for the same reason, personal self interest. Regenerus was funded by a conservative group (We don’t know what would have happened if he would have been funded by the NSF or NIH, though the social sciences should be encouraging pursuit of alternative funding sources in light of recent funding restrictions from Congress).Most of the early sociological research on this subject was done by people that happened to be gay. There does seem to me to be a need for more research on this subject, including not just the policy question about the affect on children, but also the affect of gay marriage on the spread of STDs/HIV, the mental health of the gay/lesbian population, and the economic impact of starting gay marriage.

    People are going to wear their rose colored glasses and the response to Mr. Regenerus’s article left this scholar wondering if Segal and Spaeth’s attitudinal model of the Supreme Court should be extended to academics. Not surprising that justice Scalia found it to be an area of controversy instead of a settled question.

  15. As you’ve posited it, the justice’s statement is first an appeal from/to ignorance (he doesn’t know the the facts or he hopes very few in his audience know the facts or both); next, it is a false choice (it leaves out the third possibility, offering a false choice between only two of three possibilities).

Comments are closed.