The problems with popular internet heuristics such as “Hanlon’s razor,” “steelmanning,” and “Godwin’s law,” all of which kind of fall apart in the presence of actual malice, actual bad ideas, and actual Nazis.

From my review of Dan Davies’s book on business fraud:

Fraud might be an unusual “tail risk” in business, but in science it’s usual. It happens all the time. Just in my own career, I had a colleague who plagiarized; another one who published a report deliberately leaving out data that contradicted the story he wanted to tell; another who lied, cheated, and stole (I can’t be sure about that one as I didn’t see it personally; the story was told to me by someone who I trust); another who smugly tried to break an agreement; and another who was conned by a coauthor who made up data. That’s a lot! It’s two cases that directly affected me and three that involved people I knew personally. There was also Columbia faking its U.S. News ranking data; I don’t know any of the people involved but, as a Columbia employee, I guess that I indirectly benefited from the fraud while it was happening. I’d guess that dishonesty is widespread in business as well.

This led me to an point that’s important enough that it deserves a post of its own (i.e., this one):

This also reminds me of the problems with popular internet heuristics such as “Hanlon’s razor,” “steelmanning,” and “Godwin’s law,” all of which kind of fall apart in the presence of actual malice, actual bad ideas, and actual Nazis. The challenge is to hold the following two ideas in your head at once:

1. In science, bad work does not require cheating; in science, honesty and transparency are not enough; just cos I say you did bad work it doesn’t mean I’m accusing you of fraud; just cos you followed the rules as you were taught and didn’t cheat it doesn’t mean you made the discovery you thought you did.

2. There are a lot of bad guys and cheaters out there. It’s typically a bad idea to assume that someone is cheating, but it’s also often a mistake to assume that they’re not.

A related point from that post:

Davies refers to “the vital element of time” in perpetuating a fraud. A key point here is that uncovering the fraud is never as high a priority to outsiders as perpetuating the fraud is for the fraudsters. Even when money is at stake, the amount of money lost by each individual investor will be less than what is at stake for the perpetuator of the fraud. What this means is that sometimes the fraudster can stay alive by just dragging things out until the people on the other side get tired. That’s a standard strategy of insurance companies, right? To delay, delay, delay until the policyholder just gives up, making the rational calculation that it’s better to just cut your losses.

I’ve seen this sort of thing before, that cheaters take advantage of other people’s rationality. They play a game of chicken, acting a bit (or a lot) crazier than anyone else. It’s the madman theory of diplomacy. We’ve seen some examples recently of researchers who’ve had to deal with the aftermath of cheating collaborators, and it can be tough! When you realize a collaborator is a cheater, you’re dancing with a tiger. Someone who’s willing to lie and cheat and make up data could be willing to do all sorts of things, for example they could be willing to lie about your collaboration. So all of a sudden you have to be very careful.

P.S. I talked about other problems with “steelmanning” here.

20 thoughts on “The problems with popular internet heuristics such as “Hanlon’s razor,” “steelmanning,” and “Godwin’s law,” all of which kind of fall apart in the presence of actual malice, actual bad ideas, and actual Nazis.

  1. Spending too long engaging with half-assed internet and social media versions of a thesis can cripple your ability to deal with the strong version of it. We have all seen people who thrive in that situation wilt when they encounter an actual expert. On the other hand, when you are a pediatrician whose patients are not sure about vaccinating their children, you will achieve better public health if you answer the antivaxx arguments they know rather than more sophisticated arguments that exist elsewhere. “Seeking truth” and “persuading a specific audience” do not necessarily have much in common.

  2. It also took a great deal of regulation and legislation and institution building to make fraud rare in some kinds of business. That was easier to achieve because specific people are hurt immediately when they think they bought alcohol but its actually laced with methanol, or when they buy a stock for $100 plus fees and actually their broker bought it for $96 and secretly pocketed the difference. The chain of causation between “publishing social science claims based on fake data or invalid statistical methods” to “people getting hurt” is longer and more indirect.

  3. I don’t have any good evidence myself aside from anecdotal experience. My dealings with insurers is that they say no first as a standard approach. It is rarely their final answer, but it is worth trying since some percentage of cases will settle for that answer. With medical care, I have considerable anecdotal evidence that insurers routinely deny claims that they may eventually approve – clinicians often find it not worth the effort to fight denials even when they suspect they will get approval eventually (it is a simple cost benefit calculation – I’m not thinking of major expensive treatments here, but the many smaller disallowed amounts).

    It is right to ask about evidence. We chastise researchers who don’t consider the evidence. But part of the evidence should concern motive and opportunity, shouldn’t it? Insurers have plenty of incentive to deny claims as a first line policy and plenty of such opportunity. Similarly, researchers have motive and opportunity to falsify their data and/or analysis, as we keep discovering. It seems that the main thing standing in the way of bad work or bad actions is the ethics of the actor (I am saying that penalties are of secondary importance). And these ethics are proving far less effective than I used to believe. Once the ethical guardrails disappear is it possible to get them back? I’m not at all confident. And then the focus appears to turn to better enforcement: stiffer penalties for offenders (banished from academia for life?). Where is the evidence that stiffer penalties are effective? I believe there exists such evidence as well as plenty of counter-evidence. But my impression is that this is a battle that is being lost (again, without evidence, but with considerable fear).

    • I did find a little evidence about insurance claim denials (https://www.experian.com/healthcare/resources-insights/thought-leadership/white-papers-insights/state-claims-report). I know the insurance industry itself has far better data but it is not available to me. This data comes from a small survey (250 health care professionals in various roles) and is designed to highlight the services that the sponsor provides – improved claims submission technology. But, for what its worth, it shows 44% of respondents very or extremely concerned that payers will not pay. It also claims that 41% of the providers say that 10% or more of their claims are denied (up from 30% in 2022). And it shows that the primary reason for claims denial is missing or inaccurate data (cited by 50% of respondents).

      This is hardly a damning picture of insurance practices. None of the provided reasons for denials appear to say anything about insurance industry practices. But I still think that motive and opportunity are relevant in evaluating what little data is available. This is not the first time I’ve wished that insurance data were required to be openly provided.

  4. I think of “steelmanning” as mostly a wish that people would stop making objections that are perceived as irrelevant nitpicking, to overlook trivial mistakes or political disagreements and look at what the writer is really saying. That seems fine, for the most part, though it just relocates the argument over when a mistake destroys the argument, or when politics is at the core of the argument and can’t be ignored. If a writer labels Hillary Clinton a Republican, it might cast doubt on their competence to discuss US current events, but it might also have little to do with their core argument about funding for basic research.

    Looking for the strongest possible argument for what the writer was trying to say feels to me like more than that. If someone argues we need to fund basic research because we might find the Holy Grail, giving them credit for almost but not quite arguing that we need to fund basic research because science is at the core of our knowledge of the world seems like missing the point. Most readers aren’t capable of that. Also, if there’s a dispute between two parties in some area, it’s on a writer’s allies to “steelman” their writing in this way. It’s hard to see how it can be helpful to require their rivals to behave the same way. It’s possible I’m missing some assumption here that would explain how this would be a good process. Or how it could exclude bad work.

    What I hope “steelmanning” never means is asking “why would he think HRC is a Republican? can I imagine some line of thought where that would make sense?” That feels like collapsing the difference between empirical and poetic writing.

  5. “…Even when money is at stake, the amount of money lost by each individual investor will be less than what is at stake for the perpetuator of the fraud. What this means is that sometimes the fraudster can stay alive by just dragging things out until the people on the other side get tired. That’s a standard strategy of insurance companies, right? To delay, delay, delay until the policyholder just gives up, making the rational calculation that it’s better to just cut your losses.”

    I think this is more or less backwards. Individual policy holders have a greater incentive to get fraudulent (or just bad) claims approved than insurance company employees have to deny them. As a result more bad claims are approved than good claims are denied. Just like more bad papers get published than good papers get rejected.

    • You have a point but I think it is more complicated than that. Medical claims usually involve several parties: insurer, insured, medical provider. The insured individual may have the incentive you are citing but the most common ‘victim’ of the denial is the medical provider. They often do not have much incentive to get claims approved, even legitimate ones (the costs of the battle are large enough for them to absorb the loss rather than fight to get it approved – the insured person won’t care if they are not billed for the care). There is little incentive for the patient to submit false claims since it usually involves receiving medical care that they don’t want or need. So, the individual “policy holder” you are citing is really the individual medical provider. They do have an incentive to get fraudulent or “just bad” claims approved, but the potential cost to their continued medical practice loom large. The insurer, on the other hand, can benefit greatly from a policy of denial even if many claims are eventually approved. I think the costs and benefits to the parties are more complex than you are stating.

      The publication example is interesting and I haven’t thought it through. But it does not have the same characteristic of three parties involved – there is only the researcher and the journal.

      • “…The insurer, on the other hand, can benefit greatly from a policy of denial even if many claims are eventually approved. …”

        How many of the claims that are eventually approved were submitted with errors and approved after the claim was fixed? I don’t see why an insurance company should be expected to approve erroneous claims.

        In my experience almost all medical and dental claims are approved routinely.

        • This is all anecdotal (both my experiences and yours), but the providers I have known frequently have had claims denied requiring them to jump through multiple hoops to eventually get them approved. The little data that I have been able to find contrast with your final statement that “almost all medical and dental claims are approved routinely.” It is fair to note that improper paperwork should not be approved by insurers and there is plenty of that – but it is wrong to ignore the ease with which insurers can simply deny as a first reaction, leaving it to the provider and/or insured to protest if they want to get it approved. That was certainly my experience with the only homeowners’ policy claim I ever had (when my house burned down in a forest fire) – the insurer put every hurdle they could before approving my claim in hopes that I would settle for less. Again: motive and opportunity.

        • The vast majority of claims are probably routine visits that shouldn’t be covered by insurance anyway. It is rent extraction that should be pure profit for the insurer.

          I’d expect claims for more expensive problems to be rarer and regularly disputed.

  6. Godwin’s law has done a lot of damage recently. It’s problem is at bottom that it makes the assumtion that fascist ideology is rare and unusual and so discredited that nobody is likely to agree with it. The flaw with that kind of assumption is that we know from history that large numbers of people *have* at times held fascist or fascist-adjacent views. It was a mass movement in the 1930s with majority or near majority support in several important countries and significant minority support practically everwhere. To assume then that nothing like this can ever happen any more was extrenely ahistorical.

    My hypothesis is that some temporal distance from the events up to 1945 was neded for fascism to become a relevant force again and that’s why it took roughly 80 years or a lifetime. And now we have to contend with the reality that 21st century people are no better people, morally or intellectually, than those of the 1930s. Except their capacity for doing damage is even greater.

    • Please check and see which policies you disagree with, because its literally the current DNC party platform + a few things specific to 1920s italy:
      https://en.wikipedia.org/wiki/Fascist_Manifesto

      In fact, the republican party and most EU parties are corporatist as well, which is the essence of fascism.

      Finally, most people who throw around the term don’t know there was a war between fascists and nazis in 1930s Austria.

      • The doctrine was essentially disavowed by Mussolini

        The Popolo d’Italia described itself in its subtitle as the daily organ of fighters and producers. The word producer was already the expression of a mental trend. Fascism was not the nursling of a doctrine previously drafted at a desk; it was born of the need of action, and was action; it was not a party but, in the first two years, an anti-party and a movement. The name I gave the organization fixed its character.
        Yet if anyone cares to reread the now crumpled sheets of those days giving an account of the meeting at which the Italian Fasci di combattimento were founded, he will find not a doctrine but a series of pointers, forecasts, hints which, when freed from the inevitable matrix of contingencies, were to develop in a few years time into a series of doctrinal positions entitling Fascism to rank as a political doctrine differing from all others, past or present.

        https://sjsu.edu/faculty/wooda/2B-HUM/Readings/The-Doctrine-of-Fascism.pdf

        and explicitly describes fascism as being anti-socialist, anti-liberal, anti-democracy, anti-utopian. When characterizing fascism, you always quote anyone except the actual fascist dictators who actually ruled Italy and Germany. Why?

        • Because that’s only the endgame of centralizing power by merging industry and state. A strongman and his gang take over and start oppressing everyone.

          After fascism does its thing the corruption and incompetence runs the country into the ground and the finger pointing starts. The state starts attacking internal groups, followed by external, using violence to perpetuate itself.

          Also, please don’t ever believe anything a politician says. That’s another authoritarian behavior tangential to fascism.

        • If you’re interested in studying the political rhetoric of fascism, the persuasive style that was ultimately successful in brining Hitler and Mussolini to power was not the earlier socialist-influenced economic populism that you quote, but a later pivot to an aggressive, warmongering, explicitly anti-utopian and anti-democratic style. That’s the interesting part of fascism to many historians; that a controlling coalition of people, especially young men, are attracted to not a comfortable life with material needs taken care of and broad individual freedom, but rather to a difficult life of struggle with individual freedom subjugated to the state. Furthermore, this centralization of power, restriction of liberty, and material poverty is not the bait-and-switch you suggest, but is the actual PROMISE.

          Fascism wants man to be active and to engage in action with all his energies; it wants him to be manfully aware of the difficulties besetting him and ready to face them. It conceives of life as a struggle in which it behooves a man to win for himself a really worthy place, first of all by fitting himself (physically, morally, intellectually) to become the implement required for winning it. As for the individual, so for the nation, and so for mankind.

          In the Fascist conception of history, man is man only by virtue of the spiritual process to which he contributes as a member of the family, the social group, the nation, and in function of history to which all nations bring their contribution. Hence the great value of tradition in records, in language, in customs, in the rules of social life. Outside history man is a nonentity. Fascism is therefore opposed to all individualistic abstractions based on eighteenth century materialism; and it is opposed to all Jacobinistic utopias and innovations. It does not believe in the possibility of “happiness” on earth as conceived by the economistic literature of the 18th century, and it therefore rejects the theological notion that at some future time the human family will secure a final settlement of all its difficulties.

          Anti-individualistic, the Fascist conception of life stresses the importance of the State and accepts the individual only in so far as his interests coincide with those of the State, which stands for the conscience and the universal, will of man as a historic entity.

          And if liberty is to be the attribute of living men and not of abstract dummies invented by individualistic liberalism, then Fascism stands for liberty, and for the only liberty worth having, the liberty of the State and of the individual within the State. The Fascist conception of the State is all embracing; outside of it no human or spiritual values can exist, much less have value. Thus understood, Fascism, is totalitarian, and the Fascist State — a synthesis and a unit inclusive of all values interprets, develops, and potentates the whole life of a people.

          That this rhetoric is actually appealing to so many, especially young men, is THE interesting object of the study of fascism to me. If you’re only interested in the material policies of fascism, I mostly agree with you and don’t see a whole lot of difference between the 20th century fascist states and other modern totalitarian nation-states besides maybe a greater tilt towards expansionist warfare (though with China, who knows). But an attitude like this

          Also, please don’t ever believe anything a politician says.

          dismisses rhetoric itself as even a valid object of study, which I think is to your peril. That these speeches worked is really weird to me, but they did! And I think they still kind of do work on young men today!

        • Anon…

          In the early part of the 20th century socialism, defined at the time as the workers who use them controlling the capital assets of society was all the rage. Several authoritarian groups decided they could coopt this support to create personal authoritarian control. Italian fascism, German “National Socialism”, and Lenin+Stalin’s “Soviet Socialist Republic” were the most successful at this cooption. None of these had socialism as their goal, but they used socialist fervor to effect their rise to authoritarian power.

          The rhetoric you quoted is consistent with this pseudo socialism for political authoritarian gain. Its timing early in the rise of Mussolini makes sense as well. “somebody” quotes later Mussolini after he had seized control and needed to justify his de facto dictatorship rather than his rise to power.

          The commonality of those three groups was that they were de facto made in the image of benefitting the people at the top, Mussolini and friends, Hitler and Friends and Lenin or Stalin and friends.

          Stalin even made a pact with Hitler but they began in fighting and it was reversed.

          Literally the last people any of those leaders cared about were workers in shoe factories or whatnot.

          Neither Democrats nor Republicans show any actual movement towards benefitting “the little people”. For example there were a few years when Dems had a filibuster proof majority during Obama. But no universal healthcare, laws protecting abortion, trans rights, revised tax code, bolstering of antitrust, stimulus for small business, improvement in education policy or funding, etc were forthcoming, despite all of those sorts of things being literal platform points. we did drone bomb a lot of middle easterners though.

Leave a Reply

Your email address will not be published. Required fields are marked *