Amending Conquest’s Law to account for selection bias

Robert Conquest was a historian who published critical studies of the Soviet Union and whose famous “First Law” is, “Everybody is reactionary on subjects he knows about.” I did some searching on the internet, and the most authoritative source seems to be this quote from Conquest’s friend Kingsley Amis:

Further search led to this elaboration from philosopher Roger Scruton:

. . .

I agree with Scruton that we shouldn’t take the term “reactionary” (dictionary definition, “opposing political or social progress or reform”) too literally. Even Conquest, presumably, would not have objected to the law forbidding the employment of children as chimney sweeps.

The point of Conquest’s Law is that it’s easy to propose big changes in areas distant from you, but on the subjects you know about, you will respect tradition more, as you have more of an understanding of why it’s there. This makes sense, although I can also see the alternative argument that certain traditions might seem to make sense from a distance but are clearly absurd when looked at from close up. I guess it depends on the tradition.

In the realm of economics, for example, Engels, Keynes, and various others had a lot of direct experience of capitalism but it didn’t stop them from promoting revolution and reform. That said, Conquest’s Law makes sense and is clearly true in many cases, even if not always.

What motivated me to write this post, though, was not these sorts of rare exceptions—after all, most people who are successful in business are surely conservative, not radical, in their economic views—but rather an issue of selection bias.

Conquest was a successful academic and hung out with upper-class people, Oxbridge graduates, various people who were closer to the top than the bottom of the social ladder. From that perspective it’s perhaps no surprise that they were “reactionary” in their professional environments, as they were well ensconced there. This is not to deny the sincerity and relevance of such views, any more than we would want to deny the sincerity and relevance of radical views held by people with less exalted social positions. I’m sure the typical Ivy League professor such as myself is much more content and “reactionary” regarding the university system, then would be a debt-laden student or harried adjunct. I knew some people who worked for minimum wage at McDonalds, and I think their take on the institution was a bit less reactionary than that of the higher-ups. This doesn’t mean that people with radical views want to tear the whole thing down (after all, people teach classes, work at McDonalds, etc., out of their own free will), nor that reactionaries want no change. My only point here is that the results of a survey, even an informal survey, of attitudes will depend on who you think of asking.

It’s interesting how statistical principles can help us better understand even purely qualitative statements.

A similar issue arose with baseball analyst Bill James. As I wrote a few years ago:

In 2001, James wrote:

Are athletes special people? In general, no, but occasionally, yes. Johnny Pesky at 75 was trim, youthful, optimistic, and practically exploding with energy. You rarely meet anybody like that who isn’t an ex-athlete—and that makes athletes seem special.

I’ve met 75-year-olds like that, and none of them was an ex-athlete. That’s probably because I don’t know a lot of ex-athletes. But Bill James . . . he knows a lot of athletes. He went to the bathroom with Tim Raines once! The most I can say is that I saw Rickey Henderson steal a couple bases in a game against against the Orioles.

Cognitive psychologists talk about the base-rate fallacy, which is the mistake of estimating probabilities without accounting for underlying frequencies. Bill James knows a lot of ex-athletes, so it’s no surprise that the youthful, optimistic, 75-year-olds he meets are likely to be ex-athletes. The rest of us don’t know many ex-athletes, so it’s no surprise that most of the youthful, optimistic, 75-year-olds we meet are not ex-athletes. The mistake James made in the above quote was to write “You” when he really meant “I.” I’m not disputing his claim that athletes are disproportionately likely to become lively 75-year-olds; what I’m disagreeing with is his statement that almost all such people are ex-athletes. Yeah, I know, I’m being picky. But the point is important, I think, because of the window it offers into the larger issue of people being trapped in their own environments (the “availability heuristic,” in the jargon of cognitive psychology). Athletes loom large in Bill James’s world—I wouldn’t want it any other way—and sometimes he forgets that the rest of us live in a different world.

Another way to put it: Selection bias. Using a non-representative sample to drawing inappropriate inferences about the population.

This does not make Conquest’s or James’s observations valueless. We just have to interpret them carefully given the data, to get something like:

Conquest: People near the top of a hierarchy typically like it there.

James: I [James] know lots of energetic elderly athletes. Most of the elderly non-athletes I know are not energetic.

29 thoughts on “Amending Conquest’s Law to account for selection bias

  1. > People near the top of a hierarchy typically like it there.

    This is a true statement, but not a fair summary of Conquest’s position.

    > I knew some people who worked for minimum wage at McDonalds, and I think their take on the institution was a bit less reactionary than that of the higher-ups.

    Your error is the phrase “take on the institution.” Conquest is making no claims about a McDonald employee’s take on McDonald’s as an institution. Indeed, what the CEO is doing in headquarters is very far away from a cook on the line. Conquest’s claim is about the junior employee’s views on those “things he knows best,” which is not cooperate strategy, it is the details of what happens in a McDonald’s kitchen. How hot should the burners be? When should you unwrap the patties? How many people should be working on Thursday nights? Conquest’s claim is that, on those topics, the junior cook will be a reactionary precisely because he knows them best.

    • D:

      I agree that Conquest was not making any claims about McDonalds employees. From the Amis excerpt, it seems that Conquest was talking about his upper-class or upper-middle-class English friends.

      • I think it’s more of a universal statement than your reading of it is. When I hear “Everybody is reactionary on subjects he knows about,” it sounds like a restatement of the Dunning-Kruger effect: The less we know about something, the simpler we think it is. And I don’t think that’s a a bias limited to high class mid century Brits.

        To take your example of minimum wage MdDonald’s workers. My experience is that I know quite a few “Fight for $15” proponents who have never bothered to simply divide out net profit by number of employees. But ask them what they think about relocating the fry line and you’ll get a detailed answer of why that does or doesn’t make sense. In other words, they know how day to day operations work and can therefore realize that what sounds like a good idea has non-obvious implications, but they don’t realize that the same it true of areas they are less familiar with.

        Or, to put it more universally: Most of modern management theory comes from the realization that when an employee who does the job day in and day out tells a manager his thoroughly researched process change to match industry standards won’t work, she’s probably right. Line workers are notoriously resistant to change, and a large part of that is because they’re experts in the jobs they do and can see the problems with management’s attempts to tinker. That’s what I’d consider a textbook example of being a reactionary on the subject you know best.

        It’s interesting that we were developing things like LEAN manufacturing which basically implement this principle in the ’50’s, right around the same time Conquest was articulating a different version of the same thing.

        • I agree that Conquest’s idea is a restatement of Dunning Kruger, that’s an astute observation.

          But this:

          “when an employee who does the job day in and day out tells a manager his thoroughly researched process change to match industry standards won’t work, she’s probably right”

          is false and I’ve demonstrated it so many times.

          People frequently can’t see or don’t care to bother thinking about the benefits of changes in their jobs. If you show them an obvious change they usually won’t implement it until they’re forced to do so or embarrassed into doing it by someone else implementing it successfully and beating them at their job.

          This goes from the lowest laborer right up to the most decorated academic and it’s true of both individuals and organizations. If you want proof: Amazon’s been in business for twenty years, growing like mad, but it’s only been in the last four or five years that even it’s most aggressive competitor (wal*mart) embraced the primary aspects of its business model. Kroger only started picking up the ball in the last 2-3 years after Amazon acquired whole foods. And old guard retailers like Macys, JC Penny and others are still in denial.

          When I was in grad school, I noticed my office mate trying to balance a complex chemical reaction. I showed her a foolproof method for using a matrix to solve the equation. This method takes only a few minutes. Yet she refused it and continued for a week on her manual method, never resolving the problem.

          Like many, if not most, business concepts, Lean and Six Sigma are frequently “implemented” by people who have only the foggiest understanding of the concept and no real interest in actually accomplishing an improvement. That’s why they’re necessary in the first place, right? Because people don’t want to act on common sense observations, so you need a method or a rule to force them to do so.

          I guess that’s about as reactionary as you can get! :)

      • > I agree that Conquest was not making any claims about McDonalds employees.

        No! This is the opposite of what I said. (And see Hasdrubal.)

        I/Conquest/Hasdrubal all believe that McDonalds employees are reactionary about the things they know best, just like Ivy League professors and fancy brits. This is true of everyone.

        • D:

          You may believe whatever about McDonalds employees—and I do like your idea for an experiment to study this, as you mentioned elsewhere in the thread—but I see no reason to believe that Conquest believed that McDonalds employees are reactionary about the things they know best. It’s my impression that he was talking about his friends/colleagues, and maybe also about people he knew in the Soviet Union. I could be wrong on this. In any case, the larger question is interesting.

        • > I see no reason to believe that Conquest believed that McDonalds employees are reactionary about the things

          Huh? You yourself wrote:

          > Robert Conquest was a historian who published critical studies of the Soviet Union and whose famous “First Law” is, “Everybody is reactionary on subjects he knows about.”

          My definition of “everybody” includes people who work at McDonalds. Doesn’t yours? If Conquest’s own words aren’t good enough evidence as to what he meant, I am unlikely to come up with anything that might convince you.

          But, also, just look at all the people in this thread who, like me, testify that the Law applies in many places outside of faculty lounges and British men’s clubs. If we think the law applies to “everybody,” why would you assume that Conquest does not think that it applies to “everybody,” especially when he uses the word “everybody?”

        • D:

          I’m basing this on the Amis quote: it seemed to me that Conquest’s remark was based on his personal experiences and not intended to represent a larger claim about social science. That is, I suspect that, by “Everybody,” he meant, “Everybody I know.” Just my guess.

        • I would expect Conquest’s personal experience to include discussing with people other than his upper-class English friends. He enlisted in the army, lived abroad before, during and after the war, had a Bulgarian wife… The set of people he knew was surely more varied than what you give him credit for. When Amis writes “from my own and others’ example” you choose to read “my own and his other upper-class English friend’s example”.

          I agree with other commenters that your “interpretation” (close to the popular quote from Upton Sinclair) seems too far from the original statement (more related to Chesterton’s fence and Gell-Mann’s amnesia).

  2. I don’t think people with experience and expertise in an area have to be happy with the status quo to be conservative. Being conservative doesn’t necessarily mean seeing no need for change but rather favoring incremental/tentative changes over radical changes. I am not sure Scruton’s interpretation is the best one. I think it is more of a case of the more you know the more you realize you don’t know (the solution). An expert that knows the complexity of the problem is more likely than a non-expert to be skeptical of someone promising “a giant leap.”

  3. How might we test Conquest’s Law? Easy!

    1) Come up with a random sample, to include McDonald’s cooks, Ivy league profs and others. Sadly, the sample should probably be restricted to 3 or 4 professions, at least for a given survey.

    2) Learn about those three professions! Otherwise, we can’t write sensible questions . . .

    3) Come up with plausible policy changes to the current way of doing things that are all delta away from current practices in those professions. (I realize that this is hard.)

    4) Interview each member of your sample. Give them background information on each profession. Given them a fair pro/con case for each policy change. Ask each if they would be in favor of the change.

    5) Conquest predicts that the cooks would be much more willing to try a change in the policies associated with professor work rules/standards/policies than a change having to do with cooking. And professors will be the reverse! Each will be most resistant to change — most “reactionary” — in the area that she knows best.

  4. My version of Conquest’s conjecture (nowhere close to the status of a law) is that both sides are true.

    Knowing a lot about something quite rightly makes you suspicious of most changes because you can see the ramifications and unintended consequences that a less knowledgeable person might overlook. If you intimately understand a certain system, you are unlikely to view ANY change with an “all else being equal this is an improvement” assumption.

    But knowing a lot about something also makes you feel like you have a vested interest in the status quo. Your knowledge may well position you advantageously in that field or setting and you’re only human to resist giving up that competitive advantage. So the knowledgeable expert/insider has a strong motivation to be “reactionary” purely out of self-interest.

    What’s important is that when faced with potential changes to things we know a lot about, we also have enough self-knowledge to at least try to dial back our “reactionary” tendencies in compensation for our self-interest bias. While still, in most cases, being rightly leery of overly optimistic “change is good” scenarios by those who don’t fully understand the consequences.

    P.S. I just noticed how “consequences” and “Conquest” contain similar sounds. How ’bout that?

    • “P.S. I just noticed how “consequences” and “Conquest” contain similar sounds. How ’bout that?”

      Woah! Dude! You’re just like totally blowing my mind! I did not see that at all! Now that I see it, I can’t stop thinking about it!

      So dig this man! It also sounds like “con queso”. Mmmmm. I could really go for some nachos right now.

  5. Your remarks on the base-rate fallacy are correct. It’s not clear they are useful. Even in the James example, it’s not clear how correctly assessing the natural frequency of trim, youthful, optimistic and energetic 75 year olds in the general population has any bearing on “special.” If the specialness means moving with athletic grace and force at the same time, I’m pretty sure James’ is still right that athletes are the special ones, and the comment on James was thus incorrect, despite better statistical philosophy. I rather suspect James had something like that in mind, but who knows?

    And the prime example, Conquest is similarly confused. Conquest’s Law is not a misunderstanding that people at the top of the hierarchy like it there, thus whether that claim is an error in statistical reasoning is beside the point. Conquest’s First Law (as the excerpt says, is a claim that real knowledge is inherently conservative. But, the implicit equation of prudence and authority with the the political movement of conservatism is entirely BS. That’s why it is a mistake to throw aside the dictionary definition of reactionary. Conservatives would like you to believe that it’s some sort of prudential wisdom of the ages, but that’s nonsense. Conquest would almost certainly have opposed laws against child labor if he was around when they were first proposed. Conservatives want to do away with the New Deal, as in bank regulations, so it has nothing to do with caution and prudence and experience and reasoned judgment. Further, much of the supposed organic tradition is fictional, like tartans or “under God” being part of the pledge of allegiance. Conquest was not making a claim about true expertise, he was making a claim about motives. This claim was not based on anything but wishful thinking. Upton Sinclair’s observation about how it is nearly impossible to convince a man of something if his salary depends upon not understanding it is Conquest’s First Law, without the self-serving flummery.

    Perhaps not so by the way, the notion that Conquest was a lonely scholar bravely fighting the tide of Communist sympathies is nonsense. Anti-Communism was the rule, partly because the universities were purged as much as possible, which is odd since no Marxists or Communists got tenure for decades. Conquest was treated with reserve because he tried to prove Stalin was worse than Hitler. Given the vast fortunes in blood and money spent fighting Communism over the decades, this was not a hard sell with reactionaries. So far as I know the massive figures he scame up with have not been supported. I doubt that a Stephen Cohen, Sheila Fitzpatrick or J. Arch Getty would be so bold as to court controversy by explicitly stating Conquest was a poet and a reactionary, not an honest historian, because no doubt he had the citations properly laid out. He’s still upheld in Ukraine, but they have fascists in the army and government there, so this is not a recommendation. If you feel this is ungenerous? Conquest’s First Law still implies that the experts are always right. And his Second Law, about how every organization seems to be run like it is headed by agents of an enemy organization is to explain any real-world contradictions to the First Law by…subversion. Reactionary, indeed!

    • Steven:

      I think Bill James’s statement (“You rarely meet anybody like that who isn’t an ex-athlete—and that makes athletes seem special”) described Bill James’s experience, and that’s fine. Bill James is an important person and I’m always interested in learning more about him!

      But it doesn’t describe my experience. I have met 75-year-olds who are “trim, youthful, optimistic, and practically exploding with energy”—and none of them are ex-athletes. No surprise, given that I don’t actually know any 75-year-old ex-athletes!

      My point is a real one. James thinks this trim, youthful thing is a special property of athletes—but it’s not. It’s a property of the sorts of athletes he hangs out with—and, for that matter, the sorts of non-athletes he hangs out with.

      • I did not know your idea of “trim, youthful, optimistic, and practically exploding with energy” was the same as Bill James, which of course makes your point perfectly relevant as well as entirely correct.

        • It is certainly possible that Andrew and Bill James have different conceptions of “trim, youthful and energetic”; if Andrew introduced Bill James to one of Andrew’s trim/youthful/energetic friends, James might well view him as a fat, lugubrious, slob, compared to Johnny Pesky. I think it is pretty likely that people who are well above average in trim youthfulness when they are 30 are likely to be above average when they are 75, so I don’t know that Bill James is a great example. But the principle probably still holds.

    • Mr. Johnson seems to be acting out Conquest’s Law in his comments on conservatives. Not being one himself, he mis-represents how conservatives approach problems by turning them into heartless people who care nothing for actual human beings

  6. I agree with the comments above that the heart of Conquest’s idea is knowledge and ignorance. An ignorant outsider may say “why don’t you just make cars run on water?, while a knowledgeable insider knows why you can’t do that.

    But, let me point out a somewhat opposite wisdom: insiders tend to more cynical about what they understand. The education and research industries are treated with great respect by many ignorant outsiders. Insiders, however, know how the sausage is made and tend to be more cynical.

    • > insiders tend to more cynical about what they understand. The education and research industries are treated with great respect by many ignorant outsiders. Insiders, however, know how the sausage is made and tend to be more cynical.

      I think this is true, and also a reason why taking the word “reactionary” at face value here is wrong. Generally people who are insiders have lots of complaints and unhappiness with the status quo, but are skeptical of specific solutions or the feasibility of getting those solutions in place. Even in the example provided of education, I don’t meet many experts or industry insiders who think that it’s fine as-is and shouldn’t change; they can all point out a dozen or so perversities and misaligned incentives in a minute if asked. At the same time, they would look critically and carefully at solutions before implementation.

      I disagree that we can just not take the word reactionary at face value here though; I think it’s a deliberate bit of sleight of hand on Conquest’s part to try and identify political words possessing specific meanings with ideas that are loosely related in some dimension but definitely distinct.

  7. Is child labor law that clear-cut a case? Growing up in a poor country where there was no social safety net, I can envision situations where children should be allowed to work since the alternative is starving. This is a case where people with experience (needing to send their children to do hard labor) would probably have a different perspective from people in first world society.

    • Yyw:

      The debate was not just about child labor; it was particularly about children being employed as chimney sweeps in England. You can read the relevant chapter in the book Roads to Ruin for details.

      • Andrew,

        I agree that the law made sense here. It’s interesting though it took almost 100 years for the regulation effort to succeed. I cannot imagine the master chimney sweeps themselves had that much political power.

Leave a Reply to Martha (Smith) Cancel reply

Your email address will not be published. Required fields are marked *