The role of models and empirical work in political science

Posted on by

Bill Kelleher writes:

I recently posted a review of A Model Discipline, by Clarke and Primo on Amazon.com. My review is entitled “Why Physics Envy will Persist,” at
http://www.amazon.com/gp/review/R3I8GC5V1ZSYVI/ref=cm_cr_pr_rvw_ttl?ASIN=019538220X

As you likely know, they are critical of the widespread belief among political scientists in the hypothetical-deductive method. As part of my review of the book, I offer an explanation as to why, three years later, H-D hegemony continues as strong as ever.

I responded that I actually received a copy of that book in the mail awhile ago but hadn’t actually looked at it. I indeed am a bit bothered by the whole “Empirical Implications of Theoretical Models” thing. I see lots and lots of political science projects which are set up as a set of research hypotheses that are then to be empirically tested, and the whole thing often seems bogus to me.

I’m more and more becoming convinced of Dan Kahan’s idea that the paradigmatic task of empirical science is not the testing of hypotheses but the gathering of data in order to distinguish between competing models of the world.

17 thoughts on “The role of models and empirical work in political science”

    • I’m about 2/3 of the way through Platt’s piece. (I wasn’t familiar with either his or Chamberlin’s.) Lots to stimulate an extended discussion but one thing that strikes me as a limitation of his approach is that causal models aren’t necessarily “clean”. Model A may provide more accurate predictions than Model B in some circumstances and Model B may be more accurate in others. If the models being compared are understood to be approximations of reality then “Whether A is superior to B or vice versa depends upon the circumstances.” isn’t hard to swallow. Your role as a scientist is to establish the limits of applicability of the respective models – or perhaps it’s more appropriate to say that your role is to establish the fidelity of the respective models. If an objective determination of superiority is feasible then certainly pursue it but I believe it’s a mistake to impose a literal “Prove or disprove?” approach to all scientific questions.

      Two from John Tukey:

      “We often forget how science and engineering function. Ideas come from previous exploration more often than from lightning strokes. Important questions can demand the most careful planning for confirmatory analysis. Broad general inquiries are also important. Finding the question is often more important than finding the answer. Exploratory data analysis is an attitude, a flexibility, and a reliance on display, NOT a bundle of techniques, and should be so taught. Confirmatory data analysis, by contrast, is easier to teach and easier to computerize. We need to teach both; to think about science and engineering more broadly; to be prepared to randomize and avoid multiplicity.”

      “Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.”

      Reply ↓

  3. Well, that’s how many people in psych* think they are doing their research. They will set up two competing theories, and then eliminate one (after looking at the data) as the less plausible one. The only problem here is that they already know before they start looking at the data which theory should win. No lab worth its salt is going to publish a paper saying, we compared our theory with our competitor’s, and the competitor’s theory won. Never gonna happen.

    Reply ↓

  4. paradigmatic task of empirical science is not the testing of hypotheses but the gathering of data in order to distinguish between competing models of the world.

    If one has competing models, one generates hypotheses where the competing models disagree on predictions (or, more precisely, on certain subsets of data). So aren’t the two intertwined?

    Reply ↓

    • Yeah, I think from a philosophy of science perspective (which is the perspective from which you talk about things like the hypothetical-deductive method), I struggle to see the difference.

      Perhaps Andrew has something more specific in mind for hypothesis testing, perhaps something like “we reject theta=0 at p<.05 and will thus treat as true that theta = theta.hat going forward"

      Reply ↓

    • There is a difference between gathering data because it bears a relation to important or interesting things in life, and gathering data because you can construe it to support your theory.

      Reply ↓

  5. First you want to see of you can actually distinguish different groups – say sheep from horses. Then you can go on from there.

    Sheep and horses have 4 legs, can’t use legs. Sheep are woolier but some horses have thick coats; maybe there is a wooliness measure that would work. Sheep are generally smaller; they probably cluster well apart from horses by weight. Horses eat more than sheep. OK, let’s use weight and amount of food to distinguish the groups.

    Now you may be in a position to test a hypothesis about, say, whether the amount of grazing and soil compaction (from all those animals trampling the ground) in a field would favor raising sheep or horses in a particular pasture.

    Or, to put it another way, if you can’t distinguish the meanings of two words, you can’t use them effectively. It’s like learning a vocabulary before putting it to use.

    Reply ↓

    • Why not try looking for “laws” rather than differences. Try it and watch understanding of that topic flourish. This differences idea was come up with by stats dudes because that is a problem they could address. Too bad it is of little use scientifically.

      Reply ↓

  6. So 40 or 50 years after Willard Van Orman Quine, Nelson Goodman, Tom Kuhn, and Hilary Putnam (and quite a few other philosophers), political scientists are catching up? Hilary Putnam once made the point that, when philosophers and physicists get into an argument, philosophers typically lose. But that never meant that scientists had nothing whatsoever to learn from philosophy.

    Reply ↓

    • Hugh,

      Philosophers only lose to physicists because they have fewer lasers. If the NSF had just funded my grant proposal to give all philosophers lasers, we wouldn’t live in a world with 10 dimensions full of tiny strings…we’d be right back to 3+1 dimensions full of space and time. Instead, thanks to the distortions of lame stream science and crooked grant reviewers, we have to fund all those extra shiftless dimensions with our hard earned 4-D tax dollars. #MakeTheUniverseGreatAgain #LasersForPhilosophers

      Reply ↓

    • We don’t even have to stray into philosophy of science proper to see this insight applied to the study of human beings. In *After Virtue* Alasdair MacIntyre wrote of the deep mistake of employing nomological explanations in social theory, as any social “laws” could only hold “in general and for the most part”. In that he’s only following up on insights Aristotle wrote about in 4th century BC Athens.

      The historical division between *Geisteswissenschaften* and *Naturwissenschaften* seems more applicable than ever (setting aside whatever debates we want to have about the status of a “science” employing non-quantitative methods of explanation).

      Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *