Four projects in the intellectual history of quantitative social science

1. The rise and fall of game theory.

My impression is that game theory peaked in the late 1950s. Two classics from that area are Philip K. Dick’s “Solar Lottery” and R. Duncan Luce and Howard Raiffa’s “Games and Decisions.” The latter is charming in its retro attitude that all that remained were some minor technical problems that were on the edge of being solved. In retrospect, I think that book from 1958 represents the high-water mark of the idea of game theory as an all-encompassing tool in social science. Game theory has seen lots of important specific advances since then but its limitations have become clearer too. (See, for one small example, my article, “Methodology as ideology: some comments on Robert Axelrod’s ‘The Evolution of Cooperation,’” published in 2008 but originally from 1986, making a point also made by Joanne Gowa in that year.)

The intellectual history project here is to trace game theory and decision theory from their heights in the 1940s (when game theory and operations research were used by the U.S. military to help win World War II), to Peak Game Theory in the 1950s (when it seemed that we were on the cusp of solving all the important problems of international conflict and social choice, as epitomized by someone like Herman Kahn, who we know recognize as a buffoon but who it seems was considered a serious thinker back in his day), to the business decision making era of 1960s-70s, to the modern day, in which game theory is a particular subfield of political science and economics with continuing developments but no longer viewed as a sort of master key to understanding and solving social problems.

2. The disaster that is “risk aversion.”

I’ve written about this several times before (2005, 2008, 2011, 2014, 2018) “It would be as if any discussion of intercontinental navigation required a preliminary discussion of why the evidence shows that the earth is not flat.”

In particular see this post from 2009, “Slipperiness of the term ‘risk aversion'” and this from 2016, “Risk aversion is a two-way street.”

For the point of this intellectual history, the point is not to criticize naive ideas of risk aversion that are held within the economics profession, but rather to ask how it was that this problematic model became the default. (Just for an example, go to Amazon, look up Mankiw’s Principles of Economics, Search Inside on “averse,” and go to Figure 1 on page 581.)

3. From model-based psychophysics to black-box social psychology experiments.

I think there’s an interesting intellectual history to be done here, tracing from “psychophysics” and “psychometrics” from circa 100 years ago which uses physics-inspired mechanistic or quasi-mechanistic models, to “behaviorism” from circa 80 years ago whose models seem to me to be more Boolean-inspired (do X and observe Y), to “judgment and decision making” from circa 50 years ago whose models were derived from psychophysics, to “behavioral economics” and “evolutionary psychology” which are typically studied in the way that ESP has been studied for so many years, using null hypothesis significance testing with any underlying model being a black box of no inherent interest.

There are two interesting things going on in this particular intellectual history: first, the idea that “social psychology” / “evolutionary psychology” / “behavioral economics” is in many ways a conceptual step backward in psychology, a denial of the cognitive revolution and a return to the black-box modeling associated with the pre-cognitive behaviorism of the 1940s. Second, there is the specific issue discussed in this thread that, although “judgment and decision making” directly derives from “psychophysics,” the mathematical and statistical modeling in modern JDM is typically much cruder than the models used in classical psychophysics and psychometrics.

This came up in this discussion thread.

4. The two models of microeconomics.

Pop economists (or, at least, pop micro-economists) are often making one of two arguments:
(a) People are rational and respond to incentives. Behavior that looks irrational is actually completely rational once you think like an economist.
(b) People are irrational and they need economists, with their open minds, to show them how to be rational and efficient.

Argument a is associated with “why do they do that?” sorts of puzzles. Why do they charge so much for candy at the movie theater, why are airline ticket prices such a mess, why are people drug addicts, etc. The usual answer is that there’s some rational reason for what seems like silly or self-destructive behavior.

Argument b is associated with “we can do better” claims such as why we should fire 80% of public-schools teachers or Moneyball-style stories about how some clever entrepreneur has made a zillion dollars by exploiting some inefficiency in the market.

The trick is knowing whether you’re gonna get a or b above. They’re complete opposites! I blogged about this in 2011 and it’s come up several times since then. The intellectual history question is how this happened, and how this is perceived within the economics community. This is related to the analogy between economics now and Freudian psychiatry in the 1950s, and also related to discussions of the political implications of various social science theories.

There are other questions of intellectual history I’d like to study, but the above four are a start.

51 thoughts on “Four projects in the intellectual history of quantitative social science

  1. There is a lot going on in this post! Two small comments:

    > “psychometrics” from circa 100 years ago

    Psychometrics — which is, more or less, IQ testing as seen in things like the SAT and ASVAB — is perhaps the single greatest achievement of psychology in the last 100 years. No other set of results are as true and important. Why would you lump this in with nonsense like “psychophysics”?

    > game theory is a particular subfield of political science and economics with continuing developments but no longer viewed as a sort of master key

    a) I am continuously surprised by how popular game theory seems to be among undergraduates, given how useless it is, as you point out.

    b) The easiest way to see what a backwater game theory is, relative to, say, statistics, is to note how (almost?) no research result — meaning something published in a journal — is ever used in the real world.

    • Mechanism design is a sub-field of game theory that has been extremely influential in recent years. Auction theory in particular has made billions for Google, helped the FCC to better allocate spectrum, etc. The SEC hires game theorists to help design rules that will be less subject to manipulation by regulated parties. I could go on. Andrew is right that game theory isn’t a master key, but it’s not a “backwater” either. I’d encourage you to get to grips with some of the more recent literature, in particular algorithmic game theory. There’s some pretty interesting research going on.

      • Frank:

        Good points. Above I wrote, “The rise and fall of game theory.” Game theory has fallen since the 1950s, but it hasn’t fallen to zero. It’s at a stable point, just lower than its inflated valuation back in the immediate postwar era.

        • > Game theory has fallen since the 1950s, but it hasn’t fallen to zero.

          On what metric? Here is mine: Direct use of academic research by non-academics. This is very much non-zero in statistics, chemistry, biology and so on. In game theory it is zero. No one reads, say, the Journal of Game Theory and the decides to change what they are currently doing on the basis of the latest academic results.

        • Using your metric, almost all of economics doesn’t qualify. Perhaps fortunately, since things might be worse if it was actually used.

        • I was going to say the same thing… though I think a lot of stuff could be better if some basic economic ideas were used to design some mechanisms, they could also be a lot worse if the ideas were used “against” the public good.

        • > almost all of economics doesn’t qualify

          Untrue! In fact, almost the opposite. Most topics involving government action have large associated econ literatures, articles which are read by and influence policy-makers. Example: minimum wage legislation.

        • I would challenge whether they “influence policy-makers.” Read by and used is not the same thing as influencing. There is little economic work that has actually influenced the views taken by policy-makers, in my opinion. And, getting paid well for providing “advice” can be easily mistaken for “influence.” A rare exception may be the Rheinhart-Rogoff work, and we know where that led.

      • > Auction theory in particular has made billions for Google

        1) Auction theory is not game theory.

        2) This is simply not true. Google would have made as much money if no one there had ever heard of auction theory.

        3) My memory may be faulty, but I believe that the initial ad-words system was designed by (smart!) people with no knowledge of auction theory.

        • D Kane, I work in Google Research and am surprised by your claim that “Google would have made as much money if no one there had ever heard of auction theory.” I agree the initial adwords system was designed by smart people with no knowledge of auction theory, but people with knowledge of auction theory have been improving it for close to two decades, and my understanding is they’ve made a massive difference.

        • Thanks for the inside scoop. However!

          They key info, for many of us, is confirmation that the initial set up was created by people with no knowledge of auction theory. Sort of interesting, right? People with no knowledge of, say, statistics, don’t usually end up doing very good statistics.

          > but people with knowledge of auction theory have been improving it for close to two decades,

          There are many attributes to these “people.” They (mostly) speak English, eat dinner and have two hands. But are those (true!) facts about them the *reason* that the auction has improved. No! Smart people working on thing X will improve it, regardless of their knowledge of French poetry of auction theory.

          You imply that, unlike poetry, auction theory has proved helpful in making Google more money. I doubt it. Name a specific result from the literature in the last 30 years that Google, or anyone else, has made use of. Hard to do! This is not to say that the process hasn’t improved. I bet it had! But it has improved because smart people have worked hard on it, not because these smart people took some auction theory classes in graduate school.

          > and my understanding is they’ve made a massive difference.

          “Understanding” derived from their own testimony? Amy Cuddy is happy to tell you how important a power pose is!

          And, again, I bet they have made a difference! They are smart! They work hard! But none of the nonsense published in the game theory literature has been relevant to that improvement.

    • D:

      Psychophysics is not nonsense. Psychophysics is a lot of the study of perception, and out of psychophysics has flowed many things including much important research in judgment and decision theory.

      As Frank points out, game theory is not useless; it’s just not a master key.

      • > Psychophysics is not nonsense.

        Mea culpa! I misread this. Sorry.

        > game theory is not useless

        We need more precise definitions. My claim: You are in no way worse off if you know nothing about game theory after, say, Schelling’s Strategy of Conflict. This is very different from other fields, like statistics, in which material published in academic journals has a significant effect on the world.

    • > nonsense like “psychophysics”

      I have no idea why you think understanding how sound intensity relates to perceived loudness or how colors look different depending on the lighting and other context conditions is “nonsense”. That’s what psychophysics is—understanding the relationships between physical properties and perceived sensations—and it is still a very active field of research leading to major advances in medical diagnostics and testing (in that sense, it is similar to psychometrics!).

      My guess is that people unfamiliar with the term might think it has something to do with ESP or something (like moving physical things with your mind maybe)?

    • Out of all the things that Psych has done, psychophysics is probably the single most successful. Some important technology that came out of psychophysics is basically all of recording technology, movies, video display, video compression, audio compression, digital signal processing, noise reduction, stereo and multi-source audio… I mean it’d be hard to go about 10 minutes in the modern world without the consequences of some psychoacoustic or psychovisual knowledge slapping you in the face.

    • Psychophysics is stuff like measuring the minimal detectable stimuli… How can you consider than nonsense?

      How hard you need to press your keyboard keys (or touchscreen) to type a response was determined by a psychophysics experiment (even if it was just an engineer trying out a couple things). Given all the BS in psychology, calling psycophysics nonsense is one of the most ridiculous things I’ve ever heard.

  2. Speaking of psychophysics and model-based psychology, another interesting intellectual connection is between cybernetics and information theory and cognitive psychology (first coalescing in the late Dick Neisser’s 1967 textbook). This is interesting not only because these perspectives allowed scientists to bring the kinds of quantitative models that had been developed in psychophysics and psychometrics (especially via Thurstone) to bear on questions about internal mental processing, but because this work, for better or worse, led directly to the development of the neural network (or “parallel distributed processing”) models that now control every aspect of our online lives.

    Two aspects of this are funny to me: First, computer scientists often think that they invented neural networks, forgetting that they are called that because they were designed to model systems that actually are built out of neurons. Second, it is another instance where a modeling framework that was meant to enhance our understanding of the world (specifically minds/brains) has now become a “black box” method. Maybe this is a general trend, where once a model is found to have some kind of “practical” value (as these models now do with modern computing equipment) its internal structures and assumptions get shut off from interrogation and it becomes a black box? Arguably the same thing happened with game theory and prospect theory.

    • +1 for the reference to cybernetics, which is largely forgotten today, but has an interesting (and sufficiently bizarre, to make a good story) history, starting with Norbert Wiener (who, incidentally, was born in my same hometown in the middle of Missouri) and perhaps ending (in some sense, but in a much different form) with Project Cybersyn under Allende in Chile. Actually, when John McCarthy and the symbolic logic crew met in the 1950’s, they apparently chose the term “artificial intelligence” in part to distinguish their line of work from that of Wiener and others. However, the connectionist work and Frank Rosenblatt’s landmark perceptron work (in psychology/cognitive science, largely before CS existed as the field we now know) were perhaps closer to Wiener’s ideas (than to the symbolic logic work), so in a sense, some of the high-level ideas of cybernetics continue today in deep learning.

      I had a small reference to this in the appendix of my dissertation (in the context of the history of NLP) that I may extend at some point into a longer article (and some of the relevant characters from the middle of the 20th century might still be around and it would be interesting to get their take), but it’s fairly low in the priority queue at the moment.

      -Allen Schmaltz

      • > I may extend at some point into a longer article

        Cool! I think this would be a really valuable contribution if you can get the time to make it happen. I read Dupuy’s “Mechanization of the Mind” many years ago, which remains one of the few histories of cybernetics but is also 25 years old and has a surprisingly uncritical/shallow perspective on neural network research at the time it was written (early 90’s, its original heyday). A broader and more modern perspective would be great to have (and great to force students to read!).

      • Interesting, An Introduction to Cybernetics Paperback W. Ross Ashby was probably my favorite text as an undergraduate.

        Also, in my undergrad essay for my semiotics course I argued for “artificial intelligence” as either a replacement or synonym for semiotics. Maybe one reason I got a B- on the project ;-)

        Today, I would argue for artifactual intelligence or reasoning via artifacts rather than in our heads…
        https://statmodeling.stat.columbia.edu/2018/01/04/intelligence-always-artificial-least-artefactual/

  3. Loss aversion, the persistence of disproven concepts and the race of teachers:

    A while back there was a diatribe in our newspaper claiming that there is racial bias for hiring white teachers because the race profile of teachers doesn’t match the race profile of students. The innocuous reason is obvious: students are in the system for <12 years while teachers are in the system for 40 years. Obviously the racial profile of teachers will always lag that of students.

    The same goes for concepts like "loss aversion". New ideas are like the student population. Old ideas are like the teacher population. People entering the (education/professional) system are taught a particular concept. That concept is built into the framework of their overall knowledge. So unless those people are always actively rebuilding that segment of their knowledge framework, the erroneous concept will persist as a pillar in their thinking. It will persist all the longer if it provides an apparently simple explanation for a universal phenomenon, because this will make it easier to reject any replacement idea.

  4. On 4) it seems like this is just evaluating any behavior or system through the lens of rationality and either saying it is good or bad. When people behave in interesting ways it seems useful to explain some underlying mechanism/model that drives their behavior, when you think in terms of economic rationality or efficiency (maximizing behavior, however labeled) then any analysis will look like the two “opposites” you have identified. In (a) people are actually maximizing some kind of welfare, when you look at it in this plausible way, even though their behavior might seem odd/not maximizing; in (b) people are acting in an intuitive way but really they cannot be maximizing some good thing when you look from XYZ reasonable/economics favored perspective. It’s all the same paradigm (maximizing behavior under some model of utility), the only “opposite” is in the conclusion, which seems pretty defensible, either your model approves of some behavior or it doesn’t.

  5. “The usual answer is that there’s some rational reason for what seems like silly or self-destructive behavior.”

    That is not the reason. Economics is about tradeoffs. Since we live in a world of scarcity we cannot have it all. No matter what action we choose for ourselves, once the choice is made the alternatives are no longer being pursued. As the Austrian School economists, point out, humans act. And each time they act they choose what they prefer most to the other alternatives. Start from that point and economics becomes a great deal clearer.

    If you want to pursue this, just search the term praxeology and take a bit of time to learn.

    • “And each time they act they choose what they prefer most to the other alternatives.”

      I think “the other alternatives” needs to be expanded somewhat — e.g., to “the other alternative that are available to them at the time the choice is made” (For example, their choices are limited by things like income and availability of options.)

    • I’m sure Andrew is aware of the concept of trade-offs in economics and/or behavioral economics. Just the same, in order to understand the trade-offs we have to know all the options, and that’s always going to be the rub in behavioral economics: how can you know all the options? How can you control all the variables? The fact that economist-you can dream up some form of study to measure or calculate the value of the necessarily restricted set of choices that economist-you recognizes hardly implies that economist-you has recognized all the potential controlling variables much less assessed their relative values correctly or used an appropriate proxy to measure value.

      • Yep, at each choice, people choose from among a fairly limited set of options that they can load into their short term memory and compare… They also choose on the basis of limited information about what the outcomes will be. Some choose with little or no education that would let them recognize the reality of how things work. Just think about the average person walking into a financial advisors office, or their tax preparer… Resource constrained calculations are an important part of understanding economics.

        That being said, I have a kind of fondness for the Austrian economists. In aggregate, with a reasonably well educated citizenry, you might expect total resource allocations to get close to optimal in some reasonably achievable circumstances even if individuals fail to hit their own optimum. In math, we have the concept of convergence in mean compared to say convergence pointwise. There’s no question that pointwise optimal allocations will not occur. But on average things might do much better.

      • Do behavioural economists not know about using heuristics to make decisions under resource constraints?

        That’s why 90+% of decisions are people doing what they see other people do or what whoever they got convinced was an “authority” told them to do. This *is* rational, btw. You do not have time to learn everything about everything before making a decision. But it is not foolproof (just look at NHST), you need to sanity check your heuristics at least every now and then.

        There is a giant industry designed to take advantage of this. Actually, I would say it is by far the biggest industry encompassing government/politics, world finance, healthcare, and (of course) advertising and marketing.

        So basically you seem to be implying that there is a field of economics that ignores the most common and important economic phenomenon in the world.

        • I think Behavioral economists know about it, and even regular economists know about it, but the actual use of this knowledge in development of a modern economic theory is pretty minimal in terms of applications to things you might see in say Econ 101 or even maybe 201.

          What role does intentionally misleading people about what good or service they are buying play in the market today in terms of prices, supply, demand, etc? How does having friends who are hard to dupe play into how easy it is to dupe you? How does this affect the distribution of wealth?

          Why do we have a giant industry of click farmers in Bangladesh and Indonesia? How does this affect real GDP as measured by the dollar value people would be willing to pay for products and services they are buying *if they had accurate information about what the product or service was actually going to do*?

          Good luck finding that in a textbook. There may be some cutting edge researchers interested in the issue. But I think the issue has been around forever. Certainly the advertising industry in the 1950’s or so was doing qualitatively similar shenanigans (Payola anyone?)

  6. Ha, ha, I missed #4 the first time through! That’s a great one. I’ve noticed that too!

    With (a), any chain of reasoning no matter how contorted not only *can* provide an explanation for Mystery Behavior X, but is presumed to be the **absolute indisputable truth** once Reason Y is “discovered”! It’s not *a possible* reason dependent on various shaky assumptions, it’s *the confirmed* reason regardless of the assumptions. Especially once it gets into the press.

    (b) is the recipe to provide a scientific foundation to support policies aimed at incentivizing some “moral” or “appropriate” or “desirable” behavior, as understood by a particular political fashion or activist group. This has elements of the hero scientist meme.

  7. The claim that game theory peaked in the 1950s is strange to me. Games with incomplete information — which are often the most interesting — weren’t even introduced until the late 1960s, and my impression is that modern tools to study such games weren’t developed until the 1980s.

    But maybe I’m misinterpreting what Andrew means by “peaked”?

  8. Your characterization of behavioral economics is really quite off — at least concerning behavioral economics as done by economists in the past decade (as opposed to, perhaps, when psychologists or marketing scholars use the term). The Null hypothesis is usually the standard model, which is extremely widely used, and thus of intrinsic interest. Alternatively, the Null hypothesis is a behavioral model that was built on previous empirical deviations from the standard model. Non-conceptual, anything-goes type of experiments don’t fly, certainly not in econ journals (though, admittedly, there was a bit of a bubble 2005-2010 when such things sometimes got published). A good overview of the current state of behavioral *economics* is the Handbook of Behavioral Economics (https://www.elsevier.com/books/handbook-of-behavioral-economics-foundations-and-applications-2/bernheim/978-0-444-63375-0, https://www.sciencedirect.com/handbook/handbook-of-behavioral-economics-applications-and-foundations-1/vol/1). Each chapter is available for free on google scholar.

    • Sandro:

      If you go to the wikipedia page on behavioral economics, you’ll see a section on nudge theory, which is all about purportedly big effects estimated from black-box experiments. So I think some of this bubble is still inflated, even if the nudgers like to attack their critics by calling them Stasi etc.

      But I agree that different economists have different perspectives, and it could well be that this sort of thing is not favored by many in the field of behavioral economics. This implies that I should include another twist in my item 3 above, to cover the shift back to theory-based models. So that’s good to know.

      • You’re absolutely right regarding nudge theory; that fits your description in point 3 very well. Many, if not the majority of behavioral economists (with an econ PhD) are very critical of “nudging” as a concept, starting with the fact that there does not even exist a precise definition of what a nudge is, or what it really means for “people to be better off as judged by themselves”. It’s perhaps this vagueness in the definition that gives rise to the theory-free, anything-goes studies that are sometimes done in the nudge-domain. Moreover, the fact that many studies on nudging are done by practitioners (e.g. nudge units) might also be a contributing factor.

        Nudging, however, is just one of several parts of behavioral welfare economics. Behavioral welfare economics, in turn, is a relatively small part of behavioral economics as a whole. It is those other subfields of behavioral economics to which I don’t think your description in point 3 applies.

        A really good recent example (out of hundreds) is this: https://www.aeaweb.org/articles?id=10.1257/aer.20170866. Standard econ theory implies that handing out food stamps instead of cash transfers of the same magnitude would not change behavior, because as long as one spends more money on food than what one receives in food stamps, one just pays for the food one would have bought anyway with the stamps instead of with cash, and uses the cash saved that way for whatever one wants (see e.g. here: http://www.economicsdiscussion.net/essays/economics/food-stamp-programmer-in-kind-food-subsidy-explained-with-diagram/1165). This has been taught in undergrad econ classes for decades. Hence, it is a very relevant null hypothesis. Hastings and Shapiro show that this theory is fundamentally wrong; handing out food stamps instead of cash has substantial effects on the consumption basket of consumers.

        • Indeed, a reason to just give people cash is specifically so that they will do the more efficient thing and treat all spending as equivalent. Also because “special accounts” is a way to get rent seeking behavior. Why do we buy so much healthcare in the US? Because we have a “special account” from our employer to pay for it with (actually multiple special accounts, FSA, HSA, “insurance” etc). Why do glasses cost so much? because in addition to having a near monopoly on production by Luxxotica, we also have “special accounts” that can only be spent on glasses… so all the frames cost exactly $1 less than the maximum covered amount regardless of type, quality, etc etc

          The inefficiencies caused by all of this make everyone poorer. Poor people would be better off if they got cash and could decide for example to fix their car’s breaks so they could get to work reliably, and go get more charity from a local food bank rather than have to buy only food and not be able to use the food stamps to get their brakes fixed so they could get to work and keep their job…

          Unfortunately I tend to see these kinds of econ papers supporting the idea that “we can effectively manipulate behavior by doing special thing X which classical econ theory would tell us wouldn’t work” as if that were an OK or even good thing, as if the proper role of economists was to come up with effective ways for government to manipulate its citizens. I find that disturbing.

        • If you’d want to describe the dominant view amongst economists at North American econ departments, it would certainly be this: “The inefficiencies caused by all of this make everyone poorer. Poor people would be better off if they got cash”. That’s for instance, why economists founded what, to the best of my knowledge, is the only non-paternalistic charity with recipients in developing countries: https://www.givedirectly.org

          Moreover, the econ papers (in econ journals) usually try very hard not to make any normative statements at all. There’s a very practical reason for this — it’s close to impossible to publish a paper that makes normative statements. The reason is that regardless of what normative statement a paper tries do defend, it will be fiercly criticized by a reviewer who takes the opposite view, and you can be virtually sure of having such reviewers.

          But some economists are much more publicly visible than others, and the public tends to like strong normative statements much better than nuanced statements about objective facts. So the stereotypical economist a non-economist would think of would probably be quite far from the median economist on these issues — be it that people think of Cass Sunstein as a representative behavioral economist (he is about the most pro-nudging person you can find anywhere) or of Milton Friedman or other typical Chicago-ites as representative non-behavioral economists (Friedman is very far to the right / libertarian side on the distribution of economists as a whole).

  9. game theory had two peaks in economics. An initial one in the 1950s, which failed as you described and then another, beginning in the 1970s, which started from the idea that cleverly selected solutions to repeated games would solve the problems. This produced a huge literature on “refinements of equilibrium” which went nowhere (there was no round the “folk theorem”, that any individual rational outcome could be supported as an equilibrium). But by the time this was realised, game theory had re-established itself in economics.

  10. A serious question about 4: wouldn’t this be an issue in an sort of social science that wants to have practical effects? It seems to be more or less the “floating philosopher” problem people criticize Marx for, that it’s difficult to have a complete theory explaining human behavior that allows the possibility of radically changing things.

    A less serious response to 4: suppose people are rational and respond to incentives, but for some policies it is difficult to figure out what is the best course of action. What to do in these situations? Why, hire University of Chicago economists, of course! Their very existence transcends the apparent paradox.

Leave a Reply to Sandro Ambuehl Cancel reply

Your email address will not be published. Required fields are marked *