JDM now
Dan Goldstein has two posts that should interest some of you:
What is the field of Judgment and Decision-Making (JDM)?
What Judgment and Decision Making (JDM) is and what it isn’t
In the first of these articles, Dan characterizes judgment and decision making as “a field within Cognitive Psychology” with core topics of are “risk, uncertainty, choice, decision, probability, prediction, future, intertemporal choice, heuristics, utility, forecasting, normative models, prescriptive models, and descriptive models.”
He cites a 1996 post from Barbara Mellers, which “speaks of ‘almost five decades’ of JDM research, which would point to somewhere in the late 1940s. Well after Brunswik, a few years after Von Neumann and Morgenstern’s ‘Theory Games and Economic Behavior’ and a few year’s before Ward Edwards’s Psychological Bulletin article ‘The theory of decision making.'” Dan continues by saying that “the majority of JDM research has always been about the difference between formalisms and human behavior.”
In the second article, Dan gives a “concise definition” of judgment and decision making as “The study of intuitive statistics” and a “longer definition” as “The study of human decision making behavior, formal decision models, and the differences between the two.”
Neither of these two definitions quite work for me. My problem with the “intuitive statistics” definition is that, to me, the core topics of statistics are measurement and inference, neither of which directly map to judgment or decision making. My problem with the longer definition is that it’s missing the “judgment” part.
Regarding that last issue, Dan writes:
Does the difference between judgment and decision making really matter?
Judgments (like estimating the distance of an object or the population of a country), and decisions (like choosing medical treatment A vs B given available information and risks) are different, but they’re so related that I find it convenient to roll everything up into “decision making.” . . . JDM or “judgment and decision making” is now a fixed phrase and there’s not much talk about the distinction between judgments and decisions.
To me they are different! You can read chapter 9 of BDA3 for my take on decision analysis. I think it makes a lot of sense to distinguish between “judgment” and “decision making.” Indeed, I think that theorists and practitioners of statistics have made major errors over the years by trying to frame inferences and judgments as decision problems.
That said, Dan works within the field of JDM and I’m an outsider, so I’d guess that his definition is a good summary of what people in that area are thinking about and working on.
His post is pretty long and even includes some data! I recommend you follow the link and read the whole thing.
Origins of JDM
I associate the field of Judgment and Decision Making with the classic book from 1982, “Judgment Under Uncertainty: Heuristics and Biases,” edited by Daniel Kahneman, Paul Slovic, and Amos Tversky.
Judgment and decision making is a subfield of psychology with connections to psychophysics (according to britannica.com, the “study of quantitative relations between psychological events and physical events or, more specifically, between sensations and the stimuli that produce them”) and cognitive psychology (according to wikipedia, “the scientific study of mental processes”).
I was curious who’d coined the term, “judgment and decision making.” It’s a good pairing. In 1988 Jon Baron published a book, Thinking and Deciding, a title that I like because it makes me reflect upon these two different processes. I’ve taught classes on decision analysis, but that’s not the same as thinking. That was before Dave Krantz explained to me about goal-based decision making.
A Google scholar search on “judgment and decision making” reveals multiple reviews on the topic, including a book of articles from 1986 edited by Hal R. Arkes and Kenneth R. Hammond, a book chapter by B. Fischoff from 1988, a textbook by J. F. Yates from 1990, a review article from 1998 by B. A. Mellers, A. Schwartz, and A. D. J. Cooke, a book chapter from T. D. Gilovich and D. W. Griffin from a handbook of social psychology published in 2010, and a review article from 2020 by Baruch Fischhoff, and Stephen B. Broomell, and yet another review article, this one by Priscila G. Brust-Renck, Rebecca B. Weldon, and Valerie F. Reyna from 2021.
That’s not all of it either! It makes sense that psychology is a reflective field, and psychologists like to write review articles. As a serial textbook author myself, I’m not complaining.
I don’t have it in me to read all the above reviews, but it would be interesting to compare the two articles by Fischoff that were written 32 years apart.
In the meantime, I still want to know when the term was first used. Going back to Google scholar, I’ll restrict my search to earlier decades.
For the decade 1940-1950, all I find is a reference to an article by J. Don Miller in The Journal of Business of the University of Chicago from 1947, containing phrase, “Neither college nor university training is conducive to the type of judgment and decision-making required in business.” It’s a readable article! But not relevant to the academic study of judgment and decision making that I was thinking of.
For 1950-1960 we see some references to judgment and decision making in business management and in human-factors research in psychology. So, again, no experiments or new theoretical structure. By 1960, the cognitive revolution was well established in psychology, classical (Neumann-Morgenstern) decision analysis was well established within economics and business, and researchers had started to explore various descriptive and normative problems with the classical approach—but it seems that no one had put these together into a new subfield that combined the mathematics/statistics/economics of decision analysis with cognitive psychology.
As of 1960, “judgment and decision making” was thought of as something done by managers at the workplace, not as its own field of study.
In 1960-1970, things begin to change. In 1961, Michael A. Wallach and Nathan Kogan published an article, “Aspects of judgment and decision making: Interrelationships and changes with age.” This is a serious psychology paper, with theories and data, following in the psychophysics tradition that would become so fruitful when continued by Tversky and Kahneman a decade later in their famous experiments on “the law of small numbers,” “anchoring and adjustment,” and other fallacies and heuristics of judgment under uncertainty. The O.G. researcher in this area was Laplace, back in the early 1800s, but here I’m talking about modern research in the area. Going through the references from the 1960s, the phrase “judgment and decision making” still is mostly used in the business context, but theoretical and empirical articles appear from stalwarts such as Ward Edwards (“Dynamic decision theory and probabilistic information processings,” published in the journal Human Factors in 1962) and Paul Slovic (“Risk-taking in children: Age and sex differences,” published in Child Development, 1966). The subfield is beginning to be formed, but is not cohered, nor has it been named.
The 1970s feature a flood of research papers on the topic. Just from the first page of the Google scholar search, there’s “Studies of problem solving, judgment, and decision making: Implications for educational research” from 1975, “Judgment and decision-making in a medical specialty” (1974), “The concept of weight in judgment and decision making: A review and some unifying proposals” (1980), “Studies of problem solving, judgment, and decision making: Implications for educational research” (1975), “Comparison of Bayesian and regression approaches to the study of information processing in judgment” (1971), “Human judgment and decision making: Theories, methods, and procedures” (1980), and so on.
And then come the 1980s, with the Kahneman/Slovic/Tversky book and all the rest. Hey! Here’s an article in the Annual Review of Psychology from 1984 (“Judgment and decision: Theory and application,” by Gordon F. Pitz and Natalie J. Sachs) that states:
A judgment or decision making (JDM) task is characterized either by uncertainty of information or outcome, or by a concern for a person’s preferences, or both. . . . Numerous authors have demonstrated that judgments depart significantly from the prescriptions of formal decision theory (see Kahneman et aI 1982). An earlier review of behavioral decision theory (Slovic et al 1977) was largely devoted to a descriPtion of these inconsistencies. . . . Since theorists are also human, and hence liable to the same biases as their subjects, there may exist a “bias heuristic” that leads psychologists to see biases in all forms of judgment (Berkelely & Humphreys 1982). The last chapter in this area in the Annual Review of Psychology included a critical discussion of the adequacy of prescriptive models for evaluating judgment and decision making (Einhorn & Hogarth 1981). . . .
In 1982 the newly-formed journal Medical Decision Making featured an article by Jay J. J. Christensen-Szalanski on “Recent Developments in the Psychology of Judgment and Decision Making,” and, as early as 1980, there was a book, “Human Judgment and Decision Making: Theories, Methods, and Procedures,” by Kenneth R. Hammond, Gary H. McClelland, and Jeryl Mumpower (see here for a review).
There is also a Society of Judgment and Decision Making, which began around 1980, and journal of that name, begun in 2006 (of which you are on the board as a consulting editor). See sjdm.org.
A problem with looking for origins is that JDM is like a bundle in which research themes are tied together even though they were around for years, decades, or centuries. The Port Royal Logic, 1662, contained many of the basic ideas. Pascal was a contributing author. And some would trace the field back to Bernoulli’s 1738 paper proposing a logarithmic utility function.
I like to think that the field is partly defined by the three-part distinction: normative, prescriptive, descriptive. So I see it as part of applied psychology, perhaps applied cognitive psychology if “cognitive” is allowed to include emotion and motivation. The normative standard is the ideal. We look for strengths and weaknesses of actual judgments and decisions compared to the standard. Prescriptive solutions are ways of improving performance relative to the standard. Confusions of these three approaches limit the applied usefulness of other fields. The most similar field in this regard is medical science, where “good health” is the ideal (a little fuzzy to be sure, but that’s true in JDM too).
Another textbook that you do not mention is “Thinking and deciding” (5th edition 2024), by me. The title is a bit broader, since “thinking” includes “judgment”, but I generally take the contents of that book to be the subject matter of the journal, with a few exceptions in both directions. (Problem solving belongs elsewhere, and my book says almost nothing about neuroscience … but they have their own journals anyway.)
Jon:
From my above post: “In 1988 Jon Baron published a book, Thinking and Deciding, a title that I like because it makes me reflect upon these two different processes.”
Can you explain further what you see as the mistakes made when failing to distinguish between judgement and decision making? I see them as so connected that they are really the same (as Goldstein is saying). When a judgement is made about evidence it seems to me to be tied to the decisions that are to be made – otherwise what is being judged? I see a clear difference between “evidence” and “decision-making” but the evaluation of the evidence is different than the demonstration of the evidence. One of the problems with NHST is the merging of evidence (such as a p value) with the decision (determining an effect as “real” or not). But what part of the estimation of the evidence are you seeing as judgement? (possibly I answered my own question – if you are referring to the myriad forked paths and model assumptions, then I do see these as judgements, and I also see these are best distinguished from decision making – in fact, those judgements are best made divorced from the decisions which will ultimately be based on the evidence produced).
“Does the difference between judgment and decision making really matter?”
I think it does. Coming from the “normative” side of the normative-prescriptive-descriptive taxonomy that Dan has helped to preserve, I was drawn to the study of decision-making by the promise of a logic with a broader scope than the deductive systems that dominated the attention of formally inclined philosophers and logicians for much of the 20th century. For myself — and I suspect the same of many philosophers who work in what is now called “formal epistemology” — judgments of belief and value are viewed as inputs to the decision, not entirely unlike premises to a rule of inference. Some philosophers, e.g., Isaac Levi, allow for a kind of epistemic decision-making, whereby deliberate decisions are made about what to believe. Some might regard this as a blurring of the line between judgment and decision, but even in a case of epistemic decision-making, the role of the input judgments and that of the decision rule is clear within the context of that decision. I think that those on the more “prescriptive” side of the discipline would also appeal to the importance of the distinction, unless the methods for eliciting probabilities and utilities have changed dramatically since the decision analysis boom of the ’60s.
What a hoot to see my old blog posts discussed!
Re: the difference between judgments and decisions it’s nice to hear Andrew and Jeff’s thoughts. I think my remarking that “it doesn’t seem to matter” was mostly saying that nobody seemed to talk about the distinction inside JDM and people used them rather interchangeably. What people did seem to care about was the difference between JDM and social psychology, which both posts discuss.
Interesting to dig into the history of this. Another strand is as follows. There has been a series of conferences in Europe on Subjective Probability, Utility and Decision Making, SPUDM for short, that have run since 1969, though the first two (1969 and 1970) were officially on ‘Subjective probability and related fields’. The name was changed to SPUDM for the 1971 conference, and since then they have run every two years – what must be the 30th one is later this year in Lucca, Italy. They are now run by an organisation called the European Association for Decision Making. I’ve been to only one, in Darmstadt, Germany in 1975. At the time I was a PhD student in the statistics department at University College London, working on combining subjective probability distributions from different assessors, and my supervisor (Philip Dawid) suggested we should go to Darmstadt. I don’t recall there being many other statisticians there; as I recall, the speakers were mostly from psychology, or human factors engineering, or business schools, but many other fields too. It was my first academic conference and I certainly enjoyed it and learned a lot, but I didn’t continue to work in a related area for too long afterwards so never went again. I did rather loosely keep in touch with the general area though. None of this actually involved the term ‘judgment’, but it was commonly used in talking about decision analysis and the like, often in phrases like ‘probability judgments’ or ‘utility judgments’, so regarding the judgments as part of the process (and definitely subjective) rather than an outcome of a process. In uses like that, I’m not sure it’s wrong to link it with decision making, though as final outcomes, I agree that J and DM are very different. But what do I know – I’ve been out of that research area for many decades now.
This is an insightful round-up (also enjoyed reading the various things you linked to).
I’ll add a quick, different perspective here. Most of the definitions and reflections here seem to start from the assumption that the relevant features of a decision—cues, options, outcomes, probabilities—are already given or at least specifiable. From that starting point, the job of the decision maker (and the researcher) becomes one of comparing behavior to some formal or optimal model: expected utility, probability theory, heuristics, and so on. Even critiques like Mellers’ that call for “moving beyond errors” still preserve the basic architecture, where behavior is either closer to or farther from some model of what’s rational. What tends to get missed is the upstream question: how do agents come to frame problems, recognize cues, or determine what even counts as relevant in the first place?
In almost all JDM experiments, the relevant cues are already known to the experimenter. That’s not a flaw in itself, but it does mean that the most difficult part of real-world judgment—the construction of relevance—is removed from the analysis (in fact, often subjects are distracted by the experimenter, and then said to be “blind” to something they should have seen). Real environments are messy, ambiguous, and full of competing signals. The real cognitive challenge is not choosing among prepackaged alternatives (though sometimes that is useful), it’s constructing the frame within which something becomes a candidate for judgment or decision. That kind of generative salience—in some sense: growing awareness toward what ought to be relevant—is largely outside the scope of current JDM (unless I’ve somehow missed a large chunk of the literature).
Take bounded rationality, for example. It’s rightly celebrated for challenging the idea of omniscient economic agents. But it retains a camera-like model of perception. Simon imagines an agent scanning a known “surface,” with limitations placed on vision, memory, and computation. But the relevant surface is still defined in advance—there’s always an omniscient view-from-above that specifies what the organism is failing to see. What’s bounded, in other words, is access to a supposedly objective environment (the predefined, relevant things). But the presumption of omniscience hasn’t disappeared—it’s simply moved from the agent to the scientist, who gets to decide what counts as relevant and then measure how far the agent deviates (or not). In reality, the environment isn’t pre-labeled or pre-parsed. What gets noticed—i.e. what becomes a cue—is a function of the organism’s nature, interests, and focus (varied top-down mechanisms: questions, theories, etc). Without acknowledging this, we end up confusing unawareness with error and substituting ideal observers for actual agents.
So rather than continuing to ask how well human behavior conforms to formal models (as specified by us scientists), we might ask: what does it mean to perceive something as an input at all? How do agents develop their own models? Judgment, from this perspective, isn’t really about calculation or inference, per se—it’s more about constructing the space of possibilities that make judgment meaningful. That, to me, seems like a frontier JDM hasn’t meaningfully addressed. Well, that’s my quick two cents anyway.
I very much agree that assuming that all of the inputs (including which signals are informative) is too common in work that applies decision theory. Lately I’ve been thinking a lot recently about eliciting decision problems.
When you say, “what does it mean to perceive something as an input,” do you mean what are the “signals”, i.e., the representation of the decision instances, or by input are you also thinking about other components of the typical decision problem formulation? For instance, I tend to think that the scoring rule and best formulation of the state space are harder to elicit than what’s an informative “cue” or signal. For the latter, as long as you can get some sort of external representation of the information humans rely on, you can test how informative different representations of that information are in terms of how much they tell us about the uncertain state, even if the data is unstructured (e.g., freeform text data that the decision-maker has access to).
I tend to side with Dan on de-emphasizing the difference between judgment and decision-making, especially when talking to audiences who are less familiar with decision theory. On the one hand, I do think people naturally associate the term “judgment” with beliefs, which are of course over state space while decisions operate in action space. However, I also find that many people have a very restrictive notion of what a “decision” is that prevents them from seeing how closely entwined judgment and belief are. E.g., they think it has to be a choice between a very small set of options and assume that once you start talking about beliefs there’s no strong link. So I’m often pointing out to people that you can use decision theory on tasks where you might intuitivey think things like scoring rules don’t apply. Not to mention equivalences you can draw, like the fact that for any non-proper scoring rule there is an equivalent proper scoring rule that consists of choosing the best action conditional on the beliefs.
From the post and and comments I see that discussed topic spans far beyond the formalism of making a good sentence aka definition of “judgement” and “decision making”. I cannot give examples (because I don’t have any and also I don’t want to pretend that I have them) – I would just share my subjective intuitive feeling.
I read a book O-Berger on decision making and bayesian statistics and I kind of appreciated philosophically its message. For me statistics are indeed intuitive –
for me it is a science on how to behave rationally (or, at least, striving to do that) in a stochastic environment. My belief is that rationality starts with intuition (nobody ever wrote a theorem without intuitive feeling that it is a reasonable thing) – then either intuition appears to be more-or-less correct and refined using maths or fails completely and again analysis is used to understand why (+ extra intuition). This is to say, that term “inference” does imply some form of intuition from my point of view.
For judgement and decision making – these are two stages of analysis involving posterior inference (not necessarily always bayesian one).
For me judgement happens at the stage of assessing the posterior on parameters of your model – we judge how relevant some parameters are and which are not. Decision making is closer to construction of posterior-predictive, so using our judgements we try to predict something about the real world (the risk function appears only at this stage). The mixing of both terms maybe due to cases when parameters of models may have physical sense and have interpretation in the real world –
these are early posterior checks which are not “posterior predictive ones”.
Thanks for the post. Sorry, I’ve made this answer before reading actually the original link.