This is Jessica. What influences how people make decisions in specific situations, and how they should be making decisions, are questions that have been asked for centuries. The best known theory that’s been proposed for studying decision-making is statistical decision theory. We define a decision problem as a choice of action from some action space (e.g., take an umbrella versus don’t), the quality of which depends on the realization of some state of the world (e.g., whether it rains or not), which is assigned by a utility function (e.g., biggest penalty for leaving umbrella when it rains, smallest when the action matches the weather). In the Bayesian variant of decision theory, we assume a decision-maker who knows the prior probability of the state, encounters some information signal, updates to posterior beliefs (ideally like a Bayesian), and selects an action with the goal of maximizing their expected utility.
One benefit of approaching decisions this way is that the assumptions under which the optimal decision rule can be fully characterized are well defined. And if we are willing to buy them (at least in the idealized case), we can upper bound any decision-maker’s performance (in terms of expected utility) by conceiving of the ideal Bayesian rational decision-maker’s performance, and use this and related constructs to get insight into all sorts of questions related to what people appear to be doing and how information relates to tasks.
However, lately it seems to be fashionable lately to reject decision theory, on the grounds that it is, in one way or another, unrealistic or even offensive in its implications. This is worth unpacking.
There are many informative critiques of decision theory (a good short summary can be found here). But some are less well-constructed. I suspect some of this recent negativity arises in an attempt to push back against the attitude that just about every decision process is a candidate for AI. The idea that we must reclaim our human agency by acting in ways that go against expectations, whether self-imposed or societal, is a palpable theme lately, including in statistical discussions. The impetus appears to be to prevent getting run over by corporate or political interests or something worse.
Here are some common kneejerk reactions to be careful with:
We can’t come up with an agreed-upon utility function or ground truth. Naturally, you shouldn’t use decision theory if you don’t think it’s possible to define ground truth in a situation. If this is the case however, you shouldn’t be trying to evaluate decisions at all, which is a point people sometimes seem to miss – instead they reject formal theories of decision making in favor of something “looser” but then still end up making evaluative statements, it’s just less clear what sort of commitments they are making to get there. For example, we recently observed that many papers studying AI-assisted human decisions draw conclusions about how good decisions are without clearly defining what they considered optimizing, or giving study participants a clear problem to optimize within.
When critics reject the idea that we could define a single utility function, this may be a clue that the critic is expecting the method to be an exact description of reality, rather than seeing it as an approximation that can valuable for building understanding or intervening to improve things. I see this frequently among researchers who study human behavior but reject mathematical theory (e.g., some of the human-computer interaction community), as well as people who understand math but have never tried to understand human behavior in any rigorous way (e.g., some of the ML community). There is no rule dictating that you must analyze a given problem using only a single decision framing. If you don’t know what the utility function is, look at what happens over a class of utility functions. Or choose a few candidates that seem plausible and contrast expected behavior across them. Decision theory is a method, not a description of total reality. Apply it in as many ways as you think will help you think through trade-offs in a situation.
Related to this, there is often a lot of value to be gained simply by trying to formalize a problem. It can be eye-opening for getting a better understanding of the trade-offs that exist in a situation, even if you ultimately decide it’s too hard to pin down a solution. When we shoot down the method because it doesn’t seem perfect, we deny ourselves the insight we might gain from at least attempting to define the problem in a well-defined framework. Same goes for preregistration – much of the value comes through the attempt, so rejecting it because it’s imperfect in practice can miss this.
But people are not rational! The more informed critic who says this will point out that people often deviate from Bayesian updating, or they violate axioms like transitivity or independence (e.g., Allais and Ellsberg paradoxes). They explain all this as if you, the naive rationalist, had no idea that people could deviate from assumptions. But all you really learn from such critiques is that the critic is unable to separate descriptive and normative functions of a theory.
That people do not conform to the assumptions of decision theory is not a problem if you interpret decision theory for what it is: a form of logic for dictating internal consistency of choices under uncertainty. The prescriptions it provides are a comparison point, or benchmark, for better understanding where human heuristics depart from logical coherence. Sure, we could proceed without any attempt at defining consistency with available information and statistical notions of optimizing. But then we will be limited in what we can conclude.
It’s like the fact that decisions are thought of as a more “human” endeavor (compared to say, making predictions or estimating a parameter) makes some people think we must avoid any attempts at statistical formalism to describe them. But anyone who has ever tried to seriously study human judgment and choice should know how hard it is to learn if you have no well understood benchmark to compare behavior against. We learn by checking observations against model predictions of some form. Decision theory may not be a perfect model but it’s one we can motivate from the ground up from foundational principles.
Decision theory assumes complete information, well-defined preferences, unlimited cognitive capacity, and all of these things are false in typical human scenarios. This is all true, and yet how we don’t usually know incomplete information might be for the agents deciding, or how poorly defined their preferences might be, or how limited their cognitive capacity. So instead we create an upper bound based on processes we understand well. We can also use the notion of the idealized agent under specific constraints (e.g., only has access to the prior, or constrained to act like humans in particular ways) to do things like measure the “value” of specific information like visualizations, or explanations, or decisions from other agents to a decision problem (which can be useful in experiment design, and to optimize incentives), or the cost of specific types of errors. Along the same lines, we can use it to better understand what kinds of problems we implicitly assume when we choose to communicate data in different ways. See, e.g., Dean’s work on using decision theory to contrast approaches to communicating effect size.
By using decision theory you are asserting that you know more. There’s a common kneejerk reaction that because you have used decision theory, you have claimed to know the truth, much more so than the person who does a sloppy job describing what problem they think they’re solving. And so you must be bucked from your high horse. What this inevitably overlooks is that in reality, any way that you go about evaluating people’s decisions is asserting some decision problem, the only difference is whether you are making it explicit. If you haven’t tried to formalize the objective and the assumptions, you are going to have a hard time articulating the limitations.
What’s funny about this reaction is that the critic is often implying even more confident assertions than the ones they are trying to reject. When someone tells you “decision theory cannot be useful here,” what else are they saying other than that they know best about what the right approach is?
People who use decision theory are immoral. Some people seem to associate decision theory with being socially immoral or irresponsible. I associate this with far left perspectives that optimization is anti-humanist. Some of this is undoubtedly inspired by the various cases where applying algorithms to make decisions about people did leave some groups worse off. The risk is when this reasonable concern becomes a blanket excuse for researchers to reject all attempts at optimization as immoral or elitist.
I once had someone in an audience get really angry at me during a talk where I argued that we won’t make much progress trying to improve how people use predictions for decision-making without attempting to formally specify the problem we think they are solving, and that decision theory was made for that. It was like he took my suggestion that decision theory was more appropriate than other arbitrary theories as an attempt at asserting moral superiority and limiting his agency.
Flashback to a hundred years ago or so, and think about how we made sense of people’s decisions then. If you wanted to describe why people chose to act the way they did, you might say it was because they had “instincts” or “habits” or “drives.” If you wanted to say what they should do, you might appeal to discipline, or adaptation and learning, or moral virtue. Decision theory was a move away from mystery and faith. Of course, any method can be used religiously, but then we should be attacking the specific use, not the theory at large.
Ultimately, I’m guessing we all agree that it’s unlikely that any single model or theoretical framework will ever provide a satisfying explanation of how people make decisions, or a universal description of how they should make decisions. To hate decision theory with a passion, however, is often evidence of taking a model too seriously. People do this all the time, so I guess we shouldn’t be surprised. But we should be wary of how conversations about methods can become politicized in ways that ultimately lead to good methods being kicked aside in favor of tribal thinking.
I have an objection that I don’t think is covered in what you say. I certainly believe you can specify a utility function and have an AI make decisions for many cases (not necessarily all). I also believe that these AI decisions will often be “superior” to human decisions in many cases. But I still want humans to make some decisions because they will then be accountable for those decisions. Many of the most important decisions involve hard choices where some people will lose, and some may even be harmed. While you can program an AI to make these decisions optimally (again, given that you can specify an appropriate utility function, which I am willing to agree is often possible), I think such choices should be “felt” by decision makers. That is, they need to be accountable to both themselves and to others. I think that is a critical part of what it means to be human. When we cease to feel how our decisions impact others, then we lose a large part of our humanity.
Now, you may be thinking that humans will still be accountable and feel these things, even if they use an AI to make decisions. If that is the case, then I think we need to define our terms more carefully. I am thinking of AI decision making as some automated process where the human is removed once the utility function has been specified and the AI has been programmed to make decisions. If that is not what you mean – if you are thinking of the AI decision process as simply an aid to the human decision makers, then I don’t see how it is any different than using a calculator to manipulate some relevant numbers. I can’t see any objection to using an AI as decision “support.” I have objections to having AI make some decisions automatically. And I don’t object to all automated AI decisions, but only to ones where a human sense of responsibility is important.
How would you classify my objection in your panoply of critiques?
This was already largely lost before AI. Bureaucracies (government/corporations) are already the “sociopathic” decision making entities. It is most obvious when you look at things like “war on drugs”, “war on poverty”, etc where we end up with *more* of the thing because this perpetuates the bureaucracy created to “fight” it.
The bots will make this more efficient and remove last vestiges of humanity, but its a quantitative rather than qualitative difference.
This is bleak, but I’m afraid it is indeed a largely correct picture. See for instance, Davies’ “The Unaccountability Machine” (machines here not being literal AIs, but large organisations, public or private).
But I do think that large quantitative changes are a type of qualitative change, and we should brace ourselves to where the direction of travel is taking us…
«Davies’ “The Unaccountability Machine” (machines here not being literal AIs»
To me however interesting as to the stories that book is an astute whitewash of many institutional mechanisms as “they just happen to do that” because in the a large majority of cases (including most examples the author makes) it seems pretty clear that each “unaccountability machine” makes lots of money for a vested interest group that has either designed it or protects it so that it continues to make money for them.
«but large organisations, public or private).»
DeLong has been calling organizations like corporations or states “slow AIs” for a long time IIRC from a similar concept by some historian.
Hi Dale,
Accountability is often a reason cited not to automate decisions. On having the human “feel” the decision, it reminds me a little of arguments we make in this paper about “actor-specific” decisions, where sometimes its part of the human decision-maker’s identity to have the agency to apply their expertise
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5279401
I could see how in certain professions like medical care-giving, accountability in the sense of feeling the impacts of decisions is expected as part of the practitioner’s role. But I think there will still arise difficult trade-offs between that kind of care perspective and the desire for efficiency or making decisions in ways that minimize harm over a population.
As Anoneuoid says, even without AI many organizations are already using bureaucratic optimization in some form to dictate decision rules. So it isn’t always a big change once AI gets introduced.
Jessica
That helps, and your linked paper goes part way towards addressing my concerns. One point remains contentious in my mind. Your contrast between actor-specific decisions and the organizational decisions (that are not so unique) seems to hinge on relative performance. You see AI decision-making as inappropriate for many actor-specific decisions because they do not include the many nuanced and unique factors (perhaps what I refer to as “feel”) in such decisions. But that seems to imply that, as they improve, AIs should do an increasing share of such decisions. Part of what I mean by having decision makers “feel” the consequence of their decisions is not because this will lead to better decisions (though it might). It is because I want decision makers to feel responsible for the consequences of their actions.
Let’s imagine a concrete situation. A policy decision must be made about the extent to which families will receive subsidies for purchasing health insurance. This is not an actor-specific decision. It is amenable to decision analysis and I think we would agree that such analysis would be useful. There will be difficulties agreeing on an appropriate utility function, but I think it is feasible – and desirable, since that discussion of relative values in important. Once such a function is established we could then examine alternative subsidy schemes. Inevitably, some people will benefit and some will lose out. Even if we can establish that there is a clear net gain from alternative X, the best we can hope for is that some of the losers as a result of X might receive some compensation for their losses. I can think of no cases where a policy of such magnitude will actually result in everybody becoming better off, regardless of the potential to do so.
If we use AI to automate this policy decision, I think something is lost relative to having responsible decision makers use the decision analysis to inform their decision. That is because I think it is important that whoever makes the decision, retain awareness (and accountability) for that decision. Judgements were made and I view judgement as a human activity – to do otherwise, removes what I view as an essential aspect of being human. Once, we allow the decision to be automated, we lose touch with the fact that judgements were made in choosing the utility function and enacting the policy.
Anon’s statement (that you refer to) that this is not unique to AI is absolutely correct. But I don’t find it reassuring. We are already well down the path of dulling human responsibility, judgement, and accountability. I am by no means saying that AI, decision analysis, and other tools should not be used. I use all of these myself and think they should be more widely used (in that, I share many of the concerns you expressed about the arguments made against decision analysis). But I am saying that automation is to be used sparingly. I rely on my thermostat to automatically maintain my home temperature – there is an implicit judgement about comfort, it varies across members of my household, but there is a direct connection between where I set the thermostat and how members of my family feel: when my spouse complains it is too cool, I don’t respond that it was the thermostat’s fault. However, when a decision analysis says that health insurance subsidies should be cut off at double the poverty level (compared with other potential cutoffs), it is no longer clear to me who is accountable for that decision or who feels the connection from their decision to its consequences. Even if that decision is the “best” one, I think it is important that those who implement the decision be accountable for it. Too easily that gets lost in the process of automation.
“… far left perspectives that optimization is anti-humanist.”
I’m old enough to remember when critiques of rationalization were coded as conservative. See for example Oakeshott’s famous critique of rationalism in politics. How times change!
Ryan:
Relevant to this discussion are two old posts on left and right attitudes on rationality:
from 2005: Rationality and ideology
from 2005: Contingency and ideology
from 2012: The politics of economic and statistical models
I’m planning to write more on the topic soon.
You are preaching to the choir here, so I will only say that these are objections that only appear to arise (in my experience) in a sort of academic, self-flagellating setting. There are a great number of decision-makers in policy settings – medicine, business, government affairs, etc – who are going to make decisions anyway, and we might as well formalise that process so that they can consistently apply their heuristics in a way that is more justifiable in the face of a transparent decision rule.
As I see it, policymakers are going to make decisions anyway, and those of us working in decision science have a duty to make sure those decisions adhere to the principles we want from good governance. This extends to the algorithm-assisted discourse; the field of model-derived decision support seems to dust its hands at the point of model development/validation without proposing a mechanism by which the model actually improves some terminal outcome, so forcing people to actually define a utility function that is open to criticism is an important first step towards figuring out what actually leads to benefit.
My intuition is that many people critiquing decision theory have been pulled into it by the gravitational force of the AI discourse – everyone from a variety of different fields is encountering people they would never otherwise speak to on a professional basis – and that is causing a lot of friction.
“so I will only say that these are objections that only appear to arise (in my experience) in a sort of academic, self-flagellating setting.”
Not my experience! I’m a dirty no-good economist, and these are pretty much the same objections I’ve been seeing tossed at the economics professor for as long as I can remember, almost always lobbed from outsiders who clearly hadn’t had their views refined by the back-and-forth that insiders had been exposed to.
That’s quite interesting – it’s only ever been academics from other disciplines that I’ve seen object to decision theory with the qualms in the OP. Who have you encountered that’s had those same complaints? Politicians?
This discussion reminds me of a project I worked with Dept of Fisheries and Oceans in British Columbia. We were scheduling when specific locations of the salmon fisheries should be open to commercial fishing, First Nations, and sport fishing. The goal is to meet competing goals of size of the catch and population preservation.
At one contentious meeting, someone told me that my modeling was worthless because we could not measure the salmon populations used as inputs to the model. He was correct that we used highly variable estimates as inputs, but without a model he was making even more guesses about how to manage the fishery.
Models are useful because they make people declare their assumptions and work through the impact of those assumptions in a clear way.
Decision Theory is useful because its not a theory but provides a structure to make decisions. To begin that process people need to agree on objectives and utility or at least agree to disagree in a way that leads to objective criteria and not just “feels”.
Yes to this. I think Andrew has paraphrased one of his professors — I’m going to guess Rubin because that’s who he often cites — as saying the main value of a decision analysis isn’t the results (which often depend on modeling assumptions you don’t really believe and maybe on data you don’t really trust), it’s the fact that you are forced to make your assumptions and value judgments explicit and that tends to lead to better decisions even if you don’t trust the quantitative results.
“Models are useful because they make people declare their assumptions and work through the impact of those assumptions in a clear way.”
Models are supposed to be for making predictions, approximate or less approximate (your salmon model may be useful for making order-of-magnitude approximations, but not more).
When your model has a very small number of data points or very noisy data points, it is not actually much better than a vague guess. It’s easy to get overconfident.
“Models are useful because they make people declare their assumptions and work through the impact of those assumptions in a clear way.”
This is the economist’s way of doing things. They describe it as “organizing their thinking.”
Economists must have very disorganized minds if they need to do this. ;)
I am part of the choir. But I think it is useful to make an additional distinction. In addition to normative and descriptive models, we also have prescriptive models. Formal decision analysis, of the sort advocated by Howard Raiffa, Ron Howard, Ward Edwards, Ralph Keeney, Detlof von Winterfeldt, and so on, is prescriptive. A lot of it looks like normative models, but those are idealized “as if” standards. This view is consistent with what you say. But I think that some of the objections are to the use of prescriptive models. Setting up a decision analysis is a problem of design, and alternative designs may be debated without challenging the normative models that often serve as background justifications.
Yes, I agree, it’s useful to distinguish disagreement with prescriptive use of it from disagreement with the normative perspective due to problems with axioms, etc.
Recently we had a paper where we encountered criticism of the ‘decision theory can’t be useful because its unrealistic’ variety despite showing that using it prescriptively led to observable improvements in payoff by human decision makers. This is the kind of response I struggle to make sense of.
My supervisor always tells this, supposedly true, story:
[A famous economist] was contemplating whether to take the new job he had been offered. He asked his colleague for advice.
“Easy”, said the colleague. “All you need to do is write down the possible outcomes and their probabilities…”
“Oh come on, this is serious!” said [the famous economist].
I agree that decision theory is often extremely useful, but I think whether it provides a *normative* solution should be evaluated on a case by case basis.
Mattias:
Yes, this is a famous story. And I think it misses the point, which is that decision analysis can be done in a “forward” or “reverse” fashion. In forward analysis, you state all your assumptions clearly and then evaluate the decision tree and pick the optimal decision. In reverse analysis, you apply forward analysis under a range of assumptions, and then if the recommended decision is not the decision you want to take, you go through step by step and figure out where your assumptions are wrong.
The point is that coherence of decision analysis is a feature, not a bug. But coherence of decision analysis does not imply coherence of decisions. Formal decision analysis can be a method for making decisions (what I’m calling the forward application), but it can also be a method of interrogating your assumptions. In the real world we will typically do both.
The other thing is something that I call institutional decision analysis. The point here is that often we don’t just need to make a decision, we also need to justify the decision:
So, yeah, the example of one guy making a one-time decision for himself is not the ideal application of decision analysis, because there might not be anyone he needs to justify the decision to.
Andrew:
I agree.
The point was that applying decision theory to “one guy making a one time decision for himself” is a *common* use of it – despite not being “the ideal application”.
For example, large parts of the heuristics and biases literature relies on that approach. (See Gigerenzer’s critique of “narrow norms”).
On the other hand, there is a research tradition that basically says that if your decision theory doesn’t capture the behaviour of participants, then that indicates that you haven’t modelled the decision problem in the way the participant conceives of it (“rational analysis”, Anderson). This is psychology’s version of what you call reverse analysis, I think.
That’s why I think it is often extremely useful (for reverse analysis) but that you need to evaluate whether it provides the normative solution on a case by case basis (because there might be something silly with the assumptions of your forward analysis).
But my supervisor would put it more eloquently.
Mattias:
Yes, I’ve long been annoyed that decision analysis textbooks often focus on individual decision examples such as “Your nephew is deciding where to live and is choosing between various dormitory and off-campus options” rather than on institutional examples such as, “Your company is deciding where to situate a new plant, or whether to build it at all.”
Jessica:
I agree that decision analysis is a valuable tool. We give several examples in chapter 9 of BDA3, also see what I wrote in the comment just above.
Of course decision analysis is valuable. But that seems a long way from the original post:
“I suspect some of this recent negativity arises in an attempt to push back against the attitude that just about every decision process is a candidate for AI. The idea that we must reclaim our human agency by acting in ways that go against expectations, whether self-imposed or societal, is a palpable theme lately, including in statistical discussions. The impetus appears to be to prevent getting run over by corporate or political interests or something worse.”
I took that to mean automated decision making. Maybe I misunderstood – but if the question is whether decision analysis is useful, then I think the original statement that people reject it is unfounded. Do serious people actually claim that decision analysis should not be done? Does anybody really say that AI should never be done? (I don’t even think David would say that). Does anybody seriously say that statistical analysis should not be used?
If Jessica’s post means anything, in my view, it relates to having decisions made on the basis of decision analysis, not whether such analysis is useful at all. When faced with a decision, can we deviate from what the decision analysis says should be done? If that is not the meaning, then I don’t see the point of the post and the supposed objections to decision analysis. I’m sure there are some people who will reject it completely, just as some people believe no data should ever be analyzed (and, as I’ve learned on this blog, many people believe in ghosts). But I think the serious objections to decision analysis concern how it is used, not whether or not it should be done to begin with. I think people who say it is “immoral” to use decision analysis object to it dominating the decision process, not its mere use.
>Do serious people actually claim that decision analysis should not be done?
I routinely encounter attitudes (often in paper reviews) like what I described in the post. Very often it is stuff like “this framework seems useful, but decision theory is unrealistic,” as though it can’t be useful unless we buy every assumption. Like I say above in a comment, we get this sometimes even after we’ve demonstrated that using it prescriptively can lead to measurably better utility for the problem framing we use.
So yeah, it’s hard not to conclude that many people just don’t want to see decision theory. I suspect it’s a learned association given that the reasons that are cited are often along the lines of “it can’t be realistic because real world decisions are complex.”
Whether this kind of negativity means that these people think no decision analysis should be done at all… hard to say. Maybe they would be content with some kind of looser, vibey notion of decisions. To some people (at least in computer science) research is more about putting exciting ideas out there and less about being right.
I was corresponding with Baruch Fischoff the other day on an op-ed he wrote in the Washington Post and he pointed me to his forthcoming book on this topic: https://mitpress.mit.edu/9780262553162/decisions/ At some point in the near future it will be a free download via open access (or you can purchase it right now if desired).
Hmm. My objection to the whole decision theory schtick is that it ignores personal preferences. Also, it’s _attitude_ is that it knows better than you, and that’s friggin obnoxious. It doesn’t.
For example, I’m _seriously_ risk averse. I don’t gamble, don’t buy stonks. No matter what the odds. I remember thinking I had flunked an IQ-like probability (or something) quiz, since one of the questions was “Do you take $100 today or $200 next week?” (I don’t care who you are, promissary notes ain’t worth the paper they’re written on.)
There’s got to be a zillion personal reasons for every decision that decision theory can’t model. Some uni/company may offer me a great job in a town I’d rather not live in. Or a great job where I’d probably get eaten for breakfast (as a friend pointed out would happen had I taken a job at Microsoft, or the various consulting companies I got called back for second interviews at).
This is sort of the “ground truth” objection, but it points out that you really can’t know the ground truth. So the whole idea of “decision theory” strikes me as, well, problematic.
David:
It’s fine to not want to gamble. My problem is with naive views of “risk aversion,” as discussed here.
My take decision analysis is similar to my take on multiplication. You make assumptions and then you get conclusions. If the conclusions make no sense, you look back at your assumptions. 8*7 = 56, that’s fine. If you have 8 blocks, each of which weighs 7 kg, the total weight is 56 kg. If I can easily pick up this pile of blocks, then no way it weighs 56 kg, so at least one of my assumptions is wrong: maybe I don’t actually have 8 blocks, maybe some of them don’t actually weigh 7 kg, maybe something happened when the blocks were put in a pile. The multiplication is useful, even if it gives the wrong answer, if we are willing to interrogate the answer it gives.
> My objection to the whole decision theory schtick is that it ignores personal preferences.
It’s all about personal preferences (is there another kind?).
> For example, I’m _seriously_ risk averse.
Decision theory has many issues but the inability to model your preference for $100 today over #200 next week is not one of them.
Carlos:
Just to echo my comment above, I don’t think that risk aversion, as described by David in his comment, is well modeled by a curving utility function for money. I think a better model is to ascribe a negative utility for David to some combination of the act of gambling and the feeling of uncertainty.
I don’t see why risk aversion as described by David in his comment (“I’m _seriously_ risk averse. I don’t gamble, don’t buy stonks. No matter what the odds.”) would be a problem for decision theory. Extreme risk aversion can be modelled, the minimax decision rule is almost a century old. (I agree that there are other problems!)
The problem is it doesn’t know whether or not (or to what degree) I, or some other bloke it’s modelling, is risk averse. Or a plethora of other peculiarities. So it doesn’t have the inputs and their interactions. E.g. the inputs may change with the articles I read at 4:00 am while waiting for my antihistamines to kick in so I can go back to sleep. Stonkbrokers should be happy I don’t stonk: I’d micromanage them like crazy.
It’s like ChatGPT: I’m better off reading wiki myself than spending the time/effort to create a prompt that will persuade ChatGPT to randomly-generate a halucination-laced albeit short and definitive-sounding overview of the problem; i.e. by the time I’ve told the decision system my preferences, I’ve already figured out my decision better than a curving utility function for money or even disliking uncertainty, since it ain’t either of those.
> The problem is it doesn’t know whether or not (or to what degree) I, or some other bloke it’s modelling, is risk averse.
Decision theory is a mathematical description of preferences, it doesn’t know anything. Gravitation theory doesn’t know the mass of the planets either. (Of course risk aversion is a much more diffuse concept and people are not as consistent as planets. I don’t disagree with that!)
“Decision theory is a mathematical description of preferences, ”
Ah. That’s why I don’t like it: It’s a naive attempt at doing AI with an approach that can’t possibly work.
Carlos:
David’s risk aversion can definitely be modeled using decision theory, just not using a utility function for money. The decision has to depend not just on the probability distribution of the outcomes but also on the process of getting there. Which is fine, it’s just not the default model used in economics.
David:
It’s not “whether or not (or to what degree)” you are risk averse. The point of my above-linked posts is that “risk aversion” is not a single thing that’s defined on some univariate scale. The term is used to describe many different sorts of attitudes and behaviors, and it’s an error of economics textbooks to define it as if it is some sort of single mathematical quantity.
“The term is used to describe many different sorts of attitudes and behaviors, and it’s an error of economics textbooks to define it as if it is some sort of single mathematical quantity.”
Exactly. That was what I was trying to get at with the risk averse example: decisions are made based on a lot of information (including morals, ideals, personal goals, and there are various types of risk*), not just a couple of numbers with a simple relationship. It’s going to take _symbolic computation_ to model that stuff. But current AI has gotten distracted and (mostly) isn’t working on that.
*: Purchasing a musical instrument sight unseen from Amazon, reading schlock novels by popular authors who have occassionally been known to write something interesting are a couple of recent bets I’ve lost on, and will probably make again. So there’s a whole class of things where said risk aversion goes away.
Most issues with decision theory, including some well known paradoxes and problems, fall under the following pattern. Abstractly stated it’s this: the loss or utility function is modeled as depending on some space X of choices, but in reality should use a much bigger underlying space Y.
Using X is done for practicle reasons, and almost always needs to be done, but it easily causes problems. If you have two possibilities y1 and y2 that should lead to different decisions, but they get mapped to the same x in the framework (and hence same decision) it’s an instant defect.
I’m a philosopher of science who increasingly identifies as a social scientist. But as someone trained in humanities who’s now chair of a humanities department I’m familiar with a lot of these critiques of decision theory. As I was reading this I kept thinking of the Box aphorism, “all models are wrong, but some are useful.”
A lot of the critiques in this post are various ways that decision-theoretic models are wrong, i.e., involve false assumptions or make strictly inaccurate predictions. And often Jessica’s response is that they’re still useful. Despite the many humanistic critiques of traditional scientific realism and correspondence theories of truth, humanists often fall back on those traditional views when criticizing quantitative fields. (And that’s maybe one reason I identify less and less as a humanist.)
But the trick is that a lot of scientists still work with these same traditional views. Think of how often “truth” is invoked in arguments that we should “follow the science,” or when politicized efforts to defund certain areas of research are portrayed as an “attack on truth.” There’s even a moment in this very pluralist, pragmatist post where Jessica kind of reverts to a much more simplistic realism: “in reality, any way that you go about evaluating people’s decisions is asserting some decision problem.” Those traditional views run super deep in the way our culture understands “science.”
In conclusion, everyone needs to take more philosophy of science (kidding/not kidding).
Heh, you’re right, that sentence exposes how hard it is for me not to see everything as a decision problem. Similarly, I tend to see just about every choice or judgment as a negotiation of uncertainty. But I do my best not to force my preferred representations onto others, hence my annoyance when critics are so quick to do it.
«The best known theory that’s been proposed for studying decision-making is statistical decision theory. […] In the Bayesian variant of decision theory, we assume a decision-maker who knows the prior probability of the state, encounters some information signal, updates to posterior beliefs (ideally like a Bayesian), and selects an action with the goal of maximizing their expected utility.»
It is good here that this implies a clear difference between stochastic statistics and decision theory because a lot of people confuse them which is one of my pet peeves. As soon as one talks about “maximizing their expected utility” (or “which way to bet”) that is no longer stochastic statistics but decision/operations theory.
The main example is the single event (assuming an ergodic source):
* Given a perfectly mixed urn of otherwise identical black and white tokens, what is the probability of picking a black token?
* From a statistical point of view the answer “50%” is useless because the variance is effectively unlimited. Statements about probability are only useful with “not too large” variance.
* If one adds “if you predict right and pick black you win $10, if you predict right and pick white you win $5” then obviously the decision is to predict black, given that the variance is effectively unlimited, as in such a situation we know that for a larger sample size we would pick black, and there is no downside to extrapolate to a sample of size 1.
The other difference between statistics and decision theory is that there is also non-stochastic statistics and non-statistical (or more precisely non-stochastic) decision theory.
Much of the diffidence about “decision theory” and in general “algorithms” or even more generally “experts” and “managerialism” is that as a rule they are introduced to obfuscate with a layer of cleverly constructed technicalities decisions that have already been made for the advantage of some vested interests lobby.
As someone already said all models are wrong but some are useful, and the important “detail” is that some models are more useful to some people and other models are more useful to other people, and it takes money and power to pay someone to create model that is more useful to the buyer.
Consider the controversial role of “experts” and “the science says” in court proceedings: usually both sides produce experts with authoritative looking backgrounds who both use “the science says” to assert quite opposite things.
The same happens often when people seek a second medical opinion, or even simply when a plumber or electricians looks at the previous work of another plumber of electrician…
BTW Leibniz in the 18th century was so optimistic about rationality that he posited it would be possible to create a universal language and decision theory “characteristica universalis”, “calculus ratiocinator” such that any dispute could be settled by “calculemus” (we can compute that).
https://en.wikipedia.org/wiki/Characteristica_universalis
https://en.wikipedia.org/wiki/Calculus_ratiocinator