Aleks Jakulin writes:
This is such a (didactically) beautiful piece of investigation.
Maybe hunting for imagined “bias” is a folly, and we should be maximizing the bias in favor of better outcomes.
I don’t get why Aleks refers to bias as being “imagined,” but I agree with his general point, which is that the focus should be on outcomes. Most simply, you’d want to assign a positive utility to each good outcome and a negative utility to each bad outcome. Given that this AI system is being implemented at all, the goal has got to be to do better than whatever was the existing procedure, so the net outcome will be positive. I’d think the best approach would be to maximize utility, as defined based on individual and aggregate outcomes, and then use some sort of side payments to compensate people who have been inappropriately classified.
That said, there’s nothing wrong with estimating various aggregate measures of disparity as well, although I’d recommend against using evocative terms such “fairness” which then get associated with various mathematical measures of asymmetry.
To put it another way, “ethical AI” has two limitations here:
1. According to the linked report, it doesn’t work so well at its stated goals.
2. Various definitions of algorithmic ethics, fairness, and bias contradict each other, and they seem to be based on a false intuition that it should be possible for all measures of disparity to be zero.
Andrew, you raise an interesting point about bias potentially being good. Maybe part of the challenge is that we don’t expect data scientists to have a good grasp of ethics, and increasingly specialise education towards task-oriented learning rather than becoming a person capable of thinking deeply about analysis and its implications. You can address it superficially, e.g. by defining utility as the loss function, but that doesn’t address the root cause which is that in many cases somebody likely failed to think through the implications of applying a system at scale.
Robin:
Stepping back, I think there are two things going on. First, concepts such as social or algorithmic bias are not well defined, and there are statistical artifacts of the regression-to-the-mean variety that yield counterintuitive results. Second, computer scientists of varying political orientations want to be useful, and this can lead to them making strong claims without having really thought things through.
Computer scientists currently have a lot of prestige. Fair enough: they’ve earned it through all the amazing things they’ve built. A few years ago I wrote that economics today is like Freudian psychiatry in the 1950s in being at apex academic and social prestige and influence. That’s where computer scientists are today. But then you get the same problem with computer scientists now as we had with economists twenty years ago, or psychiatrists seventy years ago, that they get treated as all-purpose gurus and they start to believe their own hype, in this case thinking that fairness is some simple thing that can be engineered.
That seems to me to be an astute summary. It’s currently very convenient for tech companies for reality to be compressed into a small group of metrics that can be optimised.
I take you point to be that using the utility function approach is a superficial treatment of the ethical issues involved with model performance and bias. In any case, that is my reaction: maximizing utility requires weighting the various objectives for the use of an algorithm and I’d rather see the tradeoffs made explicit rather than ‘hidden’ within the utility function. I also think the idea of compensating victims for misclassification is an appealing, but highly unrealistic policy. I do believe it is generally impossible to satisfy eliminating all sources of inequity – but that is all the more reason to make these explicit. At some point judgements need to be made about which inequities are most important, and possibly how to redress or compensate victims, but I’d like these judgements to be made as clearly as possible. I don’t think the utility function approach accomplishes that, if it is meant to be a mathematical function. If it is meant as a heuristic for the various effects of an algorithm, then I’m fine with it.
I have done some work previously where my colleagues and I optimised threshold selection for model-informed decision-making using explicit cost/QALY trade-offs. We know it’s reductive but the point was to make people think clearly about the cost of misclassification and prospectively be able to anticipate those costs at an aggregate level. I agree that generally speaking there is a temptation to distill complex ethical issues into a single metric and call it a day.
How about we design welfare systems such that fraud is mostly impossible. For example, universal basic income. If everyone is supposed to get paid, then really the only fraud is if youre cashing someone else’s payments… Pretty easy to cut down on. Another option would be publicly provided housing. A coop company builds it, the public buys shares in the coop to live in it, and the government manufactures the money to finance its initial construction (zero interest loans given to the coop).
The problem with welfare is its means tested, government controlled, and monitored for fraud in the first place instead of being a coordination mechanism to ensure everyone has at least a decent minimal standard of living and builds shared ownership in societies capital stock.
I think we must distinguish between policies that alleviate poverty and policies that incentivise desirable behaviour. We want to incentivise or reward certain behaviours, such as childbearing, high academic achievement, ecologically responsible behaviour, saving for retirement and charitable behaviour. Another goal is to avoid extreme poverty in our midst (well, some people have it at least).
A blanket policy like UBI could alleviate poverty with less fraud. However, such policies would not be able to incentivise other positive behaviours.
This depends on what you mean by “positive behaviors.” Suppose we define it to include care work of various kinds (caring for kids, spouses, friends, parents, etc. who are unable to completely care for themselves). It’s true UBI doesn’t directly incentivize care work, but by allowing people to spend less time selling labor, it might indirectly do so.
Next up:
Fraudulent welfare recipient sues city for being flagged by a highly biased, and possibly even racist, detection model.
Notable part of the linked to blog post: “However, investigators are empowered to request a beneficiary’s bank statements, summon them for meetings, and even visit their homes. Past reporting, including our own, has shown how these investigations can be a stressful or even traumatising experience.”
Shakes head….
Update from our correspondent
Washington, November 22, 2025
Thousands of, mostly 18-25 year old, people have shown support for the fraudulent welfare recipient who is now suing the city concerning their detection model. Many of these young people are from so-called “Parrot Places”, where they spend many hours of their days reading and talking about often vague and obscure and ill-defined things which are mostly only viewed from a certain very limited perspective.
The “Parrot Places” are a relatively new phenomenon where young people pay, or most often its really their parents that pay, lots of money to listen to a select group of other, often significantly older, people talk about all these things. The young people who spend hours a day at the “Parrot Places” benefit from this all by having a very clear distraction, and having a very specific focus this way. This in turn severely limits their own, possible original and individual, thinking. This way, the young people won’t be thinking about other things, like what they truly want in life, or what their dreams are.
This process at “Parrot Places” is often further enhanced by hearing certain buzzwords like “bias” from the select group of older people at “Parrot Places”, which often trigger something in these young people. In fact, it may be a distinct feature of these “Parrot Places” to invent and use and talk about such trigger words in a certain way. This then results in a significant distraction for the young people there concerning their own (often way more concrete) individual problems, insecurities, or doubts, etc. This is what the young people, or their parents really, pay all this money for.
This process at these places result in a further building of a fantasy and wishful-thinking world-view in these young people. A world-view which is ever more removed from reality and common sense. In fact, it has been argued by critics that the vaguer and more obscure the topic, or goal, or concept, etc. and the associated discussions are, the more the young people at these “Parrot Places” are able to be distracted, influenced, and not deal with reality. In fact, they don’t even learn or know what reality is this way, at least in the short term, for as long as they are going to these “Parrot Places”.
However, some critics have recently began to wonder more and more whether this phenomenon involving these young people at “Parrot Places” may all lead to more and more focusing on more and more vague, obscure, and irrelevant issues and associated discussions. Some critics have warned that if these young people are not careful, they could end up thinking and truly believing that 2+2=5, or that left is right, or that up is down, etc. Some critics we have contacted in light of the present article point to possible recent evidence for exactly these kinds of consequences. Several critics mentioned the recent actions of a group of “Parrots”, as these young people who go to a “Parrot Place” are sometimes called, in Washington that chained themselves to a fence at the white house.
As we have discussed earlier this week, the “Parrots” did this to demand more attention and especially more resources to help the endangered Yangtze finless porpoise. The critics mentioned they were actually present at the time to observe matters, and noted that they witnessed the following events. A young person stated that “helping the finless porpoise gives me purpose” when asked about why exactly they chained themselves to a fence by a passerby. The passerby subsequently also noticed the T-shirt the young person chained to a fence was wearing, which read “If you don’t help the finless, you’re spineless” and asked a further question. This was however greeted with snark from several other “Parrots” who collaborated together to “protect” the chained “Parrot” as they shushed the individual person and collectively drove him off. The passerby walked away with a coffee in one hand and a newspaper under his arm, shaking his head, stating that he had to go to work now.
Its interesting that there are microorganisms evolving to depend on mass supply of plastic trash getting dumped into the ocean. Suddenly reducing that supply could disrupt foodchains:
https://www.animalsaroundtheglobe.com/scientists-found-bacteria-that-eats-plastic-1-317502/
In our 1980s scifi future, there could protestors dumping plastic cups in the ocean to protect some endangered species.
Sir, this is a Wendy’s.
Why do you shake your head?
“Why do you shake your head?”
It seems to me that you might feel “empowered” by something or someone that you are asking me for more information and clarity via a question. Or at least, I have recently learned that some people might think like that, or use such words to describe things. I also learned how this can have consequences, real or imagined I would imagine. I mean, don’t you know that that asking for clarity or more information might be very “stressful” or even “traumatizing” to some.
In light of your comment, I am currently also pondering whether to contact the research or social science or journalism or data-analysis (or whatever term is most appropriate) -institute referred to in the blog post here. Your question raises a point which might deserve some more attention, and maybe they can help out with what I want to make clear or communicate to the world (or whoever is willing to pay attention).
If I can get my hands on a tens or even hundreds of thousand of dollars (or whatever), I am sure that institute is willing to try and see which measures they can find and use to try and make certain things clear. Or maybe they know of some technical terms or fancy data-analyses or certain buzzwords that all really get the point across (at least with some people).
And before you tell me that you are “just asking a question and if you don’t like that you shouldn’t post a comment on a blog post” you should maybe re-read my earlier comment that you replied to with your question. And perhaps you can try and think about how that might relate to my reply here now. Also, you can take a step back and read the blog post and look at my quote in my original comment and possibly think about what might be proportionate here, and what is appropriate in general, and those kinds of things. And if that even doesn’t make things clear, perhaps you can answer 1) what age you are (are you, for example, 18-25 years old), and 2) if you currently are, or have been, a social science student of some sort.
And if things aren’t clear for you by now, perhaps it might be useful to know that sometimes some people just don’t want to deal with something because it’s all just too sad (or whatever). If they still want to communicate something, they can shake their head. Or maybe it’s just something that happens automatically in certain situations with certain people. Perhaps it’s not needed, or even desirable, that everyone understands this behavior. If some do, perhaps that’s enough.
Shakes head….
Walks away.
You’re are responding, as a presumably American, to a Dutch study and I, being neither European nor American, are trying to understand what your comment is trying to imply because it appears to have meaning from within your culture that is not apparent to me, someone from outside that culture.
From your response, I am guessing it is something negative which you are not prepared to say explicitly because then you’d have to defend it.
The problem of incommensurability is sort of self-affirming, in that people committed to applying a commensurable framework on everything generally fail to understand it, or if they do they dismiss it as second-order. But once you allow yourself to think in these terms, it becomes pretty obvious. Different values trade off against each other, but not always in a way that allows quantification. Fairness (which of course can be quantified according to some procedure) trades off against efficiency (value of output over value of input), but they don’t cancel out. There’s a literature in philosophy on this; see i.a. Martha Nussbaum. (Or Dewey for that matter.)
This is an argument for the irreducibility of politics, along with Dewey’s inductive approach to ethics. Quant people can make a huge contribution by identifying, measuring and simplifying the tradeoffs that need to be considered.
I don’t understand why this particular issue is about “AI.” There are tons of detection or prediction models based on good old GLM. Long, long ago I helped with a model for a state prison system that basically used standard variables to predict likelihood of reoffending during something like parole (overslimplifying). Yes, we tried to eliminate race and sex bias, but for the agency the overwhelming concern was not to let anyone out who would later end up committing a heinous crime so they were much more worried about wrong release than wrong incarceration. Of course they also wanted to save money by releasing people who were very low risk, but that was really secondary.
Keep up, glms are AI now :)
I do think the inability to eyeball the coefficients (which Greenland and others argue you shouldn’t do anyway for prediction) does give decision-makers the creeps.
Mainly because Biden admin defined AI as any computer system making a recommendation. And those recidivism algorithms also are exact replicas of the issues in this fraud algorithm. ProPublica did a story on it for the COMPAS tool in Broward County, FL.
This keeps coming back to “who gets to control what can be thought?” It just bounces back and forth between different ways of the progressives saying that they should be the ones in control.