Coming to theaters near you, some summer, sometime, maybe soon

This is Jessica. I mostly try to ignore the more hype-y AI posts all over my social media. But when “a historic coalition of experts” comes together to back a statement implying the human race is doomed, it’s worth a comment. In case you missed it, a high profile group of signatories came together to back the statement: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

Intellectuals sometimes like to make statements, and some even try to monetize them, but how did this group arrive at this particularly dramatic sentence? There’s been plenty of speculation, including among experts, about scenarios where AIs like LLMs do things like trick people, or perpetuate stereotypes, or violate data privacy, and how these things could get worse when they are deployed with more agency in the world. But how did we get from here to extinction all of a sudden? Why couldn’t we just emphasize safety or something? We already have plenty of centers and initiatives for AI safety. Are we now going to see rebranding around extinction? 

It makes one wonder if this coalition is privy to some sort of special knowledge about how we go from ChatGPT not apologizing for its errors to populations dying off. Because if we can’t foresee how that’s going to happen, why make extinction the headline? What about just making the world less objectionable/unfair/deceptive/anything else we actually have evidence AI can contribute to? What exactly does upping the ante contribute to the already overhyped AI landscape? 

Maybe we need to put ourselves in the shoes of the AI researcher who has spent their career being celebrated for their advancements to the technology, but who hasn’t really engaged in much criticism involving AI’s negative potential. It seems reasonable to imagine that they might feel some pressure to sound the alarm in some way, given how much louder the critics are now that the hype machine is finally recognizing that it’s worth hearing about the darker side. I think many of us sense that it’s gotten harder be a techno-optimist without acknowledging other people’s worries, even if you haven’t really paid attention until now.

So maybe extinction appears as an easy way to signal being concerned about AI. It puts you in the headlines for your altruistic urges, but under a concern that’s indefinite enough that an equally vague gesture toward solutions, like “we need to focus more on responsible AI or AI safety” feels satisfactory. At least for me, extinction is not very emotionally evocative. It’s a nice clean way to signal terror without having to get too specific. We might as well be making vague references to something like “closing the loop.” 

But its confusing … when did it become cool again to be the dramatic AI guy? I can remember a faculty candidate once unironically mentioning the singularity in their job talk when I was a grad student, and suddenly everyone sat up a bit, like Did he actually just say that? It was sort of an unwritten rule that you couldn’t get all sci-fi extremist and expect to be taken seriously. But now existential risks seem to be experiencing a resurgence. This would not necessarily be a problem if there was some logical argument or model to warrant the extreme projection. But where’s the evidence that it’s extinction that we need to guard against, rather than the more mundane (and much more probable) human bias gets amplified, self-driving car causes wreck, family man gets put in jail due to automated face recognition etc. kind of world? 

This seemingly rapid leap–from how language models that predict the next word can generate human-seeming text to experts warning about the extinction of the human race–makes me think of Plato expressing his fear of the poet in The Republic, who he thought was so dangerous that he should be banished from the city. My understanding is that Plato’s fear didn’t seem so extreme at the time, because there was no sharp distinction perceived between what we would now call works of art versus other types of objects that seemed to possess their own creative principle, like nature. So the idea that a poem that could be in the world like a natural object carried weight with people. But even Plato wasn’t talking about human extinction from the words of the poet. His concerns with art were more with its potential for moral corruption. At any rate, Plato’s fear starts to seem pretty down to earth relative to the words of these AI experts. 

I can understand how AI researchers who have been raising concerns about AI safety for years would find this slightly annoying (many of whom are women – Margaret Mitchell, Abeba Birhane, and many others – whose work tends to go unmentioned when a Hinton or LeCun speaks up). Someone responsible for contributing to the underlying technology becomes concerned after being pretty quiet for years and its a massively newsworthy event that paints that person the new spokesperson for safe AI. I’m glad there’s some recognition when the people responsible for some of the key technical innovations say they’re not convinced it’s all good – being up front about the limitations of the methods you develop is healthy.  But when a bunch of people jump to align themselves with an extreme sentence that seems to come out of nowhere, it must be frustrating to those who’ve spent years trying to get the community to the point of recognizing any consequences at all. 

This is Jessica. I remember once hearing one of my colleagues who is also a professor talking about the express train that runs through much of Chicago up to Northwestern campus. He said, “The purple line is fantastic. I get on in the morning, always get a seat and I can get research done. Then I get to campus, and all research ceases for 8 hours. But I get back on the train and I’m right back to doing research!”

It is no joke that the more senior you get in academia, the less time you get to do the things that made you choose that career in the first place. But the topic of this post is a different sort of irony. Right how it’s deadline time for my lab, when many of the PhD students are preparing papers for the big conference in our field. It’s a very “researchy” time. What is surprising is how easy it is to be surrounded by people doing research and not feel like there is much actual new knowledge or understanding happening.

There is a David Blackwell quote that I have come to really like:

 I’m not interested in doing research and I never have been, I’m interested in understanding, which is quite a different thing.

Andrew has previously commented on this quote, implying that this may have been true at Blackwell’s time, but things have since shifted and understanding is now recognized as a valuable part of research. But I tend to think that Blackwell’s sentiment is still very much relevant. 

For example, when I think about what most people would call “my research,” I think of papers I’ve published that propose or evaluate visualization techniques or other interactive tools we create. But I don’t necessarily associate most of this work with “understanding.” On some level we find things out, but its very easy to present some stuff you learned in a paper without it ever actually challenging anything we already know. It’s framed as brand new information but usually it’s actually 99% old information in the form of premises and assumptions with a tiny new bit of something. It might not actually answer any of the questions that get you out of bed in the morning. I think most researchers would relate to feeling like this at least sometimes.

Pursuing understanding is why I like my job. I think of it as tied to the questions that I am chewing on but I can’t yet fully answer, because the answer is going to be complicated, connecting to many other things I’ve thought about in the past but without the derivation chain being totally clear. Maybe it even contradicts things I’ve thought or said in the past. On some level I think of understanding as dynamic, about a shift in perspective. This all makes hard to circumscribe linguistic boundaries around. I find it’s more natural to express understanding in questions versus answers. 

The problem is that questions don’t make for a good paper though unless they can be answered with some satisfaction. As soon as you plan the thing that will fit nicely into the 10-15 page article, with a concise introduction, related work section, and a description of the methods and results, you probably have left behind the  understanding. You are instead in the realm of “Making Statements Whose Assumptions and Implications More or Less Follow from One Another and are the Right Scope for a Research Article.” Your task becomes connecting the dots, e.g., making clear there’s motivating logic running from the data collection to the definitions or estimators to the inferences you draw in the end. This is of course usually already established by the time you write the paper, but it can still takes a long time to write it all out, and hopefully you don’t discover an error in your logic, because then its even harder to make the pieces fit and you have to figure out how to talk about that. 

But it’s the understanding that is source of actual new information, in contrast to the veneer of new knowledge we usually get with a paper. I used to think that even though it was hard to really explore a problem in a single paper, the real learning or understanding would manifest through bodies of work. Like if you look at my papers over the last ten years, you can see what I’ve come to understand. But I don’t think that’s quite accurate. Certainly there is some knowledge accrual and some influence of what I’ve said in past papers on how I see the world now. But I would say the knowledge I’m most interested in, or most proud of having gained, is not well represented in the papers. It’s more about what intuitions I’ve developed over time, about things like what’s hard about studying behavior under uncertainty, what’s actually an important problem or an unanswered question when it comes to learning from data in different scenarios, what’s misleading or wrong in the way things get portrayed in the literature in my field, etc.

The conflict arises because understanding doesn’t care about connecting the dots. It happens in a realm where it’s well understand that the dots have only a tenuous relationship to the truth status of whatever claims you want to make. But it’s hard to write papers in that world. Strong assertions seem out of place. 

Maybe this is why Blackwell’s papers tended to be short. 

It’s worth asking whether one can reach understanding without going through the motions of doing the research. I’m not sure. I think there’s value in attempting to take things seriously and make moderately simple statements about them of the type that can be put in a research paper. But then again something like blogging can have the same effect. 

On the bright side, if you can find a way to write a paper that you really believe in, then once you put the paper out there, you might get some critical feedback. And maybe then understanding enters the equation, because the critique jars your thinking enough to help you see beyond your old premises. But at least for me this is not the norm. I like getting critical feedback, but even when the paper is about something I’m still in the midst of trying to understand, often by the time things have been published and presented at some conference and the right people see it and weigh in, I’ve already reached some conclusions about the limitations of those ideas and moved on. For this reason it has always driven me crazy when people associate my current interests with things I’ve published a couple years ago. 

In terms of shifting the balance toward more understanding, being intentional about publishing less papers and being pickier about what problems you take on should help. And other possibilities I’ve posted about in the past like trying to normalize scientists admitting what they don’t know or when they have doubts about their own work in talks and the papers themselves. More pointing out of assertions and claims to generalization that aren’t warranted, even if the work is already published and it makes the authors uncomfortable, because it enforces the idea that we are doing research because we actually care about getting the understanding right, not just because we like clever ideas.

P.S. Probably the title should have been, Understanding is everywhere, even, on rare occasions, in boxes labeled research. But I like the recursion!

This is Jessica. I was talking to another professor in my field recently about a talk one of us was preparing. At one point, the idea of mentioning, in a critical light, some well known recent work in the field came up, since this work had omitted to consider an important aspect of evaluation which would help make one of the points in the talk. I thought it seemed reasonable to make the comment, but my friend (who is more senior than me), ‘We can’t do that anymore. We used to be able to do that’. I immediately knew what they meant: that you can’t publicly express criticism of work done by other people these days, at least not in HCI or visualization.

What I really mean by “you can’t publicly express criticism” is not that you physically can’t or even that some people won’t appreciate it. Instead it’s more that if you do express criticism or skepticism about a published piece in a public forum outside of certain established channels, you will be subject to scrutiny and moral judgment, for being non-inclusive or “gate-keeping” or demonstrating “survivor bias.” The overall sentiment being that expressing skepticism of the quality of some piece of research out of the “proper” channels of reviewing and post-conference-presentation QA makes you somehow threatening to the field. It’s like people assume that critique cannot be helpful unless its somehow balanced with positives or provided in the context of some anonymous format or at a time when authors have prepared themselves to hear comments and will therefore not be surprised if someone says something critical. Andrew has of course commented numerous times on similar things in prior posts.

I write these views as someone who dislikes conflict and publicly bringing up issues in other people’s work. If I’m critiquing something, my style tends to involve going into detail to make it seem more nuanced and shading the critique with acknowledgement of good things to make it seem less harsh. Or if there are common issues I might write a critical paper pointing to the problems in the context of making a bigger argument so that it feels less directed at any particular authors. But I don’t think all this hedging should be so necessary. Criticism in science should be acceptable regardless of how it comes up, and you can’t imply it should go away without seeming to contradict the whole point of doing research. This has always seemed like a matter of principle to me, even back when I was getting critiqued myself as a PhD student and not liking it. So I still get surprised sometime when I realize that my attitude is unusual, at least in the areas I work in. 

One thing I really dislike is the idea that its not possible to be both an inclusive field and a field that embraces criticism. Like the only way to have the former is to suppress the latter. It’s unfortunate I guess that some fields that embrace criticism are not very diverse (say, finance or parts of econ), and that other fields that prioritize novelty and diversity in methods over critiquing what exists tend to be better on diversity, like HCI or visualization which do pretty well in terms of attracting women and other groups. 

In a different conversation with the same friend above, they mentioned how once in giving an invited seminar talk at another university, another professor we know at that university made some critical comments and my friend got into a back and forth with them about the research. My friend didn’t think much of it, but as their visit went on, got the impression that some of the PhD students and other junior scholars who had attended saw the critique and exchange between my friend and the other faculty member as embarrassing (to my friend) and inappropriate. This was surprising to my friend, who felt it was totally normal and fine to that the audience member had given blunt remarks after the talk. I had a similar experience during an online workshop a few months back, where a senior well known faculty member in the audience had multiple critical comments and questions for the keynote speaker, which I thought was a great discussion. But others seemed to view as an extreme event that bordered on inappropriate.   

Related to all this, I sometimes get the sense that many people see it as predetermined that open criticism will have more negative consequences than positive, because it will a) undermine the apparent success of the field and/or b) discourage junior scholars, especially those that bring diversity.  On the latter, I’m not how much evidence people opposed to criticism have in mind versus they can simply imagine a situation where some junior person gets discouraged. But a different way to think about it could be, It’s the responsibility of the broader field, not just the critic, if we have junior researchers fleeing in light of harsh critique. I.e., where are the support structures if all it takes is one scathing blog post? There’s sort of an “every man for himself” attitude that overlooks how much power mentors can have in supporting students who get critiqued. Similarly there’s a tendency to downplay how one person’s research getting critiqued is often less about that particular person being incompetent than it is about various ways in which methods get used or claims are made in a field that are conventional but flawed. If we viewed critique more from the standpoint of ‘we’re all in this together’ maybe it would be less threatening.

A few months ago I wrote a post on my other blog that tries to imagine what it would look like to be more open-minded about critique, e.g., by taking for granted that we are all capable of making mistakes and updating our beliefs. I would like to think it is possible to have healthy open critique. But sometimes when I sense how uncomfortable people are with even talking about critique, I wonder if I’m being naive. For all the progress I’ve seen in my field in some respects (including more diversity in background/demographics, and better application of statistical methods) I haven’t really seen attitudes on critique budge.