This is Jessica. We often idealize science as proceeding primarily by the scientific method, where scientists approach the objects of their investigation with a healthy dose of detachment and neutrality, who become convinced only when the evidence is there, and remain open to changing their mind if new evidence becomes available. But in reality we see examples of authors becoming personally attached to their ideas despite the data, slipping into advocacy and becoming defensive or going into denial mode when presented with clear evidence they were wrong.
The seemingly irrational attachment to the ideas or findings can seem easy to dismiss as a bad thing. Yet there are also times when having some level of personal commitment makes one more effective at certain roles that scientists must play. For example, being too transparent about our own uncertainty is not always effective when presenting research to others, because the audience can become distracted and stop listening entirely, even if you have some useful insight to convey. My question is, What does our ability to now use AI to generate implementations, presentations, and even the ideas we work on themselves add to the mix?
I got a glimpse of this recently. May ended up being workshop month for me, with at least one each week. I saw a lot of presentations. A couple of these showed me something I hadn’t yet seen, at least outside of student presentations: talks comprised of obviously AI-generated slides. If you’ve tried to use the non-design optimized versions of models like GPT or Claude to create slides yourself, you will know what I mean. Almost every slide has content organized in a grid. There’s too much text—full sentences or nearly so in multiple places, headers and footers, and stylized phrasing everywhere, like “principal design levers” and “load-bearing assumptions” and “actionable pathways”.
These were not presentations by overwhelmed junior faculty or researchers I’d never heard of. They were by prominent researchers who are respected in their fields.
Needless to say, they were not very effective talks. The slides tended to have too much going on to parse in time, with way too much text. The vague phrasing was distracting, making me wonder what exactly the presenter meant by terms like “governing frictions” or “strategic bottlenecks” and whether they write like that in their papers too. Part of the problem is that the presenter tends to use their own language as they present, rather than reinforcing what’s on the slide, so you have two competing streams of information that feel like they’re from two distinct viewpoints, one which is quite confident and willing to summarize and even exaggerate, the other more reserved.
It makes sense that you’re more likely to hold at a distance what you didn’t come up with yourself, subconsciously at least, even if you think you’re selling it. In one case, the speaker also described how some of the results themselves were discovered by AI, which probably further contributes to the impression that they hadn’t fully committed to what they are presenting.
This has me wondering what the impact on diffusion of ideas will be as it becomes more standard practice to rely on AI for implementation in scientific production and communication. It’s funny how reserving skepticism for your own results often comes up in discussing epistemic virtues, but when speakers present as if holding their work at arm’s length, the result is not so informative. As we rely more heavily on AI in all stages of research, will we face more challenges in getting others to adopt our ideas?
It’s also another reminder of how few people thinking about AI for science seem to have considered all the personal stuff that goes into the practice of science, with lots of irrational investment and fixation and stubbornness and pride to drive the loop of discovery and validation and communication. Scientific discovery may be an “ocean,” to borrow an analogy associated with Leibniz, but surfing it requires strapping oneself to a board and committing to seeing where it gets you, not just keeping it in sight while you splash around somewhere else.
This also leads to a practical question of how you instill a sense of ownership, or at least commitment, to ideas that were partly produced by AI. My own experience is that it takes a lot of time to verify AI produced results before I get to the level of confidence I’d have if I’d done it myself. For complex tasks there will inevitably be decisions made along the way, e.g., about how to parameterize certain things in implementation or to deal with edge cases or other exceptions. Each of these has to be reconstructed before I can really feel that I stand behind the output.
Is there an alternative? It makes me think of the “baking guilt” that housewives supposedly felt after cake mixes came on the market, because they only required adding water. There was a loss of a sense of personal contribution and emotional ownership. The solution, which persists today, was to have them add an egg. Some psychoanalysts went so far as to interpret this as symbolic of their fertility. For AI-aided science, the closest thing to adding an egg seems to be having agents explain at length to you what was done, which can still mean a big improvement over implementing everything yourself, but not as much of a boost as it first seems.
At any rate, interpreting the new challenges of AI-generated presentations of potentially AI-generated ideas as an aesthetic problem, or of “putting style before substance,” does not seem right. Scientific ideas don’t diffuse as bare propositions. They diffuse through people who have developed some passion for them. If we’re talking about AI for science, we shouldn’t be ignoring scientists and their relationships with what they do.
From my experience, LLMs are more confident in their claims than most humans are. And they tend to hold on to those claims a bit – but when confronted with their errors usually let go faster than most humans. Your examples of AI constructed presentations seem unrelated to me – and, in any case, can be avoided by training the AI to express things differently (i.e. use a specified style).
Humans do not develop their ideas in an objective detached manner – I’m happy to claim. Of course, the AI doesn’t either, since it was trained on materials mostly produced by humans. But I don’t think the AI is attached to its claims in the ways that humans are – at least, it seems to me that this is one difference between AIs and humans (and one that I cherish).
My experience differs from yours – you say it takes a long time for you to trust the output of the AI. I find the opposite – I dangerously trust it too quickly. This partially results from how confident the AI seems. Once it goes beyond what I’ve done, I am too quick to accept its output. Usually it has missed something important – which I’m glad to say makes my insights still relevant.
Sorry about the disjointed ideas here – but I found your post somewhat without a clear theme. My last thought is one that has been bothering me lately. I’ve been working hard to use AI to present choices, alternatives, and questions rather than answers. It is a constant battle since the AIs seem designed to produce answers. I’ve been wondering why this is the case. Who is the market that is suggesting that these AI providers should produce answers – which generally means the human becomes irrelevant. It isn’t clear to me who is really looking for AI to play that role. Having AI focus on presenting alternatives keeps the human in the loop and keeps the decision-making in human hands. But I am constantly being steered towards letting the AI produce answers (and all those provided forms – slide packs, summaries, reports, etc.). Clearly the providers see this as what the market is demanding, but I keep wondering what market really is demanding that.
Perhaps you have to see some of these AI-generated talks (in some cases of AI-generated ideas) to understand what the concern is. Watching them, it’s quite obvious that if this becomes the norm, we will lose something that currently makes scientific communication effective.
Of course one could tweak the AI-generated slides to make them feel like one’s own, but it seems that not everyone sees the vaule of doing that. So I suspect we’ll see more tolerance for presentations or ideas that don’t quite feel like one’s own in the name of producing more research more quickly.
On getting AI to refrain from trying to neatly wrap things up even when it isn’t warranted … this is definitely a problem, and probably stems both from the RLHF stage of their development, where people like things that feel definitive, but also from the way we target good benchmark performance, where guessing confidently is a better strategy than not guessing at all (similar to what people do on multiple choice tests).
On your first point: if academics are willing to lose their voice and use AI-generated vanilla slides, I see that as no different than students misusing AI to do their homework without having to think. People do stupid and dangerous things, and when technology is involved, there is a tendency to blame the technology (I can’t help but think of the “guns don’t kill people….” line). There may indeed be a use for regulating the use of the technology (although it isn’t clear to me in academic settings), but the underlying problem is something else. An academic that doesn’t care whether their voice is in their presentations has no business giving those presentations. If you are seeing this happen, then I understand that you are concerned, and so am I. But we have ample evidence that many academics abuse Powerpoint in both their classes and their professional talks (and most textbooks cater to that practice). The problem has been there for a while.
Two things: first, I think this is the same weird feeling I used to get when directors of big fancy places would give talks with images of work done by the faculty in their centers. They would clearly improvise based on what they thought the work was about (ask me how I know lolsob), and most people in the audience had no idea. And now I realize that some people might perform the same sort of fabulism when they’re coauthors of the paper. Yikes.
But more importantly, I agree with you that if AI enables this en masse, that’s a serious moral hazard!
Jessica:
You refer to the AI “trying to neatly wrap things up even when it isn’t warranted.” I’ve often seen this in (human) news reporting. This is especially clear in a radio report, where the announcer pauses at the end to give the summary, but it’s also in lots of newspaper and magazine reporting as well. I’ve often said that one advantage of blogging, as compared to scientific publication or journalism, is that in blogging you can state your uncertainty and you can just stop when you’ve run out of things to say. In contrast, in science publication and journalism, there’s an expectation that you’ll come to a stirring conclusion.
Yep. But if you ask AI to write a blog post, you’ll get one with a stirring conclusion.
Quote from above: “Scientific discovery may be an “ocean,” to borrow an analogy associated with Leibniz, but surfing it requires strapping oneself to a board and committing to seeing where it gets you, not just keeping it in sight while you splash around somewhere else.”
I have written some manuscripts where the final result surprised me, because it turned out different than I anticipated. Or the final result “took me somewhere” I did not even thought about, because I only had a vague idea when I started writing, or the structure I had planned beforehand changed when writing and coming across new papers and sources. Perhaps this is similar to surfing the ocean in that one might “go with the flow” and see where it leads to.
I think my two most recent manuscripts might be good examples of that, where in one of them I encountered an online version of a classic book about the topic, which in turn heavily influenced the structure of the manuscipt. In the other one I just started writing about making a comparison between things to see how far I could take that comparison, and in turn if that somehow could turn into a complete manuscript. In both cases, I feel like things sort of fell in place like a puzzle. I cost me lots of energy and much effort, but I had sort of a conscious and explicit thought that I would simply see where things would lead. A bit of a combination of some structure and some planning and some previous ideas and thoughts with a go with the flow and using new references and sources and ideas as a guide or beacon. Perhaps that approach is a bit like going with the flow, or surfing the waves.
Perhaps going with the flow, or surfing the waves, has some additional views attached to them. For instance, I wrote one manuscript a few months ago in about two months which were very intense and focused. I don’t know which words to use, but even though some thoughts and ideas and references used were known to me before writing, the “surfing” kind of happened in a short time period. I came across some random information about some band recording songs that didn’t make it as one single album, and that information likely influenced the thought I had that perhaps when something is produced in a relatively confined and intense time period, it may give it some sort or vibe or thing that might be lost when leaving it to rest and possibly re-write things months later. I checked and double-checked my work, and then decided that this is what came from that specific intense two month-writing period and perhaps it should be published the way it is now, including possibly sub-optimal sections or phrasing. I thought letting it rest, and possibly re-writing things weeks, months, or even years later might result in losing “something”.
An additional attached view from this writing experience and perspective is that I can look at it as sort of an enjoyable experience. I searched, and looked around, and found sources and guides and signs that took me somewhere I don’t think I could have ended up doing things in a different way. When you ride the wave, or go with the flow, you might end up in a place you haven’t thought of before, or it could lead to experiencing something that might be usefull or worthwhile. It doesn’t matter perhaps what others think of the manuscript: I tried to ride the waves I saw and thought I could handle and did it the best I could in the way that fits my style.
Anyway, all this talk about the cake and cake mixes made me think about Van Halen’s “Poundcake” which I am listening to now.
I too have now listened to Van Halen’s Poundcake. It is the right song for that paragraph.
Fun fact about “Poundcake” by Van Halen, at least according to the sources I came across when looking for more information. The drill sound that can be heard in the introduction and elsewhere in the song was incorporated after an engineer left a drill in the studio in front of Eddie Van Halen while he was going to grab something. Van Halen then asked Alex to start the song again from the beginning and Eddie used the drill to scrape it on the strings for the intro.
Perhaps that’s a nice example of “going with the flow” or “riding the waves” when “surfing in the ocean”. This is a quote from Eddie Van Halen about it:
“It’s just one of those funny unplanned things that happen every so often. Since it was on the record, I ended up using the same drill every time we played the song live.”
“It’s also another reminder of how few people thinking about AI for science seem to have considered all the personal stuff that goes into the practice of science, with lots of irrational investment and fixation and stubbornness and pride to drive the loop of discovery and validation and communication.”
That made me think of the following article I came across recently: https://cacm.acm.org/research/rolling-in-the-deep-of-cognitive-and-ai-biases/
“AI is never developed in isolation; it is deeply shaped by the ways humans think, decide, and interact with the world.”
Anyway, I just figured I’d mention it in case you hadn’t seen it.
Quote from the blog post: “For example, being too transparent about our own uncertainty is not effective when presenting research to others, because the audience can become distracted and stop listening entirely, even if you have some useful insight to convey.”
Maybe to truly hear something doesn’t require it to be loud
Maybe a whisper can sometimes echo longer than a shout
Regarding slides and talks, maybe things would help if people just gave talks without slides. That’s what I do; see here for lots of examples.
When students give presentations in my classes, I ask them to use no slides. But if they insist on using slides, I say they’re only allowed to use images. No text except for labels and titles on graphs. If they find that words are helpful for them to organize their thinking, that’s fine, they can use notes. But no slides.
People were ranking different slide options in response to what I posted on social media about the cringiness of watching someone present obviously AI-generated slides. Like is it better to have no slides, or bad slides from someone who tried but doesn’t know how to make good slides? I agree with you, nothing seems better. And then on the positive side, is a whiteboard talk better than great slides? And is no slides, and no writing on the board, but pretending to write or draw on the board better than that?
I have prided myself on producing good slides for presentations I give. I once sent an example to Edward Tufte (author of The Cognitive Style of Powerpoing, an extreme position if there ever was one) and asked him if the problem was Powerpoint or bad use of Powerpoint. His response to me: “lose the Powerpoint.”
I haven’t followed his advice, but I appreciate his consistency.
Andrew:
My PhD supervisor (the most intelligent man I’ve ever met) used to say the same thing to me, and I’ve always hated this advice. Lectures should be primarily driven by the presenter, and disinterested half-awake recital of lecture notes on a screen was not an uncommon issue I encountered as a student. But to say we shouldn’t use slides at all strikes me as similar advice to saying we shouldn’t use graphs in our manuscripts. They offer a method supplemental information to enhance the exposition and can help the organizer structure their thoughts, especially for those less experienced. I’m glad you at least allow your students to display graphs – I honestly don’t know how anyone can be expected to deliver an effective talk on applied statistics without those at least – but are a few bullet points here and there really detracting the quality of a talk in your eyes?
My rule-of-thumb is usually a new bullet point slide every ten minutes (never more than three bullets, and never more than one sentence per bullet) and liberal usage of graphs where relevant. I will sometimes throw in a “busy” slide with modelling specifications and derivations, always skipped over very quickly during the talk itself, but prepared in case someone asks me a technical question in the Q/A section so I have something relevant to point at. These are things *I* like seeing in presentations. I would honestly be a bit annoyed if someone tried to explain the results of an analysis during a talk/seminar without at least a graph.
9p:
I don’t know if this will make you feel better, but let me clarify that I’m not stopping students from using slides in other settings; I’m just restricting them for their presentations in my class. I think it’s helpful for them to have practice preparing and delivering talks without slides as a crutch, and this is especially true now that they might be using the chatbot to prepare the slides.
Regarding your statement that you “honestly don’t know how anyone can be expected to deliver an effective talk on applied statistics” without graphs: I recommend you follow the link and look at some of my slideless presentations. I do think that graphs can often be helpful, but in general I prefer the discipline of saying it all in speech. I’ll sometimes convey graphs in the air with my hands–you can see this in some of my online talks–and I think this can help the audience stay on focus.
All of this will depend on the presenter and on the audience–different people have different styles, and different groups will have different expectations.
“[Slides] offer a method of supplemental information to enhance the exposition and can help the organizer structure their thoughts, especially for those less experienced.”
That is how it worked for me. I did not give a lot of presentations, so I never got comfortable with them, but just having the slides behind me kept me focused and moving forward. I know from experience that (1) it is hard to stay focused on what I want to say when standing in front of a large group, and (2) glancing down at notes just doesn’t work.
And then there is the issue of Andrew apparently never having to give a presentation on torsional vibration fatigue crack growth in a hollow shaft. Try describing that by waving your arms!
In person presentations, and lectures in general, seem inefficient in the modern age. Its sequential access without even any speed control. Like reading a scroll instead of a book.
Obvious bot-generated content indicates the presenters also treat the presentation itself as a low priority. Maybe they are there for the networking.
Anon:
If I’ve already written the material, I agree that it should be easier for most people to see it in a paper or book than hear it spoken. At best, the talk would be a complement to the written material, explaining details that were not clear in the printed version.
But my talks are often full of material that I haven’t written down. Sometimes the talk is a good way for me to work through ideas, and this can be helpful to the audience too.
I think there is value there as well, eg, like what you mentioned it gives a level of “color” that would not be included in a written document. Still, I am thinking this will be increasingly be seen as a legacy communication method. Only people who already “bought in” will spend the time to pay careful attention to these in-person presentations.
A couple of points come to mind. IMO it’s not so difficult to be dispassionate about one’s research in much of basic (i.e. non-applied) experimental science – e.g. because the aim is to find stuff out and so there’s less likely to be personal investment in particular outcomes (although we’d like it if the results accord with hypotheses and ideas are fruitful). On the other hand I know I have irrational moments (pursuing an experiment long after it’s become obvious it’s not going to work; spending ridiculous amounts of time and effort on a minor issue that in hindsight was not worth it – I do the latter way too often) and of course one does tend to become passionate towards one’s research [possible since “passionate” an “dispassionate” are not direct opposites!].
I sometimes wonder how I would manage working in social science or economics because I imagine a difficulty in being able to be dispassionate. I expect each of us envisages an ideal way in which our societies should be structured and it can’t be straightforward to set that aside when doing research with societal or political implications. I’m curious if others feel that this is an issue – certainly much of “junk science” and fraud (in all science) arises from individuals (that presumably have passions that lie in some direction) being unable, or choosing not to be dispassionate about the object of their research.
I agree with the top article that science is a profoundly human pursuit that requires some level of personal passionate commitment and for which AI is a powerful supporting tool but probably no more than that.
“I’m curious if others feel that this is an issue – certainly much of “junk science” and fraud (in all science) arises from individuals (that presumably have passions that lie in some direction) being unable, or choosing not to be dispassionate about the object of their research.”
I have thought about this kind of stuff from time to time, but it depresses me too much to think about it more than I have. I also intend not to think about this anymore. I just want to note that even without possible individual passions, some issues like publication bias or sub-optimal theory development may skew things massively.
I don’t know if the following reasoning is correct, but it’s what I have thought about from time to time: if scientists in general largely only study certain topics, and not others, and largely publish only significant results, and not non-significant results, and largely don’t engage in finding ever more appropriate and better explanatory variables and theories to include in their studies things may get warped really badly…
Science is an evolutionary process and like evolution one can’t necessarily predict where it is going to develop nor what progress might have been made if we’d only studied different topics. Of course, all sort of arguments can be made about how things should have been done with the gift of hindsight!
But if people consider there are important understudied topics they can do something about it and often do. For example, there are lots of individual or advocacy group foundations set up to direct research effort into specific rare diseases. Much of the progress in cystic fibrosis therapies was instigated by the Cystic Fibrosis Foundation, Nancy Wexler devoted her career to organising and performing research into Huntington’s disease, the Spinal Muscular Atrophy (SMA) Foundation was started by a single couple with a child with SMA and in the last 20-odd years has raised $150 million for research. There are lots of these which is testament to the drive that direct personal involvement can bring to bear on defined scientific problems that might not be otherwise well-funded.
A fundamental element of science is to find “ever more appropriate and better explanatory variables and theories to include in (their) studies” and that seems to occur pretty well in my experience.
If some have tried to get attention and resources to study certain things, it could still be possible that only certain things get studied. It may even be the case that very vocal or enthusiastic people asking for attention and resources for certain topics may cause certain other people and topics to receive less attention and resources.
With respect to your final sentence, I sincerely wonder whether this in fact happens pretty well. I am in no way an expert on this stuff, but merely reason from my awareness of some papers that, although sometimes pretty complicated for me to fully grasp, seem to make clear that certain things in psychological science or social science may not work optimally when it comes to searching for, and finding, ever more appropriate and better explanatory variables and theories. It’s been a while since I’ve read about this all, I am just more generally replying to your comments here.
Here are three papers that might be interesting in this all, which I have come across over the years. There was another paper that someone on here mentioned in the comment section about research into memory (?) or something like that that showed, or argued, that research on this all during the last 50 or 100 years may have not truly resulted in better understanding (?) or something like that. That may also be relevant here, but I don’t know the title of that one. Here are the three papers I do know the titles of:
“So you confirmed, replicated and emptied your file-drawer… now what?” A Structural Realist’s Guide to Theory Evaluation in Psychological Science by Hasselman, Cox & Seevinck (2019)
The Theory Crisis in Psychology: How to Move Forward by Eronen & Bringmann (2021)
The nature and limits of psychological knowledge: Lessons of a century qua “science” by Koch (1985)
Anatomy of medical misinformation:
1) Blog/news/comment/etc makes an unsourced and vague medical claim:
https://en.wikipedia.org/wiki/Cystic_fibrosis
2) Wikipedia is checked for the possible basis of the claim:
3) The wikipedia sources are checked, but those omit key technical details/caveats:
https://www.cff.org/medical-professionals/patient-registry
https://www.cff.org/media/38406/download?inline
4) Final technical document essentially invalidates the original claim:
https://www.cff.org/medical-professionals/patient-registry
https://www.cff.org/media/24916/download
5) Next the “progress” will be defined as something less concrete than mortality/survival, so the definition of the primary outcome(s) will also change over time.
This thread is likely to descend quickly into garbage time so I’m only going to post once and then leave it.
I don’t know what point you’re trying to make Anoneuoid (and not terribly keen to know) but in my comment on CF I was addressing the fact that in a number of rare diseases where the small patient population has discouraged therapeutic development especially by drug industry, individuals and advocacy groups have stepped in to raise funds and to support research. In other words AAAnonymous’s point about certain topics not being studied is obviously true (and couldn’t not be true) but in many cases where there is a perceived absence of research effort, individuals setting up advocacy groups have been very successful in raising and directing funds. That’s really the point; people can recognize that a topic they consider is important isn’t being studied and can do something about it.
This is objectively true with the Cystic Fibrosis Foundation which was central in instigating (with industrial collaborations) research into developing drugs that would address a near-primary cause of CF (misfolding/misfunction of the CFTR protein; the primary cause is a CFTR gene mutation that causes misfunction). This has turned out to be successful with a number of effective modulators that usefully correct CFTR misfunction being approved in last several years. A huge advantage in these modulators is that they address the disease near its root cause. Previous therapeutic approaches treated the symptoms (severe epithelial infections, respiratory problems and organ damage) associated with CF but don’t stop progression of the disease. The modulators have a greater chance in doing so since in correcting CFTR protein function they not only reverse airway insufficiency but limit infections and organ damage. Since they only been in use for the last several years long term effects will need to be monitored, but the improvement in quality of life can be dramatic (as I know from personal experience).
Just a quick response to AAAnonymous. From the papers you posted the context of your points seems to be psychology research which is a quite specific subject. I had a quick read through the “The theory in Crisis” paper (thank you!). It seems (I Googled) that there are numerous advocacy groups that promote research into what the advocates consider are under-researched psychological/psychiatric disorders, so at that level the exploration of under-studied topics can be addressed in a similar manner as in my examples from medical research.
It’s also worth saying that this isn’t a zero-sum game – research funds for focusing on perceived research topic shortcomings are raised in a charitable context and are normally additions to government funds. Of course a researcher working on one topic may be attracted by the possibility of funding from one of these advocacy organizations and so might switch his/her research efforts. That’s not so uncommon. In response to the covid crisis many researchers that worked in appropriate fields temporarily refocussed their research. The bottom line is that everything can’t be studied and everyone sets or follows priorities.
@Chris
My goal is not to troll, its to alert you (and others) to the problems with research quality/communication in this field. And if it was only cystic fibrosis I wouldn’t care enough to bother. The standards are low in these exact same ways everywhere.
I don’t want to be misled when seeking medical help either.
“The vague phrasing was distracting, making me wonder what exactly the presenter meant by terms like “governing frictions” or “strategic bottlenecks” and whether they write like that in their papers too.” – my guess is that if they’re using AI to make slides, they are or soon will be using AI to write papers too.
As I was reading this, I thought about how reluctant I am to re-read things I’ve written with these LLMs. It bores me. This is something I don’t always experience with my own writing. Then I read about ownership here. This is it. There is no ownership or voice. Yet many plagiarise or lack originality, and I consider my AI-assisted papers to be original, as confirmed by Grok and ChatGPT.
OK, this sort of hijacks this thread – but I think it is important and can’t wait. The Office of Management and Budget has proposed to make all government research funding under the guidance and control of politicians – specifically the current Administration:
https://www.regulations.gov/document/OMB-2026-0034-0001
I guess (to tie it to this thread) this is having research based on the passion of Trump. The comment period closes on July 13. I don’t know what to say other than that China seems to be the model we are following. Fitting for the 250th birthday of the country to see it follow the lead of a single-party autocratic government. How foolish of me to hate our two party system. One is so much better.
Dale:
Yes, I discussed that proposal here. The post will appear in a few months on this blog but sometimes I post things earlier on the newsletter site. I also posted my comment on the official government comments site.