Co-blogger Jessica Hullman writes:
I tried something new today for an invited talk. I’ve been getting so tired of giving talks about research that bores me, which is the case with most of my prior work on visualization at this point (my interests have been shifting a lot in the last year, becoming more theoretical in a lot of ways). So today instead I presented one of my papers, which won a best paper award, but then used it to talk about why I question the value of empirical research like it. It was refreshing to be so honest, but not sure how it went over with some of the more senior economists/psych people in the audience. I’m curious when you give a talk, how much do you try to make it honest/interesting to yourself vs. give people what they want? I’ve started really dreading giving talks because I hate feeling like I’m pretending to care about the things that I know people like to hear about.
I do have thoughts on this! Here’s a quick outline:
1. I’ve generally preferred speaking about work in progress rather than completed projects. But I’ve found that audiences almost always prefer a polished talk about work that’s already been done. Even audiences of my friends and collaborators prefer talks about work that’s already done. That’s kinda too bad, but it’s the way it goes. Work in progress is often vague ideas, and it can be diffcult for an audience, even an audience of friends and collabs, to know how to think about vague ideas. What works best is to talk about polished work that’s already been done, but then also use that as a springboard to discuss open questions.
2. I find it goes well when I question my own work. People seem to like the openness. But it’s necessary to have common ground. Once I gave a talk to a group of applied mathematicians, and I was using statistics terminology, and the problem was that I did not do a good job of conveying why my problems were nontrivial. As part of my talk, I did some humility, but they didn’t appreciate it, because they just thought I deserved to be humble because I was working on such stupid problems. A similar thing happened when I spoke at a visualization conference that was full of computer scientists. I think that more than half the audience hated my talk because I didn’t do a good job of conveying where I was coming from. In retrospect, I guess I was too relaxed and mellow before the talk: I think of viz people as my friends, and I didn’t put in enough preparation on how to bridge the gap. Other times, with an audience of statisticians or social scientists, I’m relaxed and mellow and things go really well.
3. Not every talk will satisfy everyone in the audience. I think it’s important to make clear what is your key message, so that even the people who didn’t enjoy the talk will at least have a sense of what you were trying to say. I don’t always do this so well myself.
4. Jessica’s statement about questioning the value of empirical research reminds me of that theorem that I extracted from her writings! I guess there’s more to be said on this topic. Vaguely related is this thing I wrote a few years ago about “ways of knowing” in statistics: see section 26.2 of this paper. Actually, the whole paper is relevant here.
5. My style of giving talks has changed. I used to put lots and lots of words on slides, then I put fewer and fewer words, then mostly clip art, then I found myself speaking extemporaneously before even getting to slide #1, and now I mostly give talks without any slides at all. My current recommendation to people is to write the slides for yourself but then use them as notes when giving the talk. Anyway, to continue: now that I give slide-free talks, I work off of notes. It’s helpful to have an example to get things started; once things are moving, it’s not hard to keep the talk going.
Let me again emphasize the importance of the interaction between the talk, the audience, and the audience’s expectations of the talk.
> Even audiences of my friends and collaborators prefer talks about work that’s already done. That’s kinda too bad, but it’s the way it goes.
Could this be partly due to mismatch between expectation and realization? They come in expecting insight and wisdom to be handed to them from on high, and instead receive an open problem with no clear solution. How are these talks marketed, and are they usually used to present tidy, completed work, all the ‘i’s dotted and ‘t’s crossed?
At my alma mater we used to have weekly ‘unseminars’ whose focus was explicitly to crowdsource tentative solutions to as yet unsolved research questions. There’d be the usual intro & methods slides, but the latter would be provisional, usually describing ways in which they failed to satisfy. Most of the ‘talk’ was focused on interactive discussion, trying to pin down promising avenues to explore. I really liked them, but I can sorta see how somebody expecting a more passive experience might dislike the lack of finality and polish?
I guess it also depends on the scale of the talk–it sounds like you and Jessica are mainly thinking about hour-long colloquium-type talks rather than, say, a 15 minute conference talk. The best a 15 minute talk can do is often be an effective advertisement for a paper. The audience gets an outline of what you did and why but probably not the full scope of the logic or implications (unless they have enough background knowledge to fill in the blanks). Of course, there is still a lot of room for variety in the 15 minute format! It’s just harder in that amount of time to give the audience enough to chew on to foster a meaningful discussion. So with a short time, I find that a “through-composed” talk about completed work is often more successful.
An hour-long talk is definitely enough time in principle to get some good discussion flowing, but this requires audience buy-in. I get the sense that, unfortunately, a lot of people view these longer talks as an opportunity to “zone out” rather than a chance to engage with the speaker. Obviously this is not equally true for all people or for all audiences! But consider the proportion of the audience at a conference or colloquium who are tapping away at their laptops the entire time.
I bring this up because Jessica used the phrase, “give people what they want.” I think these dis-engaged audience members “want” to answer some e-mails and maybe get some vague feeling of understanding the topic of the talk. I feel like much of the current infrastructure surrounding talks (slides stuffed with bullet points, spending 15 minutes on an anecdote that barely relates to the work, etc.) is really geared toward providing these dis-engaged audience members with what they think they want. The problem is that this leads to a feedback loop—audiences comes to expect that most of a talk will be content-free scaffolding so they learn to tune out, but then being tuned out entails giving them more scaffolding so they can get “something” from the talk, etc. etc.
Yes, colloquium/keynote style talks is what I was referring to. Looking back I’ve done quite a few over the last few years, and most were on zoom, which I thnk makes it much harder – in person, even if I’m not that excited about what I’m talking about, I get some energy from the audience. On zoom its very different, unless the chat is very active with people posing questions or interrupting. But I have found interruptions to be much rarer in virtual talks than in person talks.
Your comment on audience’s expecting to disengage is interesting. I guess from my perspective, a good talk is one where there’s enough common ground with the audience that I can get into some nuance, like walk them through my own discovery process for some line of work, and end with whatever nuances/questions are currently nagging at me. I probably assume more audience attention than I should, at least for virtual talks.
I agree with your perspective on a good talk–as an audience member, I want to feel like I’m “part of the action” rather than just a passive witness. So if I ever get a chance to be at one of your talks, you’ll have at least one appreciative audience member!
Your experiences with remote talks also mirror my own—it is hard as a speaker to get energy from the audience and people tend not to ask questions the way they would if they were in person (and engaged!). In the audience for a remote talk, I also find it is awkward to do those things. I can raise my hand, but unless there is someone moderating, the speaker probably can’t see that (unless I’m by chance one of the 4 people in the sidebar). I can’t really just clear my throat or say, “excuse me,” or something because either a) no one will hear if I’m too soft or if I’m loud enough b) everyone will hear me just as clearly as the speaker and that’s just weird and feels rude to the speaker since I’d be talking over them. Finally, although I can type in the chat, doing so takes time and diverts my attention from the talk; ironically, trying to engage means having to disengage to ask a question!
Also, I should clarify that my thoughts on audience expectations are *not* with respect to any particular speaker, but rather a part of what I think is an audience member’s general schema for “academic talks”. In part, I think Andrew’s success with no-slide talks comes from making that schema no longer apply. Because they can’t rely on their priors (expectations about how academic talks typically go), the audience has to actually engage with the data (what the speaker is saying).
Since asking Andrew my question, I have slightly more perspective… when I emailed I was going through a period where I felt like there was something I didn’t like about a lot of my most talk-worthy work, but I didn’t yet have any solutions to propose. It lasted maybe four months or so, and the talks that I gave during that time felt uncomfortable. I tried being honest, but as my email suggests, sometimes I think I was too honest and alienated the audience, and so ultimately I settled on just giving talks that I didn’t feel very excited about but at least were polished. But some of them I really dreaded having to give, because I hate feeling like I’m not being fully candid.
I think talks are hardest to give when your research program or style of work is in some kind of major transition – early on when you start to sense a need to change up what you do, but it’s still hard to express why you know it needs to be different or what exactly the new paradigm will be. Kind of like Kuhnian scientific revolutions, but at an individual level. You don’t want to be giving talks when the normal science is just starting to crumble.
Related to Nikolai’s question about mismatch betwen expectation and realization – I do think audiences expect some kind of wisdom, and receiving an open problem is not satisfying in many contexts, at least not if the open problem involves questioning the thing that you just taught them. I think being uncomfortable as a presenter can result from a similar mismatch – at least for me, getting invited to give a talk makes me feel some responsibility to teach the audience something, but if the things that are most appropriate to teach them are things that I’m doubting, then it’s uncomfortable for me too.
There are many dimensions to this, particularly the sensitivity to the type of audience and nature of the talk. But the dimension you raise about excitement is one I want to echo. For me, it is essential that I am excited about what I am going to present – this applies whether it is a professional talk, a class session, or if I am testifying as an expert witness. In all cases, my talks are much better when I am genuinely excited about what I am going to present. If I don’t think I have anything original to say – or don’t at least have a creative way to present unoriginal material – then the talks don’t go well. In fact, I’ve stopped giving talks where that would happen. It is even one reason I stopped teaching economics and moved to statistics and data science. It was difficult to get excited about presenting the economics in a classroom. For many years, I got my excitement the way many economists do – by showing people just how naive their views were. But that got old, and eventually I saw that I was only presenting the standard economics view that merely shows that everything has opportunity costs and that incentives matter. When that didn’t excite me any more, I changed fields.
At the risk of sounding snarky or rude, which I don’t intend: it’s not about you.
When you give a talk, it’s not about you. It’s about the audience. When you write a paper, it’s not about you. It’s about the readers. If you just want to luxuriate in the good feelings of thinking about things that excite you, do it by yourself, or perhaps in front of a mirror if you can’t do it without an “audience.” Don’t involve other people who don’t care.
I think the key here is choosing what talks to give in which venues. At the institutions I have worked in, some conference series are explicitly for discussion of work in progress and problem solving–and the audience at those will be engaged. Others are explicitly for those with expertise in an area to pass it on to others–those draw a different (well, somewhat overlapping) audience.
If you don’t enjoy talking about your past, polished work, then don’t. But that entails turning down invitations to present at conferences that are intended for, and whose audiences attend for, just that. Any act of communication should be tailored to the needs and wants of the people you are communicating with. There are plenty of audiences whose needs and wants will align with what you like to talk about. Seek them out, and bypass the others.
The three most important things about giving a talk.
The audience.
The audience.
The audience.
https://physicstoday.scitation.org/doi/10.1063/1.3047683
https://www.bnl.gov/lecture/docs/who-is-listening-what-do-they-hear.pdf
Robin :
Audience is the second most important thing. The most important thing is goals.
Or, as we put it in our communications class:
GOALS . . . audience
I think there’s a question of what you’re optimizing for. If you just want to think about giving a successful singular talk, sure, focusing on audience and goals is probably best. If you give talks on a pretty regular basis, like every two weeks you have some sort of long colloquium talk scheduled somewhere (which is where I was when I asked Andrew for advice), the idea that you should always cater to the audience only quickly gets old. Some part of me knows that the best thing to do is just keep improving whatever my Stairway to Heaven talk equivalent is. But I don’t gain anything from that, and I start to dread giving talks. So then the options are: turn down talk invites or find a way to make it useful to both you and the audience. I would prefer that there are ways to do the latter, since I can gain a lot from giving talks. I’m the kind of person who relies on things like blogging and presentations to help me figure out what I really think. So having the pressure of talks as a consistent thing can be good for speeding up my own process of determining what I really believe and what I’m not willing to present. So, for me there’s still a need to find some kind of balance.
It seems as though how one defines the relevant audience would be important. No audience will be homogeneous and that requires defining who is most relevant. Also, it would seem useful to consider the audience in situ as part of a larger audience network, and so it would be important to craft a message that could be carried beyond the immediate audience to the larger audience.
Also, one point implied in your comments is that the speaker could also be considered an important part of the audience. If an essential part of the thought process is to clarify ideas by articulating them in the most coherent fashion possible, then the speaker would be a necessary part of the audience.
“it’s not about you.”
Oh, I disagree most vehemently. It’s all about you: the preparation you put into the talk is what creates success or failure. That’s my universal experience: the degree of prep is equal to the degree of success, whether it’s a classroom lecture or a conference talk with hundreds of people
It doesn’t matter if the work you’re talking about is a completed bust or a lump of clay. If you’re going to give a talk you need a well-crafted framework with headings and subheadings that will allow people who can’t download your brain to follow what you’re doing. People are overplaying the audience question. People are blaming the audience for their lack of prep. When speaking about science there are three audience levels: collaborators; scientists in other fields; and non-scientists.
Even if you’re giving a weekly / monthly report or discussion, you should have a framework in which people can relate to what’s going on from one session to the next “last week/month we were here, this week/month we’re here”.
Wow – a rare opportunity to agree with jim! I also react negatively to the idea that it is all about the audience and not the speaker. If I am the speaker it is about me – of course, I care about the audience, but in the end it is my giving the talk. And what I’ve found is that if I am not excited about what I am going to say, then I shouldn’t be giving the talk at all. And this give me an opportunity to cite another of my favorite Tufte quotations:
“Surely there is something to be said for rejecting once and for all the doctrines that data graphics are for the unintelligent and that statistics are boring. These doctrines blame the victims (the audience and the data) rather than the perpetrators.”
(I actually got to use that quote in testimony once)
“a rare opportunity to agree with jim!”
Gulp! :)
I like that quote! It also hints at what I think is the most irritating thing people say about data: “the data say (this or that or whatever)” Data don’t talk!! It’s up to the (speaker, writer, scientist) elucidate what the data mean! :)
Jim:
I agree that structure is helpful. The audience doesn’t need to see the structure—it can be on a sheet of notes that I use when giving a slideless talk—but the existence of the structure seems to really matter.
Even when I give a talk with no structure at all, when I just show up with no preparation and wing it, it would be fair to say that there’s an implicit structure. Sometimes I have no idea ahead of time what that will be, and the structure makes itself clear during the talk.
“The audience doesn’t need to see the structure”
Well OK, but they don’t need pencils or chairs either – they can write with charcoal and sit on a rock. It just makes it much easier for them to follow. :)
I’m sure you give great talks Andrew! :)
I need a whole post just about how to give a talk without slides. Are you using a whiteboard? Are you just talking? I’m just not sure I understand the mechanics and how it remains interesting.
It’s a process. I think most people go through the evolution that Andrew describes. When I talk I have slides because I often have pictures to show, but mostly I don’t put words on them.
To a point made here by other commenters: being excited about your topic helps. Preparation is also important. I’m guessing the fact that Andrew’s perennially six months ahead on his blogging and tends to revisit the topics that excite him from dozens of angles is great prep for him being able to speak extemporaneously about anything you read here.
Rick:
I just talk. You can google to see what it looks like. Here’s an example.
That one worked really well – selection bias ;-)
Also, caught off guard by biases differing by hertz – really?
I dunno…! I feel like giving a talk without slides is really something only a great and esteemed professor like Andrew can get away with. If I did it, I think I’d just feel unprofessional and unserious. There’s something in terms of having some slides there at all that convinces the audience (or perhaps just myself) that I’ve actually done something worth sitting down and listening.
After reading Tufte’s “Cognitive Style of Powerpoint” I felt defensive since I think I give very good Powerpoint presentations. So, I sent one of my presentations to Tufte – his response was simple: “lose the Powerpoint.” Now, Tufte is the extreme when it comes to criticizing PP, and I only agree with him in part, but in the big picture I think he is correct. But I still use PP, and I still feel defensive about it.
(I suppose you could replace PP with Excel and say the same things)
My position is that PowerPoint presentations get worse the more extensively you use its feature set. Mostly, all its options serve to deepen the hierarchy of information and make things less clear.
Yes, I agree with that. My slide presentations work equivalently in pdf format.
I don’t see anything wrong with PP. When I give a talk I have a few headings and subheadings on power point (roughly three major headings and three subs for each) that both remind me of everything I need to cover and give the audience some guidance as to what’s coming and how it fits together, and of course any data and/or illustrations. If I don’t have access to PP I speak from an outline and use a whiteboard.
Most of the bells and whistles in PP aren’t useful most of the time, but they can occasionally be used effectively. The main purpose of it is to show short text prompts, images, charts and graphics. When photos are necessary, as they almost always are in field sciences, it’s essential.
Apart from the substance, a talk is also a performance, and I learned from an equipment failure that I give a better performance if I don’t use a microphone. Needing to project makes you talk a bit more slowly, but it also makes you stand up straight, speak in complete sentences and sound more authoritative. It also eliminates the ums and ahs, and generally gets people’s attention (especially if you are a little old man).
My PhD advisors’ opinion was that, with the exception of job talks, which are their own special beast, a talk should be about a project where some, but not all, of the research and analyses are done. If all the r&a are done, then you should write the manuscript and not talk about it. The talk should take the audience on an intellectual journey, give them the opportunity to contribute, and leave them looking forward to the publication. These talks are harder to put together and can’t be used as many times, but they’re fun and lead to better papers.
I used to do a lot of talks, and have heard thousands of them, and everything everyone says here is correct, so I have little to add. Enthusiasm, visuals that complement the talk and never substitute for it, a keen understanding of the expectations of the audience and watching the audience closely during delivery to make adjustments, voice projection to the back of the room (preferably unassisted) are all ingredients of the best talks.
But obviously, the most important part of any talk is to have TED superimposed in a corner. People are always talking about those, regardless of content. It’s all about the sample selection.
I suggest doing a dry run. A museum near me run by a major university (whose name begins with an “H”) does talks for the public. Some are excellent. Some are dreadful. Once, I was chatting with the person who organizes the talks. I asked whether the speakers do a dry run. (I asked because it seemed clear to me that the speaker for a previous talk had not done one.) She said that they are very busy people, so she thinks they often don’t. Somehow she thought it was OK for them to have 50-100 members of the public (and maybe more watching online) come and sit through an hour long talk without the speaker having done a dry run. If I was organizing the talks, I’d insist the speakers do a dry run.
Words on the slides are very helpful for the people in the audience who weren’t listening too closely for a few minutes because they were thinking about something that was said. And those who find that being able to both read and hear something makes it easier for them to grasp.
David:
It’s fine to require that speakers give a practice talk first. It will just make it harder to find speakers, but I guess you can deal with that by just asking more people until eventually you find people who are willing to do this for you.
They can do the practice talk anywhere. They don’t have to do it for the organizers at the location of the real talk. They can do it for their spouse, if necessary.
David:
Some of us have busy spouses too! But, hey, if you can find speakers who will agree to give a practice talk first, go for it.
Andrew: Are you serious? It has to take much longer to prepare a talk than to give it. Plus, there is all the time to travel to the talk location and have dinner with the organizers. If you can’t find an additional hour to practice your talk before giving it to a hundred people live, maybe you are too busy to give the talk at all. It doesn’t show much consideration for your audience if you don’t properly prepare. Would you go to the theater if the actors said they hadn’t done any rehearsals? If the actors don’t prepare, that’s called “improv”, not “theater”. I suppose you could advertise that your talk will be improv, so the audience knows what to expect. Of course, if you’ve given essentially the same talk before, then those previous occasions count as practice. As Christian said, without an audience is fine, too.
David:
Yes, I’m serious. I’ve given lots of talks and almost never have I given a practice talk first. I’m not an actor, I’m a researcher and a teacher. When people hear me speak, they’re not expecting a theater performance. The time that I might spend giving a practice talk, I’ll instead be spending on other activities of teaching, research, and service. For example, responding to blog comments!
But, again, if you can find speakers for your seminar who will agree to give a practice talk first, that’s great. You’ll get what you want and everyone’s happy. Different seminar series have different styles and different expectations.
> If the actors don’t prepare, that’s called “improv”, not “theater”.
Good improvisers spend a great deal of time preparing! It’s just that their prep time is not spent rehearsing a set performance.
I don’t know anything about acting, but I have a lot of personal experience playing jazz badly. Better players prepare by practicing riffs (roughly, snippets that form a kind of combinatorial repertoire or, at least, something to fall back on when you don’t have any better ideas), technique (boring stuff like scales, chord progressions, etc.), and building up a web of knowledge and references they can rely on to construct a long-form performance and build on what others are doing.
Gec:
That jibes with my speaking style. I have lots of riffs (examples and ideas that I’m familiar with), technique (statistical methods), and a web of knowledge (decades of experience) that allow me to improvise a talk.
If you practice your “improv talk”, then that counts as practice. But, if you wrote the slides the week before and have never tried presenting them to see whether the pacing is right or people can follow what you are saying in the time allowed, then those are things you would have learned by practicing.
David:
No, I don’t practice ahead of time! But, not to worry, I’m sure you’ll be able to fill up your seminar series with speakers who will agree to practice their talks, and meanwhile I can give talks in seminars that don’t require that their speakers practice ahead of time. Fortunately there’s room for all sorts of talks in the world of academia/business/government.
…or even without an audience.
An advantage of slides with words is that you can post them on your website after the talk so that people who couldn’t attend can read them and learn something.
All the words go in the speaker notes that the live audience doesn’t see.
If you went to the trouble to write them all out, why not show some of them to the audience? It is OK to say additional things that aren’t written on the slides the audience sees.
Because then the audience is focused on reading the words on the slides, rather than listening to what you’re saying.
But, if the words on the slides communicate the same thing as what you are saying, then that’s a good thing. The point is to communicate ideas. This isn’t an opera where they came to hear the sound of your voice (although operas now have subtitles).
Andrew: I’m curious about why you don’t use slides given your own emphasis on data visualisation and also given that some ideas can be made much clearer using appropriate visualisation. I do get how standard slide talks with pretty much all that is said on slides can be boring, can work against interaction, and also may encourage some to not paying attention, but I’d think that for almost every talk 2-5 visualisations can be found that do something good for the audience?
Christian:
I used to give talks with slides that had lots of words—for decades, that’s how I did it, since before powerpoint when I’d write the slides using felt-tip pens on transparency paper—then at some point someone gave me the advice to use fewer words, and I started gradually reducing the words until eventually the slides were just clip art. Then I started getting into the habit of warming up the audience by talking things over before the first slide—I guess this worked well in part because not having it tied to slides gave me flexibility in what to say, and partly because the audience then had the (correct) impression that anything could happen. Then one day I gave a presentation where I spoke for 25 minutes before the first slide came up—this wasn’t really my plan, I was feeling loose and relaxed and it just came out that way—but then it came to me that I could give entire talks in this manner. So I tried it out and it worked so I kept doing it.
That said, yes, I do sometimes use slides for my talks. Not always, but sometimes as you say it can work to share a few graphs. My current baseline is to use 0 slides, and for awhile I was a purist about that, but nowadays I’ll throw in some slides if it seems that’s what people will want.
I’ve got to say that in this, the age of zoom, I find that slides actually have an important role – by taking a screenshot of a slide presentation, it gives me something to refer back to when the speaker is at a different part of the talk. For example it helps me remember if I had a question about something, or if there’s something I didn’t understand, or thought I understood but a later part of the talk appears to contradict. Prior to zoom I’d have to take photos with a smartphone and that’s possibly a bit rude.
When PowerPoint was new, many speakers thought it clever that instead of people seeing the whole slide, you could expose it line by line. Of course, this just made it harder for the audience to follow since they couldn’t see where the speaker was going.
Actually I still like that – if you show more than you say, it can distract the audience.
You should say what is on the slides, but people’s brains may not be going at the same speed that you are talking. They may fall behind or they may be a bit ahead.
I’m with David here, I really hate that. I think you can make it less of a problem if you grey out unrevealed text and ungrey it as you go along, but that’s still fixing a problem you are creating in the first place. In the end different people consume the slides at a different pace anyway, and forcing them to follow the presenter’s pace just hurts more than it helps.
There’s many presentations ruined by having some complicated concepts on the final line that the presenter leaves only a fraction of a second to look at.
That’s a good point. Ideally, you would have two screens so you can put the previous slide on the second screen where people can still see it.