Stories are central to social science. It might be pleasant to consider stories as mere adornments and explications of theories that we develop and evaluate via formal data collection, but it seems that all of us—including statisticians!—rely on stories to develop our understanding of the social world. And therein lies a paradox: stories are valued for their specificity (“Show, don’t tell”) and their surprise value (“Man bites dog,” not “Dog bites man”), but these properties seem to be the opposite of the standardized data from representative samples that is valued in social science. We resolve the paradox by viewing stories as a source of counterexamples to existing social science theories: In statistical terms, storytelling is a qualitative form of model checking that is effective because of the richness (“thick description”) and unexpected (thus, model-violating) features of a story. Our theory of storytelling thus has a pleasant convergence of the descriptive and normative: the most entertaining and powerful true stories are also potentially the most effective for advancing our social science understanding.
We develop these ideas in the context of several stories.
The talk is in part based on these papers:
When do stories work? Evidence and illustration in the social sciences (with Thomas Basbøll):
http://www.stat.columbia.edu/~gelman/research/published/storytelling.pdf
To throw away data: Plagiarism as a statistical crime (with Thomas Basbøll):
http://www.stat.columbia.edu/~gelman/research/published/GelmanBasbollAmericanScientist.pdf
Convincing evidence (with Keith O’Rourke):
http://www.stat.columbia.edu/~gelman/research/published/authorship2.pdf
It’s noon Thurs 12 Feb in 601 William James Hall.
I think stories interfere with science (“God does not play with dice” being a classic case). If the social sciences want to be more like a science, they will tell fewer stories and test more hypothesis in a stringent manner.
+1
Numeric:
You know the saying, “It doesn’t matter if you believe in God. What matters is if God believes in you.”? Similarly, I think stories are so central to our social science understanding that they’re not going anywhere, so it’s best to understand how they work.
How does one validate a story? What about tempting, appealing stories that are not a good model of the underlying reality.
Rahul:
Yes, these are issues. Ultimately the issue of non-representativeness arises. The idea is that if the story has sufficient depth (what in anthropology is called “thick description”), it captures some nature of truth that needs to be part of our theories. Think, for example, of the work of Oliver Sacks. His case studies are unusual—that’s the point of them—but, by taking them seriously in their detail, we can learn things about human nature that would otherwise not be so apparent.
All of this, I hope, makes it clear why immutability is so important. Once a story can get its details changed around—once it becomes mutable—it loses its ability to contradict existing models of reality. A good story gets stuck in the craw of an existing but imperfect model. But it won’t get stuck in that craw if it’s been put in the blender first.
And this connects to the problems that Basbøll and I have with plagiarism. Once an author hides the source of a story and presents it as his own, it’s all too easy for him to change its details, destroying the immutability that is essential for the story to maintain its model-checking value.
Plagiarism is a step towards turning a story into a parable. Parables are fine and they have value of their own (consider Einstein’s parables or thought-experiments which he used to develop relativity theory) but they’re not the same as stories.
Boy, if you thing a story as presented by the “original” source is what actually happened (is immutable, in your words), I have a post-modern seminar I want to sell you. As far as your quote about God, what is the source (speaking of mutability!) The quote I’m familiar with is Trotsky “You may not be interested in the dialectic, but the dialectic is interested in you.” Was Trotsky paraphrasing the God quote or is it the other way around?
I’ve already been through with you the WWI “cooperation”, and have pointed out that a much better documented case of cooperation was in infantry assaults in the American Civil War, where, once the two lines were joined, soldiers typically clubbed each other with rifles rather than bayoneted each other (aside from the odd officer firing his pistol). This tremendously increased the odds of survival for both sides–much easier to survive being knocked unconscious than a deep blade wound. This didn’t require any repeated-game behavior, either.
As far as the comment in the paper “A stable equilibrium on both sides was to duck down in the trench, not stick your head out, and not shoot[,]” this actually was not stable–by this reasoning, the real stable equilibrium was not going to the front at all! There is always the equivalent of someone with a bayonet at your back (in WWI, the officer has only a pistol–guess what that was for?). You should be aware that soldiers in WWI were very aware of optimal behavior, particularly under what was the most optimal situation to be wounded (the average tenure in the British Army for front-line troops (who were always being rotated and were only in the front-line trench for less than fifty percent of the time) was five months).
It was generally considered that it was best to be wounded in an assault, since your entire body was exposed, whereas in the trench one was more likely to be hit in the head (anyway, this was Robert Graves contention in “Goodbye to All That”).
In actual fact the whole “shooting” development fails to understand the nature of trench warfare in WWI. The only time it mattered for a combat soldier to shoot was when the enemy was attacking, and then he had every incentive to do so, as the attacker would be in the defender’s trench attempting to kill him unless the defender shot him before he got there. Most of the time, however, the soldiers just hunkered down in the trench without shooting at anyone, as no man’s land was deserted during the day, and at night you couldn’t see anything, unless a star shell went off and a member of the enemy’s work party froze too late and at that point your machine guns would open up. No, the main job of the infantry soldier in WWI was to be shelled (constant interdiction fire). Shooting, except in defense against an assault, wasn’t done (in offense, one tried as quickly as possible to get across no man’s land, hoping your work parties had cut the barbed wire (“hangin’ on the old barbed wire”), and then, in the enemy’s trench (if you made it that far), bayonetting, firing pistols or sawed-off shotguns (essentially–the Germans considered the American version a violation of the Geneva convention), and clubbing! (as Graves states, for the first time since medieval
combat the pike made a comeback for British forces). Shooting was a rare event.
Numeric:
My quote about God is not intended to be immutable. In our paper we distinguish between stories which are intended to be immutable (although as you note this is an ideal that is not always attained in the real world of storytelling) and parables, jokes, etc., whose details can be freely invented or changed in the telling.
“Goodbye to All That” is an interesting example: Paul Fussell argued that various details in these stories could never have happened as written.
But you might well be right about the general sense of what was happening on the Western Front. In my paper on the topic, I argued against Axelrod’s model (and, more generally, against what seemed to me as the uncritical acceptance and celebration of his model) but neither he nor I is an expert in the area.
“Goodbye to All That” is a story and should by your definition be immutable, but as you point out, Fussell (wrote “The Great War and Modern Memory” for those unaware–highly recommend it) says details could not have happened and that’s almost certainly true. But read Ludendorff’s memoirs for a very consistent story that often vastly deviates from reality.
I think my main objection to your considering “stories” is that your promoting anomalities and non-mutability criteria is that it ignores the concepts of narrative and context. As an example, one can provide a “story” about the economic and social status of African-Americans that essentially says they are dumb (“The Bell Curve”) or that says slavery was not only profitable, but the slaves enjoyed it too (“Time on the Cross”). Both of these books (the latter by economists) tell consistent stories but are rejected by nearly all academics (particularly the former), though they not only tell a story but also apply generally accepted social science techniques. Where they fail, I would argue, is by underestimating the power of narrative (the consistent “blacks are inferior” meme that has run through American society since Jamestown) and context (generations of enslavement followed by generations of oppression that differed very little in many respects from enslavement).
Numeric:
You raise an important point, which is that “story” has many meanings. I’ve been talking about true stories, i.e. real anecdotes, but of course you’re right that there are fictional stories too.
Just to be clear, though: you write, “‘Goodbye to All That’ is a story and should by your definition be immutable.” But, no, I don’t define stories to be immutable. I say that immutability is a desirable property for a story, if it is to be used to increase our social science understanding.
Regarding your second paragraph: I agree that there’s a lot about stories and storytelling that I’m ignoring. Also I would distinguish between stories (in the sense of anecdotes) and theories or models of the world (such as the racist framework you discuss). These are different things, but my use of the general term “stories” doesn’t help any in distinguishing them.
We’ve run out of levels in this thread so this is in reply to your last comment (which you, as moderator, can apparently add levels). Here is a “story” for you rather than a theory or model of the world that is immutable (I think, though as far as I can tell this means originally sourced). It is referenced in
http://www.historynet.com/fighting-words-from-the-world-wars-to-korea.htm
but there was a write-up in Time about it. It involves the alleged cowardice of
African-American troops, but it is a “story”. From the above website:
THE SLANG OF “THE FORGOTTEN WAR” as the Korean conflict from 1950 to 1953 is often called, has not been forgotten. Much of it originated in World War II, because many of the American troops in Korea were World War II veterans.
One such term was to bug out, or simply to bug, meaning to run away and hence play the coward…It [the Bug-out Boogie] was supposedly first sung by black soldiers of the 24th Infantry Regiment, and one line went, “When them Chinese mortars begin to thud, the old Deuce-Four begin to bug.”
As I recall, the story in Time was nasty to blacks (impugning their bravery, patriotism, etc). Now, this story may very well be true (that is, the blacks of the 24th may have sung it first–African-American culture has typically pioneered most American musical forms–jazz, rhythm & blues, hip-hop, rap), but the reasons for Time emphasizing the black originations of this ditty were firmly rooted in the racism of the day (it should be noted that the Bug-out Boogie became very popular with the troops in Korea, white and black, and the brass forbid it to be sung). So “true” stories can have many interpretations and I’m not certain how that lends itself well to scientific inquiry. I think the post-modernists might be right on this one–why a story is being told is oftentimes more important than the story itself, and talk about forks–I’m going to choose the story that fits my theories, rather than using the theory to explain my choice of story.
numeric says: February 11, 2015 at 3:42 pm
> So “true” stories can have many interpretations and I’m not certain how that lends itself well to scientific inquiry.
Scientific inquiry always involves a story.
> I think the post-modernists might be right on this one
Well you cannot step outside your story (interpretations) to see the evidence as it really is.
> I’m going to choose the story that fits my theories, rather than using the theory to explain my choice of story.
But you need to bend over backwards to discover how your theory choice of story fails in the face of recalcitrant further experience. [Andrew’s comment “thus motivating further theorizing and experimenting.”]
Your only other choice, is the Bug-out Boogie move of why a [always relative but equal valued] story is being told is oftentimes more important than the story itself or William James’ “plastic” or utilitarian truth.
Thanks for your Bug-out Boogie story.
Since you mentioned Einstein: Stories in Physics are a completely different ball game, I think.
Thought experiments are used either as a mental crutch while developing a more rigorous model or later as a pedagogical tool to develop an intuition to how the model is operating. Or to expose paradoxes that might lead to model refinement.
But at the core are always hard numbers & cold equations. When two stories are in conflict differential equations decide the winner. If a story & its underlying model are in tension the model always supersedes the story.
I think that’s a huge difference.
Rahul:
Regarding Einstein: you may have missed this in my earlier comment, but I don’t think of his thought-experiments as “stories” in the sense that I’m using, in which stories provide a sort of empirical evidence in the form of model checking. I would characterize physics thought-experiments as “parables” which help a researcher explore the implications of his model. Which is valuable but is a different thing entirely than stories as immutable evidence. A thought experiment is mutable in its details; it is constrained not by reality but but the logic of its generating theory.
To flip the analogy around: a novel or short story by a serious fiction writer can be considered as a form of thought-experiment. Remember that thing that some writers say, that their characters take on a life of their own, so that the author doesn’t even know what’s coming next? This fits the idea that fiction is a thought-experiment in which the writer explores the implications of his mental model of the world.
Andrew: Seems some like the distinction between icons and indexes (e.g. a painting versus a film exposed photograph) in that the index is supposed to be connected to the world while icons just present a resemblance of something else may be or not in the world.
Stories are necessary to bridge what we think we have experienced (e.g. evidence) and what we hope we might experience (e.g. have evidence of in the future). All cannot be evidence right now.
To paraphrase: My fear in the social science context is that we make non-rigorous stories the ultimate end & not just a tool.
Competing qualitative stories, or even worse, competing storytellers come to decide our view of reality.
Stories are about explaining outliers. They’re about what doesn’t fit the model. The best leads in articles and the best openings in stories contain an element of the unexpected. They tell the reader something he doesn’t expect and he is compelled to read on to get the why. This isn’t in opposition to modeling, it’s a step beyond that makes the truth more nuanced.
“All theory, dear friend, is grey; but the tree of life is ever green.” That’s Goethe, though I probably misquoted him.
On the other hand, the first line of Pride and Prejudice simply states the model’s hypothesis:
“It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.” No outlier there. The outliers come later, in the form of stubborn and atypical characters.
Dan:
You write, “Stories are about explaining outliers.” That can be, but I’d say it in a slightly weaker form: Stories are about highlighting outliers, which further effort might be required to explain.
I really disagree with that. Stories are about explaining patterns.
Sometimes we explain patterns by highlighting an outlier which helps us understand the patterns better.
Elin:
I think the goal of much of social science is to discover natural laws, or regularities, or patterns. But stories can contribute to this by revealing problems with existing explanations, thus motivating further theorizing and experimenting.
They can definitely. I also think there are two kinds of stories in social science, the made up story and the story from data.
Stories from data are also very useful for describing patterns that unfold in a temporal way, as one example. Because social science is dealing with the complexity that results from studying humans in their natural settings, stories — and when I think of stories I’m not thinking of made up stories, I’m thinking of stories of particular pieces of data — are useful for helping to understand and communicate the patterns in complexity. So I actual think that a story can help with illustrating the regularities or patterns. It’s a bit different than model checking, though I think that is an interesting way to think about it and especially how you think about outliers and unusual cases i.e. use that story to understand whether or not that case is so different because of something that you can start to understand.
At the same time, when you are starting something new it is helpful to have a story (again not made up, but an actual story of data) that illustrates a problem or limitation of prior explanations or theories. That’s what you are getting at, and I agree that it is helpful and often the way we work.
In sociology I think people don’t casually tell stories, they pull stories from their data in very deliberate and conscious ways. So if Rob Sampson is in the audience he can tell you the story of a place in Chicago or maybe a story of a particular person in the Glueck study and it will most definitely not be an anecdote, it will illustrate something he learned from really digging into that data and it will help you to understand the complexity of the life histories in that sample. Stories help you visualize complexity.
Then there is the made up story that illustrates what you are talking about. Back when I was working on sentencing models I was trying hard to make people understand that you can’t assume independence when you have some people who are all being sentenced as part of the same case. It’s kind of a technical issue, but real and not that easy. But people could really understand it if presented with the story of a case where one judge is sentencing 4 people for a single crime, then it’s not just some technical issue of a model but something that has real meaning. And you are right, part of that is saying to people look here is why what everyone is assuming is problematic. Everyone can visualize that a who judge has 4 people in front of him all involved in the same event is not really going to think about each one completely separately.
I am surprised by how much of this conversation is totally divorced from the psychological literature – I am no psychologist but I am greatly influenced by Kahneman’s work so there is nothing original here. I believe stories are not only the way we communicate but the way our mind works. Therein lies the problem – when we “analyze” a situation, the stories are already there. Discussing how to use stories correctly assumes that our rational mind (system 2 in Kahneman’s terms) is able to rein in system 1. Nice thought, but most unlikely. Rather, system 1 already has fit the facts into a story and that story may or may not turn out to be correct.
The clash between stories and analysis is that one is governed by system 1 and the other by system 2. They are not always in conflict but they often are. The task is to determine when that occurs and what to do about it. To believe that there is any simple way to distinguish between the “right” stories and those that are antithetical to the analysis is not easy – and, it is more than a matter of analysis. There is no “test” to apply which will determine whether the story is a good communication tool, a test of a theory, or a preconceived notion that has guided the analysis all along.
Can you post the details for your talk, in particular room / building?
Dave:
601 William James Hall. I added a link to the post.
Good luck with your talk today, Andrew!
As a sort of reply to some of the concerns raised by Numeric above, I thought I’d add my take. My long-term project is to develop a critical approach to storytelling that exhibits some of the virtues of statistical analysis. I’m not, in general, impressed with the way statistical analysis is sometimes used to overrule a narrative account. I don’t, that is, always prefer a model to a story. But what I like about models is that they can be critiqued in very straightforward ways by competent statisticians like Andrew. Marshall McLuhan once said, “If you don’t like my ideas, I’ve got others.” Storytellers sometimes take a similar approach (“Any old story will do,” some say.) But modelling doesn’t allow us that luxury. If I don’t like your model, and can show you it’s wrong, then we’ve learned something. We don’t just shrug and bring out another one. The critique has some incisiveness, some teeth.
Interestingly, though I’m inclined to want stories in the social sciences to be “true”, I also know that statisticians are not always comfortable with describing their models that way. The approach that Andrew and I have taken, emphasises “immutability” more than truth. This lets us use stories in fiction, actually. At least there’s a canonical statement of the story; you can get the action in an episode of Star Trek wrong. Likewise, you can get the story of the soldiers in Alps wrong, either as to details (whether or not they knew it was a false map) or the kind of story it is (whether its a “story from the war” or an “incident that happened”). As we point out, if you then also obscure the source, the story becomes way too mutable to be useful. Regardless of whether or not it is true.