Polling by asking people about their neighbors: When does this work? Should people be doing more of it? And the connection to that French dude who bet on Trump

Posted on by

Several people pointed me to this news report on a successful bettor in an election prediction market:

Not only did he see Donald Trump winning the presidency, he wagered that Trump would win the popular vote—an outcome that many political observers saw as unlikely. . . . He made his wagers on Polymarket, a crypto-based betting platform, using four anonymous accounts. . . . In messages sent privately to a reporter before Election Day, Théo predicted that Trump would take 49% or 50% of all votes cast in the U.S., beating Harris. He also predicted that Trump would win six of the seven battleground states. . . .

In his emails and a Zoom conversation with a reporter, Théo repeatedly criticized U.S. opinion polls. . . . Trump had overperformed his swing-state polling numbers in 2020. . . .

So what did this bettor do?

To solve this problem, Théo argued that pollsters should use what are known as neighbor polls that ask respondents which candidates they expect their neighbors to support. The idea is that people might not want to reveal their own preferences, but will indirectly reveal them when asked to guess who their neighbors plan to vote for.

Théo cited a handful of publicly released polls conducted in September using the neighbor method alongside the traditional method. These polls showed Harris’s support was several percentage points lower when respondents were asked who their neighbors would vote for, compared with the result that came from directly asking which candidate they supported. . . .

The data helped convince him to put on his long-shot bet that Trump would win the popular vote. . . . he had commissioned his own surveys to measure the neighbor effect, using a major pollster whom he declined to name. The results, he wrote, “were mind blowing to the favor of Trump!” . . . he argued that U.S. pollsters should use the neighbor method in future surveys to avoid another embarrassing miss.

Steve Shulman-Laniel sent this to me and wrote:

If it’s such an obvious method that some random rich French guy would use it and get better results than ordinary RDD [random-digit-dialing] polling, then we’d already be using it. If that’s not true, then that also seems interesting. Either we’re just leaving money on the table (in the sense that we’re ignoring a method that would improve poll results, in expectation) or there’s some good reason why pollsters don’t habitually use this method. (I guess a third option is that we’re already using this method, that the WSJ is overselling how novel it is, and that it hasn’t borne the fruit that the WSJ claims it would.)

I would not say that the above-quoted Wall Street Journal is overselling the novelty of neighbor polling—nowhere does it say that the method is new—but I see where Laniel is coming from. The other thing is that a lot of polls don’t use random digit dialing; they use internet panels. So, yes, there are methods that give better results than RDD, or, at least, no worse than RDD, and people are using these methods.

But, what about these neighbor polls? I don’t know how long they’ve been going on, but Julia Azari and I did suggest the idea in our 2017 paper, 19 Things We Learned from the 2016 Election, where we wrote:

We recognize the value of research into social networks and voting, especially in a fractured news media environment and declining trust in civilian institutions. In future studies, we recommend studying information about networks more directly: instead of asking voters who they think will win the election, ask them about the political attitudes of their family, friends, and neighbors.

We also discussed problems with just asking respondents who they think will win the election: this approach just invites them to spit back what they’ve already seen in the news media.

I’ve put “How do you think your family, friends, and neighbors?” questions on polls from time to time but can’t right now put my hands on any of the results.

Back in 2016 when discussing “How many people do you know?” surveys, Tian Zheng, Matt Salganik, and I pointed out that one advantage of asking people about their social networks is that it’s a way to use a survey to learn about people other than respondents.

Given that differential nonresponse has been such a problem with recent surveys, there’s an appeal to asking people about their social networks as a way to indirectly reach those hard-to-reach people.

I don’t know how many survey organizations followed our advice from 2017 to asking about the political attitudes of family, friends, and neighbors, but we did discuss once such attempt in 2022. At that time, the neighbor poll didn’t do so well. Jay Livingston shared the story:

They went to the Marist College poll and got the directors to insert two questions into their polling on local House of Representatives races. The questions were:

– Who do you think will win?

– Think of all the people in your life, your friends, your family, your coworkers. Who are they going to vote for?

At the time, the direct question “Who will you vote for?” the split between Republicans and Democrats was roughly even. But these new two questions showed Republicans way ahead. On “Who will win?” the Republicans were up 10 points among registered voters and 14 points among the “definitely will vote” respondents. On the friends-and-family question, the corresponding numbers were Republicans +12 and +16.

On the plus side, that result was so far off that nobody took it seriously. Yes, it was featured on NPR, but more as an amusing feature story than anything else.

Here’s what I wrote at the time:

So what happened? One possibility is that Republican family/friends/coworkers were more public about their political views, compared to Democratic family/friends/coworkers. So survey respondents might have had the impression that most of their contacts were Republicans, even if they weren’t. Another way things could’ve gone wrong is through averaging. If Republicans in the population on average have larger family and friend groups, and Democrats are more likely to be solo in their lives, then when you’re asked about family/friends/coworkers, you might be more likely to think of Republicans who you know, so they’d be overrepresented in this target group, even if the population as a whole is split 50/50. . . .

Also it would be good to see exactly how that question was worded and what the possible responses were. When writing the above-quoted bit a few years ago, I was imagining a question such as, “Among your family and friends who are planning to vote in the election, how do you think they will vote?” and then a 6-point scale on the response: “all or almost all will vote for Republicans,” “most will vote for Republicans,” “slightly more will vote for Republicans,” “slightly more will vote for Democrats,” “most will vote for Democrats,” “all or almost all will vote for Democrats.”

I hope they look into what went wrong. It still seems to me that there could be useful information in the family/friends/coworkers question, if we could better understand what’s driving the survey response and how best to adjust the sample.

So that’s where things stood in 2022.

So what about 2024? Apparently this year the neighbor polls did better. I’d like to see the data, as all we have now is third-hand reporting—a journalist telling us what a bettor told him about some polls we haven’t seen. I’m not saying anyone’s trying to mislead us here; it would be just good to see the information on which the claims were based.

Summary

1. Polling people by asking about their neighbors is an interesting idea.

2. We have an example in 2022 where it didn’t work,

3. We have an example in 2024 where someone claims it worked well.

4. To the extent that existing survey approaches continue to have problems, there will always be interest in clever ways of getting around the problem of nonresponse.

5. If you’re doing a neighbor poll, I think you’ll want to adjust it using the usual poststratification methods and maybe more. I’m not quite sure how—it’s an applied research problem—but I imagine something can be done here.

6. These principles apply to surveys and nonresponse more generally, not just election polling!

Unlike the French dude in that interview, I’m offering all this advice for free. I’m fortunate to be supported by public funds, and the least I can do is share as much of my understanding as is feasible.

Did the French guy know what he was doing with the network sample? I can’t really say, given that my knowledge of the story is at third hand. My guess is that he got lucky, but it’s not just luck. He got lucky in that this year the surveys of neighbors got the right answer—but that hasn’t always been the case. It’s not just luck because he had to use his judgment to decide what to do with that survey data, and he gets credit for making the right call.

P.S. Economist Peter Dorman writes:

About 20 years ago I was asked to do a study for the International Labor Organization on whether child workers were more exploited than adults across a set of occupations and countries. That required collecting a lot of data, both on how much kids are paid and how productive (in economic value terms) their labor is, both compared to adults. I was worried that employers would be reluctant to reveal how much they paid underage workers or even admit they employed them, since it’s all illegal, so in addition to direct questions on the employer questionnaire (matched to worker questionnaires), I asked, “How much do you think other employers pay children for this type of work?” In the writeup I referred to these as ecological questions. Lo and behold, direct and ecological responses were very highly correlated, and not much would change from using either.

At the time I was under the impression that this was a well known technique, and I didn’t even reference it in the writeup. But was I wrong?


For thinking about some of these general issues, I recommend my article, Learning about networks using sampling, published in 2017 in the Journal of Survey Statistics and Methodology.

I did some quick searching on the topic and found this paper, Perceptions of others’ opinions as a component of public opinion, by Carroll Glynn, published in 1989 in the journal Social Science Research:

This study investigates some relationships between stated opinions and perceptions of others’ opinions, clarifying certain ambiguities in the use of perceptual approaches in public opinion research. The study provides evidence that respondents perceive others as similar to themselves in opinions and values, suggesting support for the “looking glass hypothesis.” However, there was also evidence of an “ideological bias”—respondents tend to see neighbors as having more conservative opinions than their own and to see others living in the city as having more liberal opinions than their own. The study indicates that perceptions are important in understanding public opinion mechanisms but that we are far from a full understanding of the underlying process involved.

Also this paper, Social sampling explains apparent biases in judgments of social environments, by Mirta Galesic, Henrik Olsson, and Jörg Rieskamp, published in 2012 in Psychological Science:

How people assess their social environments plays a central role in how they evaluate their life circumstances. Using a large probabilistic national sample, we investigated how accurately people estimate characteristics of the general population. For most characteristics, people seemed to underestimate the quality of others’ lives and showed apparent self-enhancement, but for some characteristics, they seemed to overestimate the quality of others’ lives and showed apparent self-depreciation.

I imagine there must be some literature on the technique of asking survey questions about family/friends/neighbors and how to adjust the resulting data to get estimates for the general population. But I’m not sure where to look, and I couldn’t find any references on the topic. Googling “neighbors” in the survey research literature gets me lots of references on neighborhood effects in sociology; googling “neighbors” and “sampling” takes me to nearest-neighbor methods in statistics and machine learning; googling “network sampling” takes me to the computer science literature . . . nothing on how to survey people and ask about their neighbors.

P.P.S. In comments, Isaac Maddow-Zimet points to this article from 2019, Evaluating sampling biases from third-party reporting as a method for improving survey measures of sensitive behaviors, by Stéphane Helleringer, Jimi Adams, Sara Yeatman, and James Mkandawire, which states:

Survey participants often misreport their sensitive behaviors (e.g., smoking, drinking, having sex) during interviews. Several studies have suggested that asking respondents to report the sensitive behaviors of their friends or confidants, rather than their own, might help address this problem. This is so because the “third-party reporting” (TPR) approach creates a surrogate sample of alters that may be less subject to social desirability biases. However, estimates of the prevalence of sensitive behaviors based on TPR assume that the surrogate sample of friends is representative of the population of interest. . . . we suggest approaches to strengthen estimates of the prevalence of sensitive behaviors obtained from TPR.

Interesting. With election polling, the issues are slightly different. We have no reason to expect systematic misreporting of vote preferences; rather, our concern is differential nonresponse: in 2024, this would be Republican voters being less likely than Democratic voters to respond to the survey in the first place.

So the motivation for asking about friends and neighbors in an election poll with concerns about nonresponse is different from the motivation for asking about friends and neighbors in a social survey with concerns about insincere responses. Still, there should be some common lessons from these two different problems.

44 thoughts on “Polling by asking people about their neighbors: When does this work? Should people be doing more of it? And the connection to that French dude who bet on Trump”

  1. “If it’s such an obvious method that some random rich French guy would use it and get better results than ordinary RDD [random-digit-dialing] polling, then we’d already be using it.”

    Heheheheh. I’m not arguing that the method is better, but the ridiculousness of that logic is impressive. There have been *all* sorts of better methods in a variety of fields that were ignored for bad reasons.

    Reply ↓

    • Total:

      If you read the news article, the French guy said he’d looked at public polls that had asked about neighbors. So, even from there, it was clear that people had already been using this survey method. The real question is what to do with these responses! In some settings, it might make sense to average them as is; other times, some modeling and adjustment will be necessary.

      I agree with your general point that, in science, there are often good methods available that are not in common use.

      Reply ↓

      • Financial Times says that the trader also commissioned his own polls, but “Théo declined to share those surveys, saying his agreement with the pollster required him to keep the results private”

        Sounds more than a bit suspicious, but who knows?

        Reply ↓

    • Back in the 80s my classmates and I presented a poster session on a structural analysis of the Jennings Activity Survey for Students (a Type A questionnaire) and found the questions and the center of the Type A space were something like “Does your family think you’re hard-driving and competitive?,” “Do your friends think you’re hard-driving and competitive?,” and “Does your spouse or partner think that you’re hard-driving and competitive?”

      In the case you’re talking about though, are we recording revealed preferences or aggregated information?

      Reply ↓

      • Barry:

        This reminds me of surveys that I remember reading about many years ago (the 80s, I guess!), where college students were asked two questions:
        1. Do you cheat on exams?
        2. Does your roommate cheat on exams?
        Or something like that. I don’t remember the details. Anyway, the way I remember it was that the percentage of Yes responses was much higher for question 2 than for question 1.

        Again, this is different than political polling, because here our concern is not that respondents are not reporting their true vote preferences; rather, we are concerned about differential nonresponse.

        Reply ↓

        • “Again, this is different than political polling, because here our concern is not that respondents are not reporting their true vote preferences; rather, we are concerned about differential nonresponse.”

          I have seen concern expressed in the past that people don’t always report their true voting intention. If for example it is an unpopular choice in their circle.

    • Isaac:

      Thanks. I added a P.P.S. to the above post, which I’ll repeat here:

      From the abstract of the the paper you link to:

      Survey participants often misreport their sensitive behaviors (e.g., smoking, drinking, having sex) during interviews. Several studies have suggested that asking respondents to report the sensitive behaviors of their friends or confidants, rather than their own, might help address this problem. This is so because the “third-party reporting” (TPR) approach creates a surrogate sample of alters that may be less subject to social desirability biases. However, estimates of the prevalence of sensitive behaviors based on TPR assume that the surrogate sample of friends is representative of the population of interest. . . . we suggest approaches to strengthen estimates of the prevalence of sensitive behaviors obtained from TPR.

      Interesting. With election polling, the issues are slightly different. We have no reason to expect systematic misreporting of vote preferences; rather, our concern is differential nonresponse: in 2024, this would be Republican voters being less likely than Democratic voters to respond to the survey in the first place.

      So the motivation for asking about friends and neighbors in an election poll with concerns about nonresponse is different from the motivation for asking about friends and neighbors in a social survey with concerns about insincere responses. Still, there should be some common lessons from these two different problems.

      Reply ↓

  5. Rajiv Sehti in the link below:

    “The question of whether prediction markets are more accurate on average than statistical models can only be answered with data, it cannot be answered by logic alone,” he wrote in an email to CoinDesk. “We need to look at individual state outcomes, the popular vote, congressional races, and all other events for which models and markets were simultaneously generating forecasts. And we also need to compare markets with each other to discover what designs work best.”

    So the comparison between polling (and fundamentals) modeling, and the betting markets, is an empirical question.

    It’s interesting to me how easy it is to lose sight of that obvious fact.

    We’re such pattern-finding machines. We see one or two high profile instances of a particular pattern and jump on it to think we have a good grasp on what’s going on.

    https://www.coindesk.com/markets/2024/11/06/polymarket-resolves-presidential-election-contract/

    Reply ↓

  6. Is there an assumption that non-response bias would disappear with 3rd party reporting on the context of elections?

    Why? If the theory is that Republicans (or Trump supporters) are less likely to respond to pollsters, what about the methodology of 3rd party polling would change that dynamic?

    Reply ↓

  7. What are some “usual poststratification methods” that you use? Could you be as specific as possible and if possible also provide a link that explains how to use them. Also, have been looking for a free resource of how to use multilevel regression and poststratification and didn’t find anything on google. Was wondering if you have a suggestion for either a textbook or a link to an online textbook.

    Reply ↓

  8. The neighbor poll idea seems very vague to me. The idea that it might get around people that don’t respond or respond untruthfully has some merit, but such polls will have inevitable issues of who are your neighbors, who is part of your social network, who is part of your family, etc. If you asked me about my neighbors, I was actually incorrect about a few close neighbors. But I don’t know my neighbors very well. My family I know better and can be pretty sure how they voted – but I don’t think polling me about that offers any advantages over just asking me. Social networks are inherently difficult to quantify – how “close” and “how many?” Ask people how they think their 10 best friends will vote – I see too many problems with questions like that. Still with response rates as low as they are, and some people unwilling to truthfully answer polls, there is plenty of room for a poor method to still be better than the usual polls.

    And if we find someone use a neighbor poll and it is somewhat accurate, then the conclusion might be that it worked. Until the next attempt does not work. We’d need quite a few of these polls to get meaningful comparisons with the usual polls. My gut feeling is that neighbor polls address one particular problem with polling, but opens up a number of additional problems.

    Reply ↓

  9. I found one question, from a poll sponsored by the Kaiser Family Foundation in September. 58% of those who had a preference said they would vote for Harris, 42% for Trump (the sample only included women). When asked how most of their friends would vote 38% said Harris, 29% Trump, 31% not sure. Exit polls showed Harris with about a 10% lead among women in the actual vote, so you could say the question about friends gave a better prediction. But in 2020, Fox News had two surveys that asked who you thought your neighbors were supporting–in August it was 39% Trump to 34% Biden, and in October 49% Trump and 38% Biden (for own vote, Biden led 52-43 in August and 53-43 in October). So the whale would have taken a bath if he’d tried his approach in 2020.

    Reply ↓

    • When asked how most of their friends would vote 38% said Harris, 29% Trump, 31% not sure.

      Seeing that, I realize that there would have to be a huge error bar on this kind of survey as some 35% of eligible voters don’t vote. Seems a good way to check this kind of survey for accuracy would be to include a “won’t vote” option to see if it came out to around 35%. (And then there’s the whole question of whether they’d even know if their neighbor is registered to vote).

      Reply ↓

      • There was also an option for “most won’t vote” and only 2 or 3 percent chose it, so the “not sure” should probably be understood as “not sure how they will vote.” The question about how your friends would vote was only asked of people who said that they would vote one way or the other, so it makes sense that the great majority thought that most of their friends would vote too. I don’t know why the “not sure” share was so much larger in the 2024 survey than the 2020 surveys.

        Reply ↓

        • I was canvassing a couple of weeks ago for my Congressional representative and knocked on the door of someone who didn’t know anything about the candidates running. He said he doesn’t care about politics, doesn’t think any politicians are any good, and doesn’t vote.

          I live in one of the most hotly contested House districts in the country, NY-19. Most people I talked to told me how sick there are of being hassled for their vote. They said they were hearing about it non-stop.

          I walked away puzzled. Who was this person who didn’t even care enough to even bother to vote? I thought about how different than he is than I am. Of course I can understand thinking your vote won’t make a difference and I can understand thinking they’re all crooks and don’t give a damn about me and it won’t make any difference who it is that gets elected.

          But still, I was puzzled – how could this guy not even care?

          OK, we all mostly live in bubbles, and that guy’s bubble is similar to some 1/3 of eligible voters.

          I wonder what surveying eligible voters who don’t vote might reveal? I wouldn’t be entirely surprised, if in some counterintuitive way, such a survey turned out to be predictive of election outcomes.

  10. Personally, I think this is just someone with a ton of money (I read that he bet 30 million) who got marginally lucky. If in some alternative universe someone had accurately predicted that Harris would win the electoral college but lose the popular vote I’d be impressed: the day before the election I told someone that was about the only scenario I thought unlikely to happen.

    I guess it goes back to whether this method is really any better than the models that, say, Silver produced. If you were to place a bet on a specific outcome, would it be terribly surprising to choose the outcome that occurred most often in the simulations? From what I understand, Trump’s actual map was the same as the modal outcome of Silver’s final model.

    Reply ↓

    • There was a model that accurately predicted that Trump would win the popular vote and also all seven swing states:

      https://www.cambridge.org/core/services/aop-cambridge-core/content/view/90DA5291682CEA6BDA943208C0E7E649/S1049096524000994a.pdf/understanding-bidens-exit-and-the-2024-election-the-state-presidential-approvalstate-economy-model.pdf?utm_source=substack&utm_medium=email

      Note: this model, like those I’ve studied and that Andrew occasionally mentions, use no polls and only a few economic variables and the president’s approval rating. Also, the paper is disappointingly missing key mathematical details of how the “simulations” (their description) are done.

      Reply ↓

      • Sam:

        Interesting. Prediction based on economic conditions makes sense. Without looking at all the details, I’m guessing that the simulations don’t matter so much, nor will the state-level economic data be so relevant. The forecast worked out because they predicted the national popular vote accurately. Given that, the states lined up as expected.

        Their fundamentals-based national vote estimate was pretty close to others I’ve seen. There will be differences based on exactly which economic predictor you use and how much you include presidential approval as a predictor. And the model has residual standard deviation. I guess what I’m saying is that their model is fine, I wouldn’t say it’s better or worse than other fundamentals-based regressions, but, sure, they get credit for happening to predict with zero error this time, which is fair enough given that their model would get dinged if their prediction happened to be one standard deviation away from the outcome (which of course would be expected to regularly occur).

        Reply ↓

  11. Is any of this related to Galton’s famous “wisdom of the crowd”?

    https://en.wikipedia.org/wiki/Wisdom_of_the_crowd

    Somehow or other, averaging opinions of the neighbors is better than the faulty/misleading scale of the individual in question. An unexpected example from the above website is

    “Non-human examples are prevalent. For example, the golden shiner is a fish that prefers shady areas. The single shiner has a very difficult time finding shady regions in a body of water whereas a large group is much more efficient at finding the shade.”

    This article also uses the term “dialectical bootstrapping” which must mean something truly deep.

    Reply ↓

    • To my surprise, the term, dialectical bootstrapping, has been around for awhile:

      https://www.science20.com/news_releases/dialectical_bootstrapping_better_answers_when_you_cant_use_lifeline#:~:text=Dialectical%20bootstrapping%20is%20a%20method%20by%20which%20an,be%20created%20and%20combined%20in%20the%20same%20mind.
      ———————————————————————————–
      “Dialectical bootstrapping is a method by which an individual mind averages its’ own conflicting opinions, thus simulating the “wisdom of the crowd.” In other words, dialectical bootstrapping enables different opinions to be created and combined in the same mind. For example, in this study, participants were asked to identify dates of various historical events. After they gave their initial answer, the participants were asked to think of reasons why the answer may be wrong and were then asked to come up with an alternative second (dialectical) answer.

      The results in Psychological Science reveal that the average of the participants’ first answer with the second answer was much closer to the correct answer, compared to the original answers on their own. In addition, the dialectical bootstrapping method (that is, thinking about why your own answer might be incorrect and then averaging across estimates) resulted in more accurate answers compared to simply making a second guess without considering why the first answer may be wrong.”
      ——————————————————————————————————
      My first reaction to the above is that this sort of mental gymnastics must be culturally specific. Upon reflection, I agree with what I just wrote.

      Reply ↓

  12. How about the solution to the riddle with the two guards, one that lies one that tells the truth

    Ask them, if I asked your neighbors who would they say you’re going to vote for?

    :) I’m sort of joking, but do wonder whether this style of questioning could be revealing, just like it is in the riddle.

    Reply ↓

    • Jessica:

      Interesting. I followed the link. The abstract makes a claim, “We show that, compared with own-intention questions, social-circle questions that ask participants about the voting intentions of their social contacts improved predictions of voting in the 2016 US and 2017 French presidential elections,” that did not seem to be supported by the summary data in the paper’s Table 1, which resents results for the 2016 election. It says in Table 1 that one of the social-circle polls estimated Clinton to have 53.5% of the two-party support, and the other estimated Clinton to have 47.9% of the two-party support. Actually she received 51.1%, and the aggregate polls gave her 52.2%. Both of the social-circle polls were off by a lot compared to the aggregate polls. So I don’t get how in the caption to that figure, they say that both of their polls “have satisfactory accuracy.” They write, “USC’s social-circle questions were the only ones that predicted Trump winning the majority of the electoral votes”—but that was a poll that was way off, it just happened to be way off in that particular direction. Also they are proud that their approach “correctly predicted” swing states, but the swing states were so close that predicting their sign is basically a coin flip.

      In short, I do think these sorts of questions have promise, but I think they’re also subject to hype, as in that linked paper. I remain very sympathetic to further research in this area—with realistic expectations.

      Reply ↓

      • Yeah I’m not sure how they are drawing those conclusions (was just posting the paper because it claims to study the general method). Elsewhere they say “In the United States, social-circle questions were more accurate than own-intention questions in predicting the whole distribution of vote shares for different candidates” and point to Table 1. But Table 1 seems very mixed … seems the electoral college vote estimates from social circle overestimate Clinton’s performance less, and two of the error measures are lower for these methods, but looking at the vote share estimates I don’t get it either.

        Good to know about the USC poll being way off… after 2016 I remember hearing several economists praising it as if it were better than others. Maybe because they read this paper!

        Reply ↓

      • Author of the paper in question here. We have investigated the social-circle question in several elections. Just to be clear, we do not claim that this question used in a single poll outperforms the aggregate of 1,106 national polls. What we show in eight elections so far is that the social-circle question often outperforms the own intentions question asked in the same poll. In other words, predictions based on the same participants are often more accurate when the participants are asked about their social circles than about their own intentions. We’ve found this in the US elections 2016, 2018, 2020, and 2024, the 2017 French and Dutch elections, 2018 Swedish elections, and 2024 Austrian elections.

        We and others before us have investigated variants of such wisdom-of-crowd questions, including the well-known winner expectation question – references can be found in our papers below. Our results up to 2020 elections have been summarized in (all papers available from my website):

        Galesic, M., Bruine de Bruin, W., Dalege, J., Feld, S. L., Kreuter, F., Olsson, H., Prelec, D., Stein, D. L., & van der Does, T. (2021). Human social sensing is an untapped resource for computational social science. Nature, 595, 214–222.

        and in more detail in:
        Olsson, H., Galesic, M., & Bruine de Bruin, W. (2023). Social sampling for judgments and predictions of societal trends. In K. Fiedler, P. Juslin, & J. Denrell (Eds.), Sampling in judgment and decision making (pp. 385-416). Cambridge University Press.

        And specific elections in the following papers:
        Olsson, H., Galesic, M., Bruine De Bruin, W., & Prelec, D. (2023). Combining survey questions with a Bayesian bootstrap method improves election forecasts. Manuscript submitted for publication. https://doi.org/10.31219/osf.io/nqcgs

        Bruine de Bruin, W., Galesic, M., Bååth, R., de Bresser, J., Hall, L., Johansson, P., Strandberg, T., van Soest, A. (2022). Asking about social circles improves election predictions even with many political parties. International Journal of Public Opinion Research, 34

        Galesic, M., Bruine de Bruin, W., Dumas, M., Kapteyn, A., Darling, J., & Meijer, E. (2018). Asking about social circles improves election predictions. Nature Human Behaviour, 2, 187-193.

        Reply ↓

      • Kevin:

        Given that they said in their 2018 paper that their method performed well in 2016, but that claim was contradicted by the actual data in their 2018 paper, I’m skeptical about their claims about 2020. Also, I think they’re missing the boat by focusing on the idea of inaccurate responses rather than considering the problem to be nonresponse.

        That said, I do think this is all worth looking into. I did some googling and also found these relevant papers: https://osf.io/preprints/osf/nqcgs and https://drive.google.com/file/d/1bZFYalCMNUomhZ1Z99JHXX-x0FOnZqvG/view?pli=1

        Reply ↓

        • Again, just to be clear: the social circle question often outperforms own intention questions within the same poll. No claims are made about that one poll outperforming aggregate polls (see the references cited in my post above). To clarify another point: as described in more detail in these papers, our main explanation for the advantage of the social-circle question is actually related to nonresponse – people’s reports about their social circles seem to compensate somewhat for the systematic biases due to nonresponse (e.g., Galesic et al., 2021, Nature). In addition, perceptions of own social circles might influence own voting in the future, and as is well known, the social-circle question can help with biases due to social desirability.

        • Mirta:

          For political polling, I don’t think social desirability bias is an issue. My concern is more with differential nonresponse and uncertainty about turnout. But, sure, political polling is not the only reason for surveys, and, in other settings of social and health surveys, social desirability bias is a concern.

  14. This discussion reminded me of an old paper by Orley Ashenfelter that studies the impact of education on earnings using survey data. One big problem seemed to be that there is a lot of measurement error in self-reported education. As a solution, they’ve asked twins to report on each others education & used twin A’s reported education for twin B as an instrument for twin B’s education. You can find an ungated version of the paper here: https://piketty.pse.ens.fr/files/AshenfelterKrueger1994.pdf

    Reply ↓

  15. A classic paper on this issue is “Public Beliefs About the Beliefs of the Public” by James M. Fields and Howard Schuman
    Public Opinion Quarterly , Winter 1976-1977, Vol. 40, No. 4, pp. 427-448. The abstract reads:
    Perceptions of the opinions of others are examined on a variety of issues using
    data from three sample surveys of metropolitan Detroit. A greal deal of inaccuracy in such
    perception is evident. Three broad tendencies or patterns can be discerned: “looking glass
    perceptions,” the general propensity to believe that others’ opinions are the same as one’s
    own; “conservative bias,” the belief that the population is more conservative on racial is-
    sues than it actually is; and limited response to reality constraints. The overall findings suggest
    that perceptions of public beliefs and attitudes are personally and socially constructed
    to a much larger degree than is often assumed.

    Reply ↓

    • Kris:

      Thanks for the link. It could be that methods for reducing biases in survey responses (as in the paper you link to) will work differently than methods for reducing biases from differential nonresponse (which I think is a problem with election polls).

      Reply ↓

  18. The neighbors question reminds me of Oscar prediction opinion pieces, in which they draw the difference between “who you think should win” and “who you think will win”. Have anyone studied this type of questions?

    Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *