Total survey error

Erez Shalom writes:

It’s election time in Israel and every week several surveys come out trying to predict the ‘mandates’ that each party will get (out of a total of 120).

These surveys are historically flakey, and no one takes the ‘sampling error’ they come with seriously, but no one has a good idea of what the right error bars should look like either.

So, the question is this: Assuming I have a set of surveys over several weeks/months, as well as an equivalent set from the last election (plus that election’s results), is there a reasonable way to infer the right uncertainty we should have over such survey results?

My reply: You could start with a graph! I agree that you won’t get much out of the sampling error. If you can break things down in some ways, that could help. In the U.S. we can do this using the 50 states (as in this article with Kari Lock); Israel possibly has regions of some sort where votes are tabulated and survey respondents can be classified?

I don’t know anything about Middle Eastern politics but I do recall that Israel has a zillion little political parties. Voting with multiple parties and candidates is typically hard to predict, compared to two-party, two-candidate elections.

Beyond all this, I assume some Israeli journalists are doing something like this already, so you could start with that.

4 thoughts on “Total survey error

  1. Project 61 is supposed to be the Israeli equivalent of fivethirtyeight, but I don’t know enough Hebrew or enough about Israeli elections to figure out if that comparison is apt.

  2. Just ran across this (I do rws Hebrew). There are many difficult things about election predictions in Israel, not the least of which are the number of parties represented :). Just sayin.

    The US interpretation of voting in Israel would be akin to ‘party ticket’ voting, except that the number of votes don’t typify binary outcomes (win/lose), but rather rank based on proportion of votes. Voting rules are also not based on defined geographic boundaries, like states, the whole country is treated as a single basket ‘o’ bananas.

    There are about 120 seats to fill as I recall – most with a 4 year term – but some seats are filled (for whatever reason) on off-election years. The thing being predicted here is a proportion of popular voting for each party to determine seats for each, and the voting distribution is skewed right enough that such a survey (it seems like the table for project 61 hints well at this) is unlikely to catch observations but for the roughly 4-7 top parties (and I may be stretching that). So, the question then becomes ‘how low you wanna go?’

Leave a Reply to Jay Cancel reply

Your email address will not be published. Required fields are marked *