## Question 4 of my final exam for Design and Analysis of Sample Surveys

4. Researchers have found that survey respondents overreport church attendance. Thus, naive estimates from surveys overstate the percentage of Americans who attend church regularly. Does this have a large impact on estimates of time trends in religious attendance?

Solution to question 3

From yesterday:

3. We discussed in class the best currently available method for estimating the proportion of military servicemembers who are gay. What is that method? (Recall the problems with the direct approach: there is no simple way to survey servicemembers at random, nor is it likely that they would answer such a question honestly.)

Solution: I was talking about the work of Gary Gates, combining an estimate of the percentage of gays in the population with an estimate of the probability that someone is in the military, given that he or she is gay.

1. DK says:

No, it does not. The slope remains the same, only offset changes.

2. zbicyclist says:

No.

The assumption is that p(say you attend church | you don’t) is reasonably constant.

3. awm says:

This depends on whether or not the level of overreporting has changed over time. My guess would be that overreporting has increased, but this is an empirical question that I assume someone has looked into.

4. conchis says:

Depends on what you think drives the overreporting. If overreporting is driven by societal expectations of attendance, then it may shrink as attendance falls, which would lead to an overstatement of the time-trend.

5. Priapus says:

The proportion that are less than honest will be related to the proportion attending (social pressures, shame etc.) So yes.

6. Peter Flom says:

It depends on whether the over-reporting is constant or not.

7. Simonini says:

Even if a constant proportion of non-churchgoers report going to church, the fraction of the population that these non-churchgoers make up changes over time. So yes, it will have an impact, although it could be large or small.

8. Simon says:

My first reaction is to say that it should make no difference to the overall trend. It may look as though the figure has dropped from 80% to 70% over a set period of time, but if the reality is that it’s dropped from 70% to 60% during that time, it doesn’t matter.

But now that I think about it, I realise that it’s more complicated than that. There will probably be a tipping point (or, more realistically, a series of tipping points in each sub-demographic) where people no longer feel the need to pretend that they’re going to Church, and this will be linked very closely to how many people are actually going. If 70% of the public are going to Church, then there’s a strong incentive for the other 20% to hint that they are as well. If 30% of the public are going, that incentive becomes much weaker for the non-Churchgoers.

This means that if Churchgoing fell from 70% to 20% over a long time frame at a consistent rate, it may well look as though the fall started off gently, then got much faster and then slowed down again even though the truth of the matter is that it had remained steady.

In other words, the over-reporting should have no impact on the DIRECTION of the trend, but may have a significant impact on the MAGNITUDE of the trend.

9. […] yesterday: 4. Researchers have found that survey respondents overreport church attendance. Thus, naive […]

10. Robert Birkelbach says:

Another (yet rather similar) method would be to use this approach: http://econ.ucsb.edu/~doug/240a/Lectures/Conditional%20Probability%20and%20Bayes%20Rule%20-%20Transparencies.pdf
Andrew: Which one would you prefer?

• Robert Birkelbach says:

(In reply to question 3 of course)