HN2new | past | comments | ask | show | jobs | submitlogin

Two fallacies:

1. The non-responders are not responding for similar reasons.

2. Those reasons are somehow related to their opinions.

Both are clearly false. You cannot assume anything about their opinions from the fact that they didn't respond. All you have is their (non)action - which could have been caused by being ill, busy, distracted by personal issues, absent, drunk, or for any other number of reasons.

None of the answers are correct. If you wanted representative data you would have to:

1. Confirm with a different sample but also...

2. Use multiple different channels and methods (phone, in-person survey on campus, maybe an email survey with some kind of benefit to encourage participation, etc) to try to eliminate the non-responders who have personally valid but statistically irrelevant reasons (away, ill, etc).



As the blog post says non-responders do tend to have different opinions on average than responders. Typically they do try to demographically determine who those non-responders are so they can adjust and increase the weight of that group proportionally to their limited response. This is how most polling works in the real world and it does provide better accuracy.


Suppose the non-responders are identical to the responders. Then nothing is lost by trying to get them to participate again. But if they are not, replacing them will lose information.

> ill, busy, distracted by personal issues, absent, drunk

But being ill correlates (lightly) with being older, and being drunk heavily correlates with opinions. Being absent might correlate with wealth (and not even linearly). Personal issues probably correlate with education. Add them together, and you've got a fairly large part of the population missing from your sample.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: