Kudos to the Record for taking a poll of undergraduate Presidential preferences. The results, 78/14 for Kerry, are not that surprising, but I might have expected Bush to do just a little better. Some comments:

1) Where is the poll of faculty members? It borders on journalistic malpractice for the Record not to at least try to fairly gauge faculty opinion on the election. I realize that faculty members may not be in a rush to fill out an automated poll (the method that the Record used for undergraduates), but that is no excuse for not trying. Moreover, it wouldn’t be that hard to just ask 25-50 faculty members directly.

2) Where is the analysis of other faculty indicators? I have already looked into faculty donations this electoral cycle, but the Record could do a much more thorough job. It would also be interesting to know the party breakdown of faculty voter registrations.

3) The Record ends by noting that “The survey was sent to 600 students, with 375 responses. The margin of error was 4.6 percent.” First. shouldn’t the Record provide a better description than “margin of error”? I suspect that what they mean is that, speaking frequentistly, there is a 95% probability that the percentage of students planning to vote for Kerry is between 69.2 and 87.6 — the mean value of 78.2 +/- 2 times the standard error of 4.6.

But even this is problematic for two reasons. First, I calculate a standard error of 2.1%. I haven’t done these sorts of statistics in a while, but I am not sure how the Record gets 4.6. Maybe they aren’t really calculating a standard error? Did they use N = 600 (total surveys) instead of N = 375 (total responses)? Using N = 600 is definately wrong.

Second, I don’t think that these are even the right formulas to use. Recall that the Record is only interested in calculating the voting preferences of current Williams students. So, they have actually surveyed around 1/6 of the target population. This is very different from the typical poll that tries to draw inferences about the entire US population. This suggests, perhaps, that the Record’s confidence intervals should be much more narrow than, say, the New York Times would be even if they both have the same sample size.

Consider the extreme case where the sample is 2,000. The New York Times (since it has a sample of 2,000 out of 200+ million) would still have a non-smallish standard error. But the Record would have a standard error of zero since it has surveyed everyone who it cares about, i.e., the entire population of Williams students. It knows the answer perfectly.

In other words, I would expect that the 95% confidence interval for the Record should be much more narrow than the standard formulas would suggest.

I’ll leave this as a simulation exercise for the students in STAT 201.

Print  •  Email