- EphBlog - http://ephblog.com -

“Vote Kerry” say Ephs

Kudos to the Record for taking a poll of undergraduate Presidential preferences. The results, 78/14 for Kerry, are not that surprising, but I might have expected Bush to do just a little better. Some comments:

1) Where is the poll of faculty members? It borders on journalistic malpractice for the Record not to at least try to fairly gauge faculty opinion on the election. I realize that faculty members may not be in a rush to fill out an automated poll (the method that the Record used for undergraduates), but that is no excuse for not trying. Moreover, it wouldn’t be that hard to just ask 25-50 faculty members directly.

2) Where is the analysis of other faculty indicators? I have already looked into faculty donations this electoral cycle, but the Record could do a much more thorough job. It would also be interesting to know the party breakdown of faculty voter registrations.

3) The Record ends by noting that “The survey was sent to 600 students, with 375 responses. The margin of error was 4.6 percent.” First. shouldn’t the Record provide a better description than “margin of error”? I suspect that what they mean is that, speaking frequentistly, there is a 95% probability that the percentage of students planning to vote for Kerry is between 69.2 and 87.6 — the mean value of 78.2 +/- 2 times the standard error of 4.6.

But even this is problematic for two reasons. First, I calculate a standard error of 2.1%. I haven’t done these sorts of statistics in a while, but I am not sure how the Record gets 4.6. Maybe they aren’t really calculating a standard error? Did they use N = 600 (total surveys) instead of N = 375 (total responses)? Using N = 600 is definately wrong.

Second, I don’t think that these are even the right formulas to use. Recall that the Record is only interested in calculating the voting preferences of current Williams students. So, they have actually surveyed around 1/6 of the target population. This is very different from the typical poll that tries to draw inferences about the entire US population. This suggests, perhaps, that the Record’s confidence intervals should be much more narrow than, say, the New York Times would be even if they both have the same sample size.

Consider the extreme case where the sample is 2,000. The New York Times (since it has a sample of 2,000 out of 200+ million) would still have a non-smallish standard error. But the Record would have a standard error of zero since it has surveyed everyone who it cares about, i.e., the entire population of Williams students. It knows the answer perfectly.

In other words, I would expect that the 95% confidence interval for the Record should be much more narrow than the standard formulas would suggest.

I’ll leave this as a simulation exercise for the students in STAT 201.

Facebooktwitter
Comments Disabled (Open | Close)

Comments Disabled To "“Vote Kerry” say Ephs"

#1 Comment By (d)avid On November 1, 2004 @ 10:31 am

Margin of error is a term of art in polling. It is basically the spread of the 95% confidence interval divided by 2.

The formula for calculating the standard error of a polling estimate is sqrt[p(1-p)/N] where p = percentage of respondents for Kerry (or Bush) and N is the number of subjects who completed the survey. Using the numbers you provided above, I get a 2.1% standard error which would translate into a confidence interval of 74%-82% or a margin of error of 4.2.

Why do they get 4.6? Well, pollsters frequently maximize uncertainty by using p=0.5 when calculating the standard error. That would push the margin of error up around 5. Maybe they averaged the two numbers? I don’t think using a t-distribution could account for the difference.

As for sample size, you are forgetting one of the joys of the law of large numbers. Since the total population is assumed to be infinite, it doesn’t matter. Your standard errors will be the same whether you are sampling for a population of 10 thousand or 10 million. The purpose of reporting the number of surveys mailed is to gauge the completion rate. Non-response bias is a big problem for surveys (note: the 63% completion rate for the Record survey is excellent. Phone surveys often have a completion rate of around 20%).

There is a branch of statistics that deals specifically with small samples sizes. However, even 375 out of 2,000 is bigger than the small sample statistics consider.

From what you reported, it basically sounds like the Record analyzed the survey correctly.

#2 Comment By (d)avid On November 1, 2004 @ 10:41 am

By the way:

1) Why does not conducting a faculty poll “border on journalistic malpractice?” There is only one person I hear agitating for such a poll. Believe it or not, most students are more interested in the political views of their classmates than their professors.

2) The survey could be taken to imply that a left-leaning faculty is not a “problem” at Williams. Perhaps the college has been savvy in matching the ideology of the faculty to the students. My current university, Notre Dame, attempts to do that. There are certainly colleges out there (Pepperdine and Antioch, for example) that explicitly cater to an end of the political spectrum.

3) Honestly, how high a response rate do you think you would get from the faculty?

#3 Comment By Mike On November 1, 2004 @ 10:47 am

Kane:

The Record did an article on faculty politics on March 4, 2003:

http://www.williamsrecord.com/wr/?view=article&section=news&id=4308

#4 Comment By (d)avid On November 1, 2004 @ 11:39 am

Mike, one article is not sufficient for a problem of this magnitude. It should be brought up at least once a semester.

#5 Comment By Mike On November 1, 2004 @ 12:36 pm

(d)avid: I suspect you are correct. Of course that would impact the Record’s ability to make sure that every second of Morty’s life is spent “thinking long and hard about how to make Williams better.”

Perhaps the student newspaper could just take some staff off of covering issues that students care about to close this resource gap.

#6 Comment By Lora Kolodny On November 1, 2004 @ 4:43 pm

I’m a staff reporter at a business magazine. As such, I’d love to see our alma mater offer a serious news publication, not just the guide to what’s hot or not on a 2,000-student campus, plus AP headlines rehashed.

However, The Record has no obligation to monitor professors’ partisan affiliations. In fact, I’m not sure even the school does. Doing so threatens to sacrifice an equal opportunity employment and non-harassing work environment.

What the paper has an obligation to do, however, is reveal — not in the fine print either — when a poll they’ve conducted may have been from a statistically unstable, or not random population, and/or what their polling methodology was.

The dangers of reporting with unsteady statistics? Well, we’ve seen it on both the Republican and Democratic side lately…:
http://slate.com/id/2108887/

Anyway. Curious to hear how many of eligible students actually voted at Williams. Do you think a 63% response rate to their survey will be weak or strong versus the number who manage to get in to the polls?

#7 Comment By Ronit On November 1, 2004 @ 4:58 pm

“It borders on journalistic malpractice for the Record not to at least try to fairly gauge faculty opinion on the election.”

I think “WTF?” is the only appropriate response.

#8 Comment By Mike On November 1, 2004 @ 5:20 pm

Lola: The Record does not publish AP articles or articles from any news wires. That decision has been made year after year due because students don’t look to their weekly school newspaper for news on the nation or world. This is especially true nowadays with CNN.com available for constantly breaking news. Given your utter lack of familiarity with the content of the Record, my only response to your characterization of the Record as not a “serious news publication” is to resent the implication entirely.

I would submit that the Record’s excellent coverage of the drinking situation on campus, the existence of an underground fraternity on campus that is a clear violation of community standards, the debates over community life, the role of athletics on campus, and so forth are far more important than comprehensive reporting on David Kane’s outrage du jour.

If you’d like to debate the content of the Record, which I think has been quite good this year and which I will gladly defend for the three years I was personally involved, I would be happy to do so after you do the research necessary to have an intelligent conversation. Until that point, I would recommend you not disparage the 30-50 hours a week that 15 editors each devote to producing a paper that I think compares quite favorably to any paper in the nation.

#9 Comment By Mike On November 1, 2004 @ 5:22 pm

By “any paper in the nation” I mean college paper. Though I think the Record compares favorably to a number of our nation’s professional rags as well.

#10 Comment By CBS On November 1, 2004 @ 6:46 pm

certainly the Record does a better job factchecking than 60 Minutes!

#11 Comment By (d)avid On November 1, 2004 @ 7:16 pm

One of the things I liked about the Record was that it focused upon the campus and the community. In a town where the New York Times was available on a daily basis, why rehash AP wire stories? Focusing on local issues and student life and views played to the Record’s comparative advantage and probably increased its readership.

It doesn’t compare to (good) college papers where the university has a journalism school, but it knows what it is and doesn’t over reach. The fact that it comes out only once a week is also a plus — college papers with five issues a week are usually filled with garbage and have maybe have one paper’s worth of solid material.

Kudos to the Record!

#12 Comment By Todd On November 2, 2004 @ 1:38 am

It sure is wonderful that we have EphBlog and David Kane to point out when the Record has been involved in “journalistic malpractice”, since obviously EphBlog is a bastion of masterful reporting. Oh for the day when the Record speculatively finger-points for the sake of “sunshine” (all while apologizing to the falsely accused, which makes it all better). Then they’ll have some journalistic integrity. I think a publication singlehandedly responsible for reducing the level of online Williams dialogue to something halfway between the National Enquirer and the Drudge Report has no business pontificating about “journalistic malpractice”.

According to Barnett’s Sample Survey: Principles and Methods, OUP, NY: 1991 you can estimate the needed sample size given a certain confidence and accuracy w/the following formula:

n >= N / [1 + ((N-1)/(P(1-P))) (d/za)^2)]

Where N is the population size, P the percent of people who will vote for Kerry (or Bush… doesn’t matter which), d the desired accuracy, and za the double-tailed alpha point of a normal distribution with zero mean and variance of 1 (that’s 1.960 if you want a 95% confidence interval).

You’ll need the highest sample size when P is .5, i.e. it’s equally likely that people will vote for Kerry. It’s obviously not, from the results we got. If you estimate that probability to be 78/92, based on the results we have, the minimum sample size for 95% confidence and 4.6% error is about 210 people. If you assume P=.5, you’ll need about 370 people. Assuming that both the 600 they chose and the 375 who responded out of those were selected and elected to respond randomly (which you really can’t do) you’re oversampling either way. Considering that P is probably a whole lot closer to 78/92 than .5, their error is in all likelihood a lot less than they say, for confidence 95%. If anything they misquoted/mislabeled the error. I’ll agree with Lora that they should publish their methodology, but I don’t think anyone’s blatantly misrepresenting the facts here.

#13 Comment By David Kane On November 2, 2004 @ 5:56 am

Upon reflection, I was wrong to suggest that the Record’s failure to survey faculty voting intentions “borders on journalistic malpractice”. Thanks to several commentators for pointing this out. Instead, I should have said that it is a shame that the Record failed to do so.

I do not think that the lack of ideological diversity among the faculty is the most important issue at Williams. Nor do I think that the Record should write articles about it every semester. But I do think that it is an issue, one that Presidential elections bring into sharp focus. I think that the Record should address it every four years or so.

I (vaguely) remember the Record doing it in 1984. I think that the results were less lopsided than they would be today — perhaps 95/5 Mondale/Reagan. Back then, there were at least a few faculty members who were public Reagan supporters. I seem to remember a debate between Professors Bossert and Burns.

Those with better memories are invited to comment.

By the way, I stand second to none in my praise of the Record. It is a fine publication. If I am occasionally too critical, it is only because I pay the authors in the Record the compliment of taking seriously what they write.

Also, if you’re an alum and you read the Record on-line, you really ought to contribute, as EphBlog has suggested in the past.

#14 Comment By Ronit On November 2, 2004 @ 4:49 pm

David:

I, for one, do not care who my professors are voting for. It would be very amateurish for a professor to bring his personal politics into the classroom, and I have not yet experienced this explicitly. Sure, some profs bring philosophical biases, but I’ve never seen anyone mention petty Democrat/Republican biases.

In most subjects, political preferences make no difference to whether the professor can do a good job teaching the class. It is his business who he’s voting for. I suppose it might matter in certain PoliSci and American History courses, but I would suspect that Williams professors generally leave their preferences for Kerry or Bush outside the classroom. And so this really isn’t a matter of interest to most Williams students.