One of the more (only?) impressive factoids that the CUL has on its side in the debate over anchor housing is the claim that only 13% of Williams students are very satisfied with social life at the College. Assuming that Joe Shoer’s ’06 notes from the first CUL forum are correct, we have these paraphrases for Professor Will Dudley, head of CUL.

A declining percent of graduating seniors say Williams has an excellent social life.

The “data is spotty.” We only have two data points: last year, 13% of graduating seniors said that social life at Williams was very satisfying. In 1999, that figure was 25%. This is from a voluntary survey at colleges, we only have data that gets released. Morty has this data in “some kind of lockbox in his house.”

13% of graduating seniors think social life here is very satisfying.

[Q] We need some kind of “exit survey”…how was the 13% reached?

I don’t know.

EphBlog is here to help! Provost Cappy Hill, with help I believe from David Brodigan, was kind enough to answer my questions about this survey (which was distributed to all seniors) and give me permission to publish them here. Read on for the details, but the bottom line is that the more that I learn about CUL’s evidence, the less persuausive its argument becomes.

According to Professor Hill:

The question is

How satisfied are you with each of the following services or aspects of your college?

quality of campus life:
Social life on campus

  • Very satisfied
  • generally satisfied
  • Generally dissatisfied
  • very dissatisfied
  • Not relevant

We report and tend to use “very satisfied” when using these data. That is the 13% reported by Will.

Why does this information not necessarily support the CUL’s claim that there is a “problem” to which anchor houses are the solution. Let us count the ways.

  1. The number has gone from 25% to 13% in the last 5 years. Hmmm. What has happened in the last 5 years that might account for that change? The CUL has implemented a bunch of other wonderful improvements! To the extent that you place much faith in these sorts of surveys, the evidence suggests that all the changes made by CUL in the last few years (CLCs/HCs/smaller room draw groups/ending WSO plans) have been detrimental to student life at Williams.
  2. The number is based on a survey to which only 1/2 the students respond. Who believes that these students are an unbiased sample of all seniors at Williams? Not me. The people who tend to take the trouble to return surveys are different than the people who don’t. Now, it isn’t clear which way this bias cuts, but it is obvious that year-to-year changes in any number need to be taken with a huge grain of salt since the changes might (just as easily?) be due to changes in the type of student who responds rather than changes in average student opinion. In other words, perhaps the same number of seniors are “very satisfied” with social life now as as in 1999, but different subsets are deciding to respond. Now, this may or may not be a big issue in this case. Unless the College decides to release the data, there is no way to know for sure. (Looking at these sorts of questions would make for some great senior theses. I often think that the math/stats department should encourage more work along these lines to go along with the fine efforts that students in economics have done, often under the direction of David Zimmerman.)
  3. We have no information on how the percentages in other buckets have changed over time. If the percentage reporting being “very dissatisfied” has dropped from 30% to 5% in the same time frame, then I would assert that the College is improving. That is, there is often a trade off in which making some people less miserable goes along with making others less happy. It is not clear what the correct balance between competing goals should be.
  4. We have no information on how different the number would be at other colleges. The CUL has hinted that the number is higher elsewhere, but have made nothing on this public. Now, if it is true that 40% of students at, say, Bowdoin are very satisfied with social life, then that might be cause for concern. It would be interesting to try to understand the causes of any differences. But, unless and until CUL makes this comparison possible, there is no way for an outsider to know if there is any meaning here. After all, just because CUL meets with the Student Life Staff at Bowdoin and they report that senior at Bowdoin think that everything is wonderful does not mean that everything is wonderful at Bowdoin.

The central issue, again, is that it is unscholarly for CUL to just throw out a number like 13% without giving the rest of the community a chance to judge whether the number has any meaning. CUL should not be trying to win a debate. It should be doing everything possible to provide the community with access to all the information that it has access to. For example, perhaps there are many aspects of student life at that are rated much higher at Williams than at other colleges. Perhaps those aspects will be hurt by the end of free agency. Who knows?

It is simply unfair for the College to only trot out snippets of data that support its case without making the data available to the entire community. Note that the blame lies with the College as an institution. It is not clear that Dudley has the authority to make the data public. It is not clear that Hill sees the need to do so. It is not for me to say which individuals are at fault, but the process is deeply flawed.

Thanks again to Cappy Hill for taking the time to answer my questions on this. She (and David Brodigan) have much more important work to do then to help out a cranky old alum. I suspect that they would be more than willing to make much of this data public soon if the request to do so came from someone like Dudley (as opposed to some trouble-maker like me). Perhaps someone could ask Dudley why he hasn’t done so at the forum tonight.

Print  •  Email