- EphBlog - http://ephblog.com -

Measuring Success

Related to the below post is the question of how to measure Williams’ success as an institution. The most important measures are probably (hypothetical) comparisons between the effect that Williams has on its graduates, relative to the effect that other schools would have had. Like any non-experimental situation, this can never be estimated perfectly, but, conceptually at least, it is easy to think about. A high school senior, call her Michaela Kane, could (I wish) go to either Williams or school X. The test of Williams success is not that, after 4 years at Williams, Michaela is a better writer, clearer thinker, happier person, more comfortable with diversity or whatever than she was 4 years ago. The test is whether, by these or whatever metric you like, she is better off having gone to Williams than if she had gone to school X. The test is not Michaela before versus Michaela after. It is Williams Michaela versus non-Williams Michaela.

Again, since Michaela can only go to one school, we can never know for sure what the outcome would have been had she made the other choice. So, our estimate of Williams’ success will always be an estimate. But this is the standard approach to all sorts of statistical problems in which randomized selection is not possible.

Yet, this is not the measure of success that I want to mention in light of the WSJ article. There is a cruder measurer of success that one could use. It is relative yield. Now, Williams already measures “yield”, the proportion of students who choose to enroll out of the pool that has been accepted. A mroe direct measure of success, relative to a specific school, is the relative yield. That is, out of the pool of students who are accepted by both Williams and school X and who choose one or the other, what percentage choose Williams? I would wager that the relative yield of Williams as compared to Amherst and Swathmore is around 50% but that it is less than 20% for places higher up in the academic food chain like Harvard and Yale.

Surely there is a reader out there who has a sense of what the real data is like.

In any event, one way to measure the success of, for example, President Schapiro, is to look at the increase in Williams relative yield during his term in office. Note that this is far from a perfect measure. After all, Williams might switch resources from teaching to advertising, therefore making Michaela more likely to select it but less likely to learn a great deal. But it would be interesting, to say the least, to see these numbers and watch how they change over time.