Provost Will Dudley’s discussion (pdf) of financial aid at Williams is equal parts useful and misleading. Let’s spend five days discussing it. Today is Day 3.

We run a program in the fall called WOW—Windows On Williams. We’ve nearly doubled the size of this program in the last couple of years because it’s so effective. We fly in, at our expense, about 200 low-income and first-generation students to spend a couple days on campus, meet each other, meet other Williams students and attend classes. The program is competitive; we get about 1,200 applicants. The students we select are very strong candidates for admission, and getting them here on campus dramatically increases the chances that they apply and will choose to enroll here if we admit them.

We have a similar previews program in the spring for admitted students who haven’t already participated in WOW and can’t afford to come here on their own. We want to make sure they get a chance to experience this place in person before they decide where to go to college. Our admission office travels to high schools where low-income and first-generation students are likely to be found. … That’s what need-seeking is: doing everything we can in a very active way to admit as many talented, low-income students as we can.

Interesting stuff. Comments:

1) There is a great senior thesis to be written about how “effective” (or not) WOW is. Recall Peter Nurnberg’s ’09 excellent thesis about predicting which accepted students will choose Williams. This is an important topic, of interest to Williams and to elite colleges more broadly.

2) Has anyone at Williams — including the dozens of capable folks who report to Dudley — actually studied this? I have my doubts. But tell us about it if you have! Note that obvious selection bias inherit in Will’s claim that “getting them here on campus dramatically increases the chances” such students apply and enroll. The problem is that comparing students who do WOW with students who don’t do WOW is, potentially, useless because, almost by definition, students who do WOW are much more interested in Williams than students who don’t do WOW. They would be much more likely to apply/enroll than other students even if WOW did not exist. In fact, for all Will knows, WOW might actually decrease the percentages of such students who apply/enroll. (My bet, of course, is that Will is right and that WOW works.)

The right way to test WOW would be a standard A/B approach. Randomly select 10% (or whatever) of the students who met the criteria for WOW and then don’t invite them. If those students apply/enroll at the same rates as the WOW students, then WOW doesn’t do anything.

The College, like most modern bureaucracies, ought to do much more randomized testing to find out what works and what does not.

3) Even without a proper randomized controlled trial (RCT), you can still try to estimate the causal effect of WOW using various statistical approaches. A statistics major ought to jump on this opportunity.

Print  •  Email