Let’s continue our discussion of “Athletics and Alumni Giving Evidence From a Highly Selective Liberal Arts College” pdf by Jessica Holmes, James Meditz and Paul Sommers. Today, my focus is on the claim that hockey success at Middlebury leads to increased alumni donation rate.

Hockey success, when measured by a national or league championship title, is associated with a 7% higher likelihood of giving.

This is either very sloppy or very wrong or both. First, let us start, as suggested by Vicarious ’83, with a simple chart of the data.

The problem is obvious: There has been a significant secular increase in participation rates over these 15 years. Alumni were much more likely to donate in 2004 (47%) then they were in 1990 (34%). Although the rise has not been perfectly monotonic, it has been steady and significant.

Unfortunately, the authors fail to take that increase into account in their statistical analysis. That means that anything — average SAT scores, number of faculty, NCAA hockey wins, Republicans in Congress, e-mail messages sent, gas prices — which is higher post 1995 will be correlated with increased alumni participation even if there is no causal connection.

Second, consider the phrase “national or league championship title.” Here (pdf) is the hockey team’s record. They won a championship title every year from 1995 through 2004, except for 2003. Any comparison of alumni campaign results which distinguishes between hockey championship years and non-championship years is almost identical to a comparison of pre-1995 and post-1995 giving.

In order to do statistics, you need variation. You must have some years when X is true and some when it is false. If X is always true (or if X is perfectly correlated with some other factor that you know is important), then you can’t (easily) tell what effect X has.

Third, even if you view the lack of a championship in 2003 as somehow causally connected to the fall off in donation rate for that year (which I find absurd), you still have to deal with the team’s success. They made the NCAA Semifinals! Is there really a Middlebury alumnus who would have given that year if the team had won two more games but, because they were only one of the 4 best teams in the country, declined to send in a donation? Implausible!

Fourth, note the timing problems. The 2003 semifinal loss occurred on March 21, 2003. Middlebury, like Williams, runs on a fiscal year that ends on June 30. So, by the end of March, the vast majority of fund-raising had already been completed. (I called the Middlebury Alumni Fund and they provided a very rough estimate of more than 75% of the donations received by the end of February.) So, unless alumni had a time machine that told them, when they were donating in December, what the hockey team was going to do 4 months later, the effect is even smaller. Essentially, you have to argue that a large part of the small proportion of alumni that donate after March won’t donate if the hockey team only makes the semi-finals but would have donated if they had won it all. Does anyone believe that?

Print  •  Email