Currently browsing posts filed under "Athletics Committee 2009 Report"

Follow this category via RSS

Statistical Details of Athletic Report

Professor Heather Williams, chair of the Athletic Committee for 2008-2009, kindly replied to my request to know more about the details of their statistical analysis (pdf).

As chair of the committee, I am responsible for the statistical analyses. The design of the analyses was the responsibility of the committee, and several of us are statistically competent. The analyses themselves were run by the Provost’s office, and although you are certainly free to ask Chris Winters about them, I would imagine that he’ll be referring all questions to me. The report will not describe all of the results in detail, but will provide enough information about how the analyses were set up and the results to let the readers understand exactly what we did.

A quick summary, we ran a number of simple linear models (and logistic regressions, where appropriate to the question) that included a variety of variables that seemed likely to have affects on academic performance: reader rating, gender, class (fr., so., etc.), a proxy for socioeconomic status, and sport category (high/low profile), as well as a number of interaction terms.

I’m sure that, as an economist who is into numerical analyses, you would prefer to have more information, but I wouldn’t be comfortable about giving any additional details beyond this summary of how the analyses were done. The more details we give, the more people want to know, and it’s important not to make it possible for anyone (including us) to gain too much information about individual athletes.

Perfectly reasonable! I still think that the Committee ought to share the regression results (which are available in an Appendix on file in the office of the Dean of the Faculty) with the entire community, but this is a minor quibble. The Report is high quality and represents the best of the Williams tradition of faculty governance and transparency. Kudos to all involved!

For those interested, below is the e-mail that I sent to Professor Williams.
Read more


CGCL VI: Report by the Athletics Committee

This year’s athletic report is much, much shorter than the one I first discussed in 2004 (known as the MacDonald report); this one is just a follow-up. It is only four pages long, so any of you who are really interested should consider reading the whole thing. I have excerpted over 50% of the entire document below.

First, good news:

We find that the gap in academic performance, as judged by grade point average, has narrowed substantially overall and has essentially disappeared for female athletes and for male athletes in low-profile sports. The gap for male athletes in high-profile varsity sports (which we defined as football, ice hockey, basketball, and baseball […]) appears to be narrowing, but persists even after we adjust for 1) academic qualifications prior to enrolling at Williams College, 2) socio-economic status, and 3) the individual’s year (e.g. sophomore, senior). Thus academic under-performance by male varsity athletes in high-profile sports continues, and cannot be attributed to academic credentials prior to Williams or to socioeconomic status.

Five years ago, the females’ difference was very small — less than .05, if I recall — so I am not surprised that the effect has disappeared completely. Also, note that five years ago, men’s football and hockey were the only sports singled out in various statistics (which led to their being called, colloquially, “helmet-sport athletes”) such as the following:

Football players are 47% more likely than students who are not football players to take easy courses, and men’s ice hockey players are 93% more likely than other students to take easy courses.

Now, the list of “high-profile,” i.e. “low-achieving” teams has grown to include men’s basketball and baseball as well.

Note that in this paragraph, the committee says that “academic under-performance by male varsity athletes in high-profile sports continues, and cannot be attributed to academic credentials prior to Williams.” Okay, next paragraph:

The narrowing of the overall academic performance gap since 2002 could be due to any of a number of factors (perhaps including changes in team culture during the past decade) but one likely factor is the change in admissions standards for athletic “tips”. The minimum qualifications required for admission to Williams have been raised during the intervening years, and are continuing to rise.

Thus varsity athletes’ academic preparation for Williams College is increasingly similar to that of the rest of the student body. Our data indicate that academic under-performance by male varsity athletes playing high-profile sports can largely be attributed to those who are less well-prepared academically for Williams, and thus it is our sense that the “raising of the floor” for admissions tips may have been an important factor in reducing overall difference in the GPAs of varsity athletes and non-athletes.

Wait, first the difference is not attributable to differences in preparation before Williams, and they say that “academic under-performance by male varsity athletes playing high-profile sports can largely be attributed to those who are less well-prepared academically for Williams.” Huh? I sense a contradiction here. True, the first paragraph discusses a static difference (between GPAs of athletes vs. non-athletes just this year) and the second discusses a change (the GPAs have gotten closer together over time), but the underlying forces are the same. These two paragraphs certainly appear to contradict each other. Thoughts?

The report goes on to discuss that high-profile athletes tend to congregate in Division II majors, and that all athletes take tutorials and write honors theses at a lower rate than non-athletes. This is important stuff, but it appears to be unchanged since the MacDonald report.

Now the committee does an interesting comparison:

Some possibilities [for athletes’ underachievement] that have been raised in the literature and in our discussions are 1) “stigma” – that some faculty members have a negative view of athletes and that this could affect grades and admission to courses; 2) that athletes experience stereotype threat; 3) that recruited varsity athletes are primarily committed to their sport and coach and do not place as high a priority on academics as do other students; 4) that some teams’ culture promotes a disengagement from academics; and 5) that investing time and energy in athletics reduces time spent on other activities.

I agree with this analysis. I cannot think of any significant reason that is not on this list. I think — and this opinion is heavily influenced by taking a course with Sheafe Satterthwaite, who spoke of football being a “fifth class” for football players — that #5 is the most probable reason. The writers of the report apparently agree, because they did a great comparison:

We did investigate the “time and energy” hypothesis indirectly by asking whether JAs (a group of students with a substantial time commitment to an extracurricular activity that spans the entire year) demonstrate any under-performance in terms of GPA and low rates of enrollment in tutorials. After correcting for academic preparation, sex, and socioeconomic status, the grades of JAs were statistically indistinguishable from those of other juniors, although they enrolled in tutorials at a slightly lower rate than did their peers. Which of the other explanations might help to explain academic under-performance by male varsity athletes in high-profile sports at Williams College is not clear. It is our view that this question merits further study, and we recommend that the college take advantage of a future COFHE questionnaire to ask students about the rationale for their choices and priorities with respect to participating in tutorials and writing theses.

Alas, the result of the analysis was inconclusive. Such is science. “Statistically indistinguishable” makes one wonder which GPA was actually higher, but perhaps it is better that we don’t know, since the difference is not statistically significant — p > 0.05, move along, nothing to see here. Sigh.

The footnote in this section would be of interest to hwc:

A recent study using Swarthmore varsity athletes as subjects demonstrated an effect of stereotype threat; the MacDonald report raised concerns about “athletic culture” discouraging academic engagement; and Bowen and Levin, in “Reclaiming the Game”, favored the explanation that recruited athletes’ first priority is athletics. However, we have no evidence to support or refute any of these hypotheses.

I would think that there would be more of a stigma against athletes at Swarthmore, where the stereotype is of nerdy bookworms, than at Williams, where the stereotype is either “Mens sana in corpore sano” (for the pro-athlete) or “Of course, this isn’t a college. It’s a Nike Camp with enrichment classes” (for the anti-athlete).

Now for the committee’s nine recommendations:
Read more


2009 Report by the Athletics Committee

Kudos to Professor Heather Williams, chair of the Athletics Committee for her decision to make public their latest report. (Permanent archive here as well.) The conclusions were described in an earlier Record article.

The Athletics Committee conducted a report on academic performance of varsity athletes versus non-athletes to be completed and submitted to the faculty in May. The first study conducted since Michael MacDonald, former chair of the committee, released a report in 2001, it found that the overall gap in academic performance has been halved, and that gap has been eliminated for females when averaged across all sports. Despite this progress, a considerable discrepancy does remain for high-profile male athletes in sports such as football, basketball, baseball and ice hockey.

Of course, what EphBlog readers really want is my line-by-line analysis. Andiamo!

In the Spring of 2002, an ad-hoc committee chaired by Michael MacDonald reported on its review of the role that varsity athletics played at Williams College.

We have been referring to that document as the Report on Varsity Athletics, but Williams et al use the term “MacDonald report.” I will do the same. It is beyond pathetic that Director of Public Affairs Jim Kolesar refused my request to make that report public. It is only thanks to the kindness of Professor MacDonald that any of us were allowed to read it.

The history of the MacDonald report is fairly obvious, if always unstated. Morty arrived at Williams and was upset by the magnitude of the admissions preferences given to athletes and the (resulting?) disengagement of many athletes from the intellectual life of the College. And, being a smart president, he knew that the way to change things was to appoint a committee of like-minded souls who would recommend the changes that he wanted.

The 2009 Report continues:

Shortly thereafter, Bowen and Levin published “Reclaiming the Game”, a book that reviewed varsity athletics in three conferences (Ivy League, NESCAC, and UAA) whose institutions place a high value on academics. Both the book and the MacDonald report concluded that varsity athletics (within NESCAC and at Williams) posed problems for some aspects of the academic mission of colleges such as Williams.

Indeed they did. If you haven’t read the MacDonald report yet, you ought to. It is superb and compelling. But note that the reference to “Reclaiming the Game” is somewhat curious. The book that had a major influence on the MacDonald report was “The Game of Life” by Shulman and Bowen. Both books are co-authored by Williams Bowen and make almost identical points: elite colleges put more emphasis on athletics than they used to; this is a bad thing.

The 2009 Report:

Specifically, they found that a) varsity athletes, and particularly male high-profile varsity athletes, under-performed academically;

This is highly misleading! Did the Committee members not read my criticism of the MacDonald report? I demonstrated, fairly conclusively, that there was no evidence for under-performance by Williams athletes, once we account for their high school academic credentials, despite the fact that the Committee went out of its way to look for such underperformance and even spun its results to make athletes look bad. The MacDonald report itself states that:

The grades of our athletes are lower than the grades of our non-athletes, which should not surprise us. They are often weaker students when they enter Williams, and they commit much time and effort to their sports. It is predictable that they would be weaker students in Williams, suggesting a second question: do athletes underperform academically, controlling for their academic ratings at the time they were admitted? The data is mixed, but suggests on balance that our athletes achieve about the same grades as non-athletes with similar academic ratings.

Day is night. Ignorance is strength. And varsity athletes underperform academically.

Isn’t this pathetic? Let me highlight the conflict again.

MacDonald report: “[O]ur athletes achieve about the same grades as non-athletes with similar academic ratings.”

2009 Report: “[T]hey [the MacDonald Report] found that a) varsity athletes, and particularly male high-profile varsity athletes, under-performed academically.”

Malice or incompetence? You make the call!

Now, to be fair to Professor Williams and the rest of the committee, the “they” in their sentence refers to both the MacDonald report and to Reclaiming the Game, and it is true that Bowen and Levin conclude that “recruited athletes “underperform:” they do even less well academically than predicted by their test scores and high school grades.”

But this has always been a highly suspect claim (to me). First, it was clear from the start that Bowen, at least, wanted to find exactly this result. He thinks (has always thought?) that elite colleges should be more like Cal Tech. Athletic excellence should not matter to admissions. Bowen, who does not have a reputation as the world’s greatest statistician and almost certainly did not do any of the empirical work himself found exactly what he wanted to find. It could be true, of course, but I would like to see some independent verification.

Second, the claim of underperformance of athletes was similar to the underperformance claims made about African American students in Bowen’s “The Shape of the River.” But how could both sorts of underperformance be true at the same time? The vast majority of students in the bottom 20% of the SAT distribution are either athletes or African Americans. (The Shape of the River ignores Hispanics.) It could be true that one group does less well than the other, but how could both groups underperform given that they make up the vast majority of the population at those SAT levels? I think that there was some very sloppy regressions run to generate these results.

The 2009 Report continues:

b) varsity athletes tended to cluster in certain majors and c) some faculty members felt that lack of intellectual engagement of varsity athletes posed problems for their classes. The purpose of this report is to re-examine some of these findings, seven years later.

Fair enough. The MacDonald report did, indeed, note the clustering (mostly within Division II) and report accurately about the concerns of these faculty. But there is no evidence that this clustering was unusual given their academic credentials. Are their a lot of football playing math majors? Probably not. But this has nothing to do with football per se. The main cause is that there are not a lot of math majors with academic ratings below 3.

Side note: This Report mentions that “The statistical analyses used in preparing this report are summarized in an appendix that is not part of the report, but will be placed on file in the Dean of Faculty’s office.” Why not make it public too? Would an alum (like me) be prevented from viewing the report if he showed up in Hopkins Hall, say during reunions? Academic institutions should be as transparent as possible. There is no plausible reason for keeping the appendix a secret.

In any event, there is much of value in the Report. Alas, it will take me a few days to wade through it all. Stand by for more commentary.


Currently browsing posts filed under "Athletics Committee 2009 Report"

Follow this category via RSS