The Ottawa Citizen has won a great victory! It now has the right to access undergraduate grades from Carleton University for the past twelve years, and never mind student privacy.

(But let’s put that last bit in parentheses for now. Small classes will certainly allow the identification of students with specific “anonymized” grades, but whether or not there is even a right to privacy seems a little up in the air. Later for that.)

Why is the Citizen so eager to get its hands on the data? Like others who left comments under the Glen McGregor story, I (a Carleton alumnus), wondered aloud. Then McGregor stated—in the comments, not in the story itself—that he was interested in looking at grade inflation.

Hoo, boy.

This seemed, in fact, a bit of an afterthought on McGregor’s part. Getting access to the data for its own sake appeared more salient: as McGregor himself blurted in a late-night Twitter-spat, he had a right to the data, even if he just wanted to wallpaper his bathroom with them. But let’s take him at his word, and even try to help him out a little.

Via another Tweeter who got into the act, we can see from this scholarly article by two University of Windsor economists, Paul Anglin and Raymond Meng, that the question is by no means an easy one to resolve. Two things become clear at once in this study of selected Ontario universities: first, the alleged inflation is inconsistent from discipline to discipline, and sometimes nonexistent; and 2) the suggested reasons for it are, for the most part, arrant speculation.

In the paper just cited, the discipline of English experienced the largest inflation from 1973-4 to 1993-4, followed by Biology and Chemistry. Of 80 course-university combinations with data for the beginning and end periods of the study, grade inflation was observed in 42, there was no significant inflation in 25, and grade deflation was observed in 13.

In some cases, inflation was at the low end of the grading spectrum—fewer Fs, more Ds. In other cases it was at the higher end.

So what the authors make of this hodge-podge?

A vast number of hypotheses have been put forward to explain grade inflation. To generalize, grades could increase because: (a) professors are more generous for a given group of students, (b) the quality (e.g., intelligence, study effort) of a group of students has increased over time, (c) for a given group of students, teaching methods improved, and (d) random effects that apply to a particular observation (i.e., specific course-university effects). Most people believe that the first hypothesis is the most prevalent and represents the greatest concern to policymakers while acknowledging that the second and third make careful analysis difficult. We acknowledge the final hypothesis by noting which changes are statistically significant. The essential problem in studying or verifying any of these hypotheses is that they rely on data that are not readily available. Hopefully, our results will encourage researchers who have access to the required data to conduct appropriate studies.

With these caveats securely in place, the authors go on to indulge in speculation. Professors, they suggest, feel compelled to award higher grades to keep enrollment numbers up. To buttress that hypothesis, they point to one study that indicated that students who get a low mark in an introductory course may tend to choose other majors. As one of today’s students might say, “Well, duh.”

The authors conclude with a dark warning:

[I]t is possible that grades will continue to rise and employers will have an increasingly difficult time distinguishing a well-trained student from someone who just “gets by.”…If a credential no longer provides added value to the economy then a university degree represents an enormous expense for a government without producing a public benefit.

Well, we aren’t there yet. But let me offer a little speculation of my own.

Those who graduate from university these days are usually saddled with huge debts, as tuition fees have skyrocketed over the years. That’s a considerable investment, without the former virtual guarantee of a decent job at the end of it, and marginal students who might once have attended university might prefer other less-risky options today. The expanding community colleges program in Ontario offers attractive alternatives as well, and a more immediate return on investment, to those who want to learn specific skill-sets but are less academically inclined. Those remaining in the university system may, therefore, generate higher average grades without any inflation at all.

Student demographics have changed as well. Recall the infamous “Too Asian” article in Maclean’s magazine in 2010? Despite the racist tenor of the piece, it is undeniable that the serious attitudes towards study among certain groups cannot help but boost general averages. Again, those numbers would reflect real excellence, not increasingly generous professors.

Grades these days have also changed in another respect. Once they were very largely based upon the final examination, with a midterm and (in arts disciplines) some essay marks added. Now, a student’s work is graded continually—it can include such things as class participation, regular quizzes, and weekly assignments. Teaching assistants are often responsible for much of this grading, as well as helping with the final grading for the course. It would require a somewhat conspiratorial frame of mind to imagine that so many people could be involved in the artificial raising of grades in courses to help boost allegedly falling enrollments.

(I should disclose interest here: I was a TA in 1970, and again in 2008-10. I was never once asked to “mark hard” or “mark soft”: the only concern was that TAs in a specific course marked consistently among ourselves.)

“Inflation” means a decrease in worth. But what we might well be seeing, for the reasons just given, is a real increase in excellence, rather than “inflated” grades. It is noteworthy, in this respect, that this isn’t a phenomenon observed only in the so-called “soft” disciplines; English has shown higher grades over time, but so have Biology and Chemistry.

By themselves, increases in the numbers have polemical value only. The “standards are slipping” folks will run with it, as well as the anti-intellectual set. But the trick is to account for what increases there may be, based upon something other than guesses and a priori assumptions. One does that by obtaining enough data to found a specific hypothesis, while eliminating alternative hypotheses. A mere observation that there are rising numbers tells us next to nothing.

So the Citizen’s Glen McGregor, best known for his excellent work on robocalls, apparently wants to conduct one of those “appropriate studies” that Anglin and Meng were suggesting. Frankly, he has his work cut out for him. It will be interesting to see how he deals with the host of confounding variables that inevitably affect any supposed “grade inflation.”

It’s a Herculean task. And, while I dislike intensely the way he got his mitts on the data, it would be churlish of me not to wish him well as he undertakes it.

UPDATE: My Internet pal Craig McFarlane weighs in.