Last month and the month before we reported on the discoveries of math professor Michael Thaddeus that Columbia University was feeding implausible numbers into its U.S. News ranking.
Thaddeus has dug up more! These are all for the engineering program. Click on each image below to see its source:
I agree with Thaddeus that these numbers seem implausible.
Just to be clear: I’m not claiming that anyone at Columbia is trying to lie or mislead. My guess is that it’s someone’s job to fill out these forms, and paperwork is no fun. Average class size, graduation rates, etc etc., these don’t have clear definitions, it takes lots of work to track down all the numbers—just for example, how do you count students who drop out or switch to auditor status halfway through the semester?—so at some point it’s easier to just put stick some numbers into the damn form and move on. I’d like to say that the university should just stop filling out the form, but apparently these ratings are important to many potential students—even to current students and recent graduates who benefit from Columbia’s continuing high reputation. So, yeah, I get it. If it were my job to figure out these numbers, I wouldn’t want to either. But, now that the cat’s out of the bag (see above image from Zad), I think the university is kind of obliged to do it right or not do it at all.
As I wrote before, I think Columbia is a wonderful institution. I love it here, and my colleagues throughout the university do cutting-edge research and quality teaching, with student involvement at all levels. I’m proud of so many things that are done here. My high regard for Columbia is why it makes me particularly sad to see this sort of thing happen. I get it, everybody makes mistakes, there’s lots of pressure to keep that ranking high, people around the world see these rankings, etc. I respect that. I hope the university spokespeople could start by thanking Thaddeus and admitting they did some things wrong, and then they can go fix it. Or if Thaddeus messed up somehow, he should apologize—but that would require that someone explain why the above numbers aren’t ridiculous.
When I clicked on the source for the retention rates there was a bunch of data including class size – it showed average class size of 8 and maximum size of 20. When I clicked on the first image (which shows the class size) above, I get the average size of 20 and maximum of 20 as shown. The source for both is archived US News pages. I agree that the data is implausible, but I also suspect there are issues with inconsistent data sources (timing, updating, etc.).
That’s correct. The U.S. News ranking web pages present a confusing jumble of data. Each of my links points to the relevant page but not necessarily to the relevant part of the page (which I know of no easy way to do in html). You may have to scroll down or open a tab entitled “Academics” or “Applying.” Furthermore, since these are archived pages, not all graphics elements may load properly in all browsers. You may have to click on the relevant words or even burrow into the source file of the page. Nevertheless, every image I provided to this blog was taken from the web page to which it is linked.
There certainly are issues with inconsistent data sources, in that some of the data displayed here directly contradicts other data (with retention rates in 2010-11, 2011-12, and 2012-13, for example). My attempts to seek clarification from the Columbia administration on such matters have not met with success.
” graduation rates, etc etc., these don’t have clear definitions, it takes lots of work to track down all the numbers”
Come on, Andrew, this is ridiculous. These are all easy to measure and are *standard* things for universities to keep track of, and your questions about ambiguous edge cases all have standard answers. Graduation rate is even a standard part of the higher education Common Data Set (https://commondataset.org/), but of course, Columbia doesn’t make theirs available. (Here’s my university’s: https://ir.uoregon.edu/cds ) It is definitely the case that Columbia knows these numbers perfectly well, and anyone involved in the process knows perfectly well where to find them.
+1
Just because there is a standardized way of defining graduation rate doesn’t mean Columbia is using it or that all their databases use that definition. If someone pulls data from the wrong source, then they will get the wrong answer. I think Andrew is just pointing out that that could be the cause of the implausible numbers. I think he is giving a face saving way for Columbia to say, “yes, we screwed up.” Every company of sufficiently large size that I have ever worked with could not answer the question of how many employees they had. It depended why you were asking the question.
+1
+1
Class size is a difficult one to parse. I remember back some decades when I was an undergrad chem major. Organic chemistry (the great leveler) started with just over 100 students in the lecture. By the third quarter it was down to between thirty and forty. What number do you use for class size???
Three terms of organic chemistry is not one course, it is three, each of which will have an enrollment number. I’m sure the chemistry department tracks the attrition between terms — ours does, as it’s a major concern. It is true that even within a term the number is different at the start than the end, and as with any measurement there is therefore some uncertainty, but it is also true that e.g. the common data set doesn’t even ask for an exact number but a bin into which the class size falls (…, 40-49, 50-99, 100+). (Yes, they’re weird bins.)
I agree with the commenters who say that these terms are slippery for the layman to define, but I also agree with Raghu who points out that the professionals have arrived at standard conventions for defining these terms, which they generally agree upon. The Common Data Set is a joint initiative of the College Board, Peterson’s, and U.S. News, and it gives a detailed definition of undergraduate class size, which the U.S. News survey also follows. The U.S. News survey stipulates that overall enrollments for the Fall semester are to be measured on October 15. It doesn’t stipulate this for class sizes, but this would, accordingly, be the most reasonable choice.