Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

U.S. News pulls Columbia University from its 2022 rankings

forbes.com

Post reply on HN
Print view View original

161–170 of 348 posts

Re: U.S. News pulls Columbia University from its 2022 rankings

#161

Post by geebee »

One interesting thing to come of these discussions - a lot of the problems with US News methodology arise from the attempt to push certain metrics from 90% to 100%. Class size, faculty with a "terminal" degree, class size, admissions rate. Many of these things are treated by the rankings as "the higher (or lower) the better." As an analogy, I'd use lean muscle mass percentage for elite athletes. Generally speaking, lower is better, and in many sports, all elite athletes are all under 10% body fat. But much below this starts to become harmful, and would eventually be fatal.

I think this is kind of what's happening with Columbia here. For example, Columbia claimed that 100% of the faculty had a terminal degree. Prof Thaddeus (if you read his full blog post) questioned this from two angles. The first was to say it can't possibly be true. The second (which I found more interesting) was: why would you even want this to be 100%? Seems like Columbia has valuable faculty who don't have a PhD or MFA or whatever is the "terminal" degree int he field.

The harm here isn't in wanting a terminal degree for a high percentage of faculty, it's trying to get this to 100%. I'd say that it probably is a good sign, overall, that most of the faculty has a doctoral degree, as this is the main training degree for research and teaching. But the way US news does it, you get more points for 95% than 90%, more points for 99% than 95%. Seems like a lot of harm comes from trying to wring out that last bit, since even if you do generally agree with the value of a PhD for research (or other terminal degree), it does seem like you'd build a stronger faculty overall with the ability to hire 1 in 10 or 1 in 20 based on some other credential.

So much else fits this pattern. Small class sizes.. kind of the same thing. Sure, it's a good sign, but they're only an indicator, not a goal in and of themselves. Low admissions rates, high test scores. All good metrics, but things that can become not just harmful but harmful and maybe even fatal to a university if pursued with a single minded intensity as if total purity in the metric is the goal.

Honestly, overall we just have to reject the US news rankings. I appreciate what Prof Thaddeus did here, and it was a useful and very well reasoned and sourced takedown. But now and then, I realized that there isn't much to be added, just new and interesting ways to say what everyone knows at this point - yousnoozeandworlddistort rankings are pretty well idiotic. I actually think college rankings in general can be useful if managed cautiously and read critically, but this one is just a turkey.

Re: U.S. News pulls Columbia University from its 2022 rankings

#162

Post by gumby »

colinmhayes wrote: post #117
gumby wrote: post #86

I really wonder why Columbia even cares: it's a globally well known ivy league school obviously producing a good education and is in NYC.* Doesn't US News need Columbia more than the other way around? * No I have no connection to the school except I do have some friends who teach there and some others who attended decades ago.

Having high ranking is like the entire job of administrators. Especially at Ivys they’re selling prestige, being highly ranked gives you prestige.

If you're in the small top 'n' I think the prestige goes the other way. Will anyone care if MIT is +1 or -1 some year? I assume those top schools can just run themselves the way they want and still get highly ranked.

And if US news were serious they could simply have kicked columbia out for a year, but it would have hurt them to do so. They'd probably do it for a small college ranked 100 or below though.

I'd be embarrassed for Columbia that they gamed the numbers. Outside that small n though, I presume gaming the numbers is commonplace.

Re: U.S. News pulls Columbia University from its 2022 rankings

#164

Post by humanistbot »

burlesona wrote: post #3

Per the article this came in response to questions raised by a Columbia faculty member critiquing the university's data. I thought that report[1] was actually more interesting than the original article. As per the quote at the top of the report: > Rankings create powerful incentives to manipulate data and distort institutional behavior for the sole or primary purpose of inflating one’s score... It would be shocking t…

See https://en.wikipedia.org/wiki/Goodhart%27s_law

Re: U.S. News pulls Columbia University from its 2022 rankings

#165

Post by pishpash »

hackyhacky wrote: post #7
burlesona wrote: post #3

Per the article this came in response to questions raised by a Columbia faculty member critiquing the university's data. I thought that report[1] was actually more interesting than the original article. As per the quote at the top of the report: > Rankings create powerful incentives to manipulate data and distort institutional behavior for the sole or primary purpose of inflating one’s score... It would be shocking t…

Your suspicion is correct. I've taught at several US universities, and it's an open secret that the administrative takes steps to manipulate numbers in order to secure better rankings. Typical example: a small, prestigious liberal arts college always has smaller class sizes in the fall semester than in the spring semester. Why? Because classes with less than 30 students are ranked higher by US News, but they only tak…

What's that adage? Every metric that becomes an optimization target ...

Re: U.S. News pulls Columbia University from its 2022 rankings

#166

Post by snowwrestler »

verst wrote: post #92
poulsbohemian wrote: post #56

Earlier quoted context omitted.

>Anecdotally, you get better class material and TA staff in larger lectures. Where is this anecdote coming from? A billion years ago when I studied at a small liberal arts college with small class sizes, professors were keen to share with us exactly why they were teaching from particular texts or why they had selected certain materials to make it clear that while the class wasn't being taught by that famous guy from…

Exactly this. I went to a liberal arts college and had small class sizes (most around 9-15 in size, with the largest being 30) - I could see my professors 1:1 pretty much anytime I wanted, but there was also a preceptor (TA) for each class who didn't teach, but was available to answer questions related to assignments, projects etc. I went to the house of many of my professors for food or drinks too. Generally also wa…

Many professors at small liberal arts colleges still conduct research. And because of small class sizes, there is a good chance for a motivated student to get to know their professor and participate in that research.

There are also research consortiums of small colleges. I did original geological research 2 out of my 3 summers in a small college via the Keck Consortium:

https://keckgeology.org/

Re: U.S. News pulls Columbia University from its 2022 rankings

#167

Post by rel2thr »

bee_rider wrote: post #11

I wonder how much these measurements actually matter, for a place like Columbia. I mean, I can see why, like, people might want to look at the list and see "hey this school I've never heard of is actually OK." But like, people don't find out about Columbia from this list, right? It is, uh, older than the country "U.S. News" is named after and an Ivy League school. Not having them on the list just reflects poorly on t…

Not really , Knowing about Columbia is kind of a new york, upper class thing

They don't have good sports teams, no way a person in middle america would have heard of them if not for school rankings

Re: U.S. News pulls Columbia University from its 2022 rankings

#168

Post by humanistbot »

ethbr0 wrote: post #68
mtlmtlmtlmtl wrote: post #32

Earlier quoted context omitted.

This is a good example of Campbell's Law at work: https://en.wikipedia.org/wiki/Campbell%27s_law

An obvious takeaway in this case: US News shouldn't be the primary ranker. If statistics were distributed in open format, transparently, and there were multiple rankers (each with their own weights and mixes), universities would be less incentivized to game "the US News metrics".

It already exists and is a free US Dept of Education service. Schools have to report that data to the Dept of Ed already, so they have both an interactive search tool [1] and the raw data available [2].

[1] https://collegescorecard.ed.gov/search/?page=0&sort=threshol...

[2] https://collegescorecard.ed.gov/data

Re: U.S. News pulls Columbia University from its 2022 rankings

#169

Post by Leary »

lumost wrote: post #45
hackyhacky wrote: post #7

Earlier quoted context omitted.

Your suspicion is correct. I've taught at several US universities, and it's an open secret that the administrative takes steps to manipulate numbers in order to secure better rankings. Typical example: a small, prestigious liberal arts college always has smaller class sizes in the fall semester than in the spring semester. Why? Because classes with less than 30 students are ranked higher by US News, but they only tak…

Why is small class size a good thing? Anecdotally, you get better class material and TA staff in larger lectures. Why aren't Universities able to innovate here?

One thing is closer interaction with faculty. Crucial if you want to apply for grad school and need letters of recommendation.

Re: U.S. News pulls Columbia University from its 2022 rankings

#170

Post by echelon »

kmlevitt wrote: post #156
lumost wrote: post #45

Earlier quoted context omitted.

Why is small class size a good thing? Anecdotally, you get better class material and TA staff in larger lectures. Why aren't Universities able to innovate here?

Small class sizes are better precisely because the class can then become more than a lecture. I am a professor, and if you give me a hall full of students I’ll give a lecture, because that’s all that can really happen under those circumstances. If it’s a small group, I can actually learn everybody’s names, get to know them, personalize the material more and spend big chunks of the class just answering your questions…

In my three student organic chemistry III class, the professor had each of us reading ahead and taking turns teaching the material ourselves. We'd be corrected for mistakes, of course, but it was very much a fire drill. We moved a lot of electrons.

That was quite the course.

Post reply on HN