Live data from Hacker News

The Economist's US college rankings

economist.com

81–90 of 108 posts

Re: The Economist's US college rankings

#81
post #66

My university asked for my earnings only one time. The categories maxed out at 70k+. They might be higher on that list if they took a more precise survey.

The data used here does not come from the universities, but from the government. Specifically, from matching federal student loan applications (from the Department of Education) to tax returns (from the IRS).

Re: The Economist's US college rankings

#82

Something feels "off" about a methodology that basically ranks all of the "top" state schools as poor values. Take the top-3 state schools in Virginia (by most other rankings, that's UVA, W&M, and VT). All three are nationally recognized. And all three are competitive that you need very good grades and a solid set of extra-curricular activities to be accepted. UVA ranks the lowest of the three, yet has the highest ac…

I don't think the Economist's ranking can be used for choosing which school to attend. It tells you how good a job schools do at boosting the earnings of the people who attend there---conditioned on who attends. In other words, they tell you how much the school does for a typical member of its student body. But if you're not representative of that student body (and you certainly won't be representative at many of the schools, especially the outliers!), this will not tell you how much the school would do for you.

Re: The Economist's US college rankings

#83

> "The government generated the numbers by matching individuals’ student-loan applications to their subsequent tax returns, making it possible to compare pupils’ qualifications and demographic characteristics when they entered college with their salaries ten years later." So the dataset excludes students who do not apply for loans? (i.e., this analysis penalizes schools who admit the folks most likely to make lots of…

> So the dataset excludes students who do not apply for loans? (i.e., this analysis penalizes schools who admit the folks most likely to make lots of money, and the schools that have the lowest expected student contribution.) It penalizes (or at a minimum, undersamples in a nonrepresentative way) schools with their own, non-loan, aid programs. This probably very badly hurts schools like Caltech, since it means that t…

Axlines were suspended some time ago, I think they are still proactive with need based aid but I think with their tuition skyrocketing they can't possibly be as gracious as they once were.

Re: The Economist's US college rankings

#84

Earlier quoted context omitted.

That was my thought too, to make a simplistic example if a university had both say an engineering school and an art school, it might presumably do worse than a university with only an engineering school. So this metric might favor smaller, focused schools which happen to concentrate on education areas with high median salaries...

> So this metric might favor smaller, focused schools which happen to concentrate on education areas with high median salaries... I don't think that part's necessarily true. If a school focuses on an area with high median salaries, the model will take that into account in the predicted salaries, so the school will have to have even higher actual salaries than typical for the field (and its input demographics, SAT sco…

Not all the majors at Caltech are going to be value adds, and of those that are, they are generally leading to bench/engineering rather than business dev/marketing/executive positions

Re: The Economist's US college rankings

#85

Earlier quoted context omitted.

This is not bad. It is simply one of many possible ways to rank colleges.

But it is a serious flaw in the methodology. It says nothing about the quality of education or the boost in earnings that a college will provide, if you are specifically going to a school to do so . At the very least they need to somehow account for individuals that go to the top schools but aren't looking to go into lucrative careers. CalTech is a common example being cited here. It is basically impossible to deny r…

I think the money thing is paralleling something else entirely - given the supposed caliber of student that comes into Caltech, is Caltech really making them better scientists than if they had gone elsewhere (or at least better scientists than they were turning out in the 1970s). That to me is a open question.

Re: The Economist's US college rankings

#86

Earlier quoted context omitted.

That's my point -- who decides what expectations are? Their results are incredibly dependent on the model specification. I imagine if they changed which indicators they used, the results would vary widely. Here's another way to see my concern. Suppose you had a classifier that achieves 1.0 R^2; then since it perfectly predicts each school's expected value, it'll assign each school a score of 0. I'm pretty suspicious…

> Suppose you had a classifier that achieves 1.0 R^2; then since it perfectly predicts each school's expected value, it'll assign each school a score of 0. I'm pretty suspicious of an approach where the results get worse with better predictive power. If I'm understanding correctly, that result would indicate a world where the college you attend has no effect on your earning power. ie. choose any college you want, bec…

I'm not saying that each college you attend has no effect on earning power. It's just that I can perfectly predict the effect of each college on your earning power. Does that make sense? If I have an oracle that tells you

"if you got to Harvard, you will make $80,000, if you go to MIT, you will make $86,000",

and the oracle is exactly correct, then under this model, The Economist assigns every college a score of 0.

Re: The Economist's US college rankings

#87
post #5

The Brookings study [1] mentioned by the the Economist offers this interesting analysis: 1. Graduates of some colleges enjoy much more economic success than their characteristics at time of admission would suggest. Colleges with high value-added in terms of alumni earnings are often focused on training for high-paying careers in technical subjects. ... 2. Four college quality factors are strongly associated with high…

> (Personally, I think the most valuable things gained in college or in any education don't happen to have much monetary value.) Well, someone does, because they're charging for it.

> Well, someone does, because they're charging for it.

Good point; I should have said, I don't think they are things that earn you money.

Re: The Economist's US college rankings

#88

Earlier quoted context omitted.

I definitely agree with you. I wasn't trying to explain how I think the article is flawed; rather, I was just trying to argue that its conclusions can be dismissed immediately. In particular, I believe that your (3) describes what I see as the biggest flaw with their methodology; that (at least for elite colleges) what they are largely measuring the proportion of students that choose to go into high-paying profession…

I just find your kneejerk response of "Anything that rates my school poorly is wrong" and pocket dismissal is counterproductive to actually rating school performance. Selection bias is an actual demonstrable problem in school rankings, and coming up with some way of separating the effect of a school's selectivity from it's educational results is critical if we wish to improve the productivity of educated professional…

> I just find your kneejerk response of "Anything that rates my school poorly is wrong" and pocket dismissal is counterproductive to actually rating school performance.

I see what you are saying. If their methodology ranked my school in the middle of the pack (instead of in the top 2%, which I believe it deserves), saying in effect that the advantages of a Rice education were those that its students brought with them, then I would be annoyed but I can see how that could be fair.

But the bottom 0.5%? I went there and I know better. (And as an academic I know a lot about how universities work and how they differ from each other.) I do feel that I am justified in calling bullshit immediately.

Re: The Economist's US college rankings

#89
post #26

This is hilariously bad. Schools that produce academics as opposed to those who go into high-paying professions are penalized, when in fact (at many of the top institutions) producing academics is one of the main goals.

This is not bad. It is simply one of many possible ways to rank colleges.

If I ranked colleges based upon something silly such as the length of the college's name, that would also be "one of many ways to rank colleges" but it would also be "bad" in any traditional sense of the word as applied to our problem of ranking colleges based upon educational quality.

Re: The Economist's US college rankings

#90
post #26

This is hilariously bad. Schools that produce academics as opposed to those who go into high-paying professions are penalized, when in fact (at many of the top institutions) producing academics is one of the main goals.

That doesn't make this ranking list bad. The rankings system explicitly explains, in painstaking detail, that the system is designed to measure return on degree, empirically. That means they must use empirical data.

A characteristic is not valuable simply because it can be measured with empirical data.

You also presume it is not possible to measure academic success empirically.

Post reply on HN