Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Economist's US college rankings

economist.com

Post reply on HN
Print view View original

101–108 of 108 posts

Re: The Economist's US college rankings

#101

Post by TheCoelacanth »

morley wrote: post #34
slg wrote: post #31

Earlier quoted context omitted.

Isn't that an inherit flaw in the system? It is also based on the assumption that everyone prioritizes money over everything else. If I worked in the Yale marketing department I could easily spin this as a being caused by the high character of Yale graduates who are more willing to dedicate their life to lower paying but higher impact jobs like academics or public service while those greedy kids from Harvard only eve…

Every other publication's college rankings prioritize "everything else" over money. The Economist just came up with a different ranking based on expected value.

It's not a problem that their ranking is based on optimizing for income, since they state upfront what they are ranking them on. However, it could be a problem that not all of the students included in the data are optimizing for that. A school that was very good at helping students increase their potential income, but that attracted mostly students who aren't interested in maximizing income would be ranked low, but it should receive a high rank.

Re: The Economist's US college rankings

#102

Post by jldugger »

impendia wrote: post #88
jldugger wrote: post #75

Earlier quoted context omitted.

I just find your kneejerk response of "Anything that rates my school poorly is wrong" and pocket dismissal is counterproductive to actually rating school performance. Selection bias is an actual demonstrable problem in school rankings, and coming up with some way of separating the effect of a school's selectivity from it's educational results is critical if we wish to improve the productivity of educated professional…

> I just find your kneejerk response of "Anything that rates my school poorly is wrong" and pocket dismissal is counterproductive to actually rating school performance. I see what you are saying. If their methodology ranked my school in the middle of the pack (instead of in the top 2%, which I believe it deserves), saying in effect that the advantages of a Rice education were those that its students brought with them…

> instead of in the top 2%, which I believe it deserves

You would have to know how undergraduate education works in far more institutions than the average academic attends to have experiential data affirming that hypothesis. This is a problem in consulting academics about undergraduate college rankings, they know the researchers far better than the graduating classes, and generally rate one another that way.

> an academic I know a lot about how universities work and how they differ from each other

Which means you helped bring the average down, because grad school is 4 or 5 years of crap earnings.

Re: The Economist's US college rankings

#103

Post by jldugger »

obastani wrote: post #65
krstck wrote: post #58

Earlier quoted context omitted.

> They're basically arguing that you're better off going to a school in the middle of nowhere because "hey, for being in such a crappy location, you did pretty well!". Well, no, not exactly. It's a subtle distinction, but what it's actually ranking is how well that school exceeds expectations , not best outcomes . This is not necessarily a list that will give a student the best school to go to, but rather (what it sa…

That's my point -- who decides what expectations are? Their results are incredibly dependent on the model specification. I imagine if they changed which indicators they used, the results would vary widely. Here's another way to see my concern. Suppose you had a classifier that achieves 1.0 R^2; then since it perfectly predicts each school's expected value, it'll assign each school a score of 0. I'm pretty suspicious…

> Here's another way to see my concern. Suppose you had a classifier that achieves 1.0 R^2; then since it perfectly predicts each school's expected value, it'll assign each school a score of 0. I'm pretty suspicious of an approach where the results get worse with better predictive power.

This depends on your view of the importance of undergraduate education, and what worse is. From my point of view, undergraduate education is an institutional obligation used to fund or justify faculty's personal objectives: research.

The reason that the model counts location is simple: universities tend to place candidates locally. I'm pretty sure the recruiters attending fairs at Stanford and Berkeley have higher starting wages than the ones at University of Kansas, and that a lot of that difference is simply regional cost of living. If you don't factor that in, you risk a bad school in an expensive place ranking higher than a good school in a cheap place.

Re: The Economist's US college rankings

#104

Post by selimthegrim »

carpdiem wrote: post #91
_delirium wrote: post #40

Earlier quoted context omitted.

> So this metric might favor smaller, focused schools which happen to concentrate on education areas with high median salaries... I don't think that part's necessarily true. If a school focuses on an area with high median salaries, the model will take that into account in the predicted salaries, so the school will have to have even higher actual salaries than typical for the field (and its input demographics, SAT sco…

Interestingly, having gone to Caltech, I suspect that its extreme focus on STEM research actually hurt it here. Mostly because that focus results in a very large portion of undergrads going on to grad school (much larger % than any other university), and grad students don't earn very much.

Hi there, Mike! Care to expound on any solutions to make Tech more industry oriented? Are there things you'd rather have been exposed to more in your undergrad education given where you're at now?

Re: The Economist's US college rankings

#105

Post by jldugger »

vmarsy wrote: post #36
impendia wrote: post #9

I am a proud alumnus of Rice University and I see that it is on page 64, fifth from the bottom. Also, even more surprisingly, very near the bottom is Caltech. This is obviously wrong. If their methodology says that Caltech is in the bottom 2% of US colleges, then one concludes that their methodology is worthless, or at least that what it's predicting is not very closely related to the quality of the education provide…

If you sort by Median Earnings, Cal tech shows up in the top. we ran the scorecard’s earnings data through a multiple regression analysis So they ran some machine learning algorithm, the Over/Under column just shows the errors in their final model. If their choice of variable was perfect, then every school would have an over/under of $0, some metrics are just not linear and not easy to get (school reputation, network…

> who cares if they think a cal tech graduate should make $82,126, it still makes $74,000

Mostly someone who wants to measure the effectiveness of their undergraduate curriculum. As a student, there's a strong case to be made for ignoring this data and seeking the most selective alma mater possible. But if the law surrounding Duke v Griggs ever gets overturned and IQ / SAT scores are allowed in screening job applicants, you definitely want to know how much of a school's function is the actual learning and placing bit.

Re: The Economist's US college rankings

#106

Post by amscanne »

obastani wrote: post #86
haberman wrote: post #71

Earlier quoted context omitted.

> Suppose you had a classifier that achieves 1.0 R^2; then since it perfectly predicts each school's expected value, it'll assign each school a score of 0. I'm pretty suspicious of an approach where the results get worse with better predictive power. If I'm understanding correctly, that result would indicate a world where the college you attend has no effect on your earning power. ie. choose any college you want, bec…

I'm not saying that each college you attend has no effect on earning power. It's just that I can perfectly predict the effect of each college on your earning power. Does that make sense? If I have an oracle that tells you "if you got to Harvard, you will make $80,000, if you go to MIT, you will make $86,000", and the oracle is exactly correct, then under this model, The Economist assigns every college a score of 0.

I think you are missing the key ingredient in the analysis.

The Economist is attempting to build such an oracle via statistical regression. HOWEVER, the Oracle is intentionally limited in input to a specific list of things: SAT scores, sex ratio, race breakdown, size, public or private, earning power in the city where it is located, etc.

The things that are omitted constitute the actual value the University brings to the table: quality of teachers, instruction, organizations on campus, etc. (1)

So however far off the model is for two given Universities must be explained by all the missing inputs, i.e. largely how "good" the University is.

If the Oracle was able to perfectly predict your earning power given that limited set of inputs, then it would basically mean that a University is completely defined by SAT scores of students, sex ratio, race, etc. and there's absolutely no value they add or subtract beyond that. That was be a very, very interesting result. But you can see why it's unlikely.

Hopefully this makes sense?

(1) Of course it's possible that there are factors like "how many trees on campus" or "how many vowels are in the name" which might also affect earnings. But we can probably agree that it's less likely to be important than the aforementioned ones ("quality of instruction", etc.).

Re: The Economist's US college rankings

#107

Post by alistairSH »

tpudlik wrote: post #82
alistairSH wrote: post #78

Something feels "off" about a methodology that basically ranks all of the "top" state schools as poor values. Take the top-3 state schools in Virginia (by most other rankings, that's UVA, W&M, and VT). All three are nationally recognized. And all three are competitive that you need very good grades and a solid set of extra-curricular activities to be accepted. UVA ranks the lowest of the three, yet has the highest ac…

I don't think the Economist's ranking can be used for choosing which school to attend. It tells you how good a job schools do at boosting the earnings of the people who attend there---conditioned on who attends. In other words, they tell you how much the school does for a typical member of its student body. But if you're not representative of that student body (and you certainly won't be representative at many of the…

I guess I just find it extremely hard to believe almost all of the nations "top" state schools have a negative impact (actual earnings less than expected) on the students that attend.

Maybe I need to dig into the source of the expected earning figures. I'm definitely biased to some degree, being a graduate of one of the state schools with a poor ranking - I'm sitting here wondering where else I could have gone to get a better value.

Re: The Economist's US college rankings

#108

Post by the_watcher »

et2o wrote: post #90
the_watcher wrote: post #29

Earlier quoted context omitted.

That doesn't make this ranking list bad. The rankings system explicitly explains, in painstaking detail, that the system is designed to measure return on degree, empirically. That means they must use empirical data.

A characteristic is not valuable simply because it can be measured with empirical data. You also presume it is not possible to measure academic success empirically.

> return on degree, empirically

Not academic success. Measurable return (which I may be incorrectly presuming that most would read that as a financial metric).

Post reply on HN