Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Economist's US college rankings

economist.com

Post reply on HN
Print view View original

31–40 of 108 posts

Re: The Economist's US college rankings

#31

Post by slg »

armaansarkar wrote: post #11
impendia wrote: post #9

I am a proud alumnus of Rice University and I see that it is on page 64, fifth from the bottom. Also, even more surprisingly, very near the bottom is Caltech. This is obviously wrong. If their methodology says that Caltech is in the bottom 2% of US colleges, then one concludes that their methodology is worthless, or at least that what it's predicting is not very closely related to the quality of the education provide…

Didn't they kinda address this? "The bar is set extremely high for universities like Caltech, which are selective, close to prosperous cities and teach mainly lucrative subjects. If their students didn’t go on to extremely high-paying careers, the college would probably be doing something gravely wrong."

Isn't that an inherit flaw in the system? It is also based on the assumption that everyone prioritizes money over everything else. If I worked in the Yale marketing department I could easily spin this as a being caused by the high character of Yale graduates who are more willing to dedicate their life to lower paying but higher impact jobs like academics or public service while those greedy kids from Harvard only ever chase money.

Re: The Economist's US college rankings

#32

Post by 6stringmerc »

impendia wrote: post #9

I am a proud alumnus of Rice University and I see that it is on page 64, fifth from the bottom. Also, even more surprisingly, very near the bottom is Caltech. This is obviously wrong. If their methodology says that Caltech is in the bottom 2% of US colleges, then one concludes that their methodology is worthless, or at least that what it's predicting is not very closely related to the quality of the education provide…

As a long-time Texas resident and qualified applicant for Rice University who likely could have attended (if I hadn't refused taking SAT IIs), I'd also like to note your esteemed alma mater is likely the most picky University in all of Texas, and, if US News' numbers are correct, just a shade less picky than Johns Hopkins. In short: Selection bias is not to be ignored.

Re: The Economist's US college rankings

#33

Post by wtvanhest »

fiatmoney wrote: post #13

There seems to be a major issue with cost-of-living adjustments as well. Several rather obscure California colleges are highly ranked, and I assume this is because both wages and costs are higher in California, where many of their graduates stay after college.

Absolutely. Cost of living in an area will have a massive impact on exit salaries. The economist did not realize that the South and Mid-west have the lowest cost of living when they wrote the following:

> Its upper tiers are dominated by colleges that emphasise engineering (such as Worcester Polytechnic) and attract students with high SAT scores (like Stanford). The lower extreme is populated by religious and art-focused colleges, particularly those in the south and Midwest. This number represents the benchmark against which we subsequently compare each college’s alumni earnings to produce the rankings.

Re: The Economist's US college rankings

#34

Post by morley »

slg wrote: post #31
armaansarkar wrote: post #11

Earlier quoted context omitted.

Didn't they kinda address this? "The bar is set extremely high for universities like Caltech, which are selective, close to prosperous cities and teach mainly lucrative subjects. If their students didn’t go on to extremely high-paying careers, the college would probably be doing something gravely wrong."

Isn't that an inherit flaw in the system? It is also based on the assumption that everyone prioritizes money over everything else. If I worked in the Yale marketing department I could easily spin this as a being caused by the high character of Yale graduates who are more willing to dedicate their life to lower paying but higher impact jobs like academics or public service while those greedy kids from Harvard only eve…

Every other publication's college rankings prioritize "everything else" over money. The Economist just came up with a different ranking based on expected value.

Re: The Economist's US college rankings

#35

Post by beachstartup »

leroy_masochist wrote: post #17

So basically, the methodology is that they do a very complex analysis of the student body to predict what their average earnings would be if they went to "college in general" and then compare that with what the graduates actually make. There's one thing that jumps out at me when I see the actual rankings: At least three of the eight schools that received a perfect 100 score -- W&L, Babson, and Bentley -- have a very,…

> My hunch is that the Economist's "expected earnings" methodology wasn't granular enough to take these idiosyncratic attributes into account

or, given the economist's core demographic (rich borderless global elites) this is exactly what they optimized for.

Re: The Economist's US college rankings

#36

Post by vmarsy »

impendia wrote: post #9

I am a proud alumnus of Rice University and I see that it is on page 64, fifth from the bottom. Also, even more surprisingly, very near the bottom is Caltech. This is obviously wrong. If their methodology says that Caltech is in the bottom 2% of US colleges, then one concludes that their methodology is worthless, or at least that what it's predicting is not very closely related to the quality of the education provide…

If you sort by Median Earnings, Cal tech shows up in the top.

we ran the scorecard’s earnings data through a multiple regression analysis

So they ran some machine learning algorithm, the Over/Under column just shows the errors in their final model. If their choice of variable was perfect, then every school would have an over/under of $0, some metrics are just not linear and not easy to get (school reputation, networking power?)

In the end this is not a very valuable metric, who cares if they think a cal tech graduate should make $82,126, it still makes $74,000. What is more interesting to me would be how much money that graduate spent on school to get there.

Re: The Economist's US college rankings

#37

Post by jldugger »

impendia wrote: post #9

I am a proud alumnus of Rice University and I see that it is on page 64, fifth from the bottom. Also, even more surprisingly, very near the bottom is Caltech. This is obviously wrong. If their methodology says that Caltech is in the bottom 2% of US colleges, then one concludes that their methodology is worthless, or at least that what it's predicting is not very closely related to the quality of the education provide…

It's important to distinguish that this system is not rating graduates, but institution performance. Sort the table by expected earnings, and Rice is on the front page. Students admitted to Rice are smart and successful people the moment they step into classes on day one.

The question they're trying to answer is: once a smart and successful person chooses a college, how much value does that college add? Looking at that same expected earnings sorted table, the actual earnings is quite varied. MIT graduates earn 8k more than expected, and Rice earned 10k less. That's quite a difference!

There's many reasons this dataset is flawed, but not for any of the reasons you mentioned:

1. Financial aid applicants are not a representative sample of students. 2. Highly selective schools have a very high bar to exceed. 3. Students may prefer other opportunities that pay less, like civil service, or politics. 4. Earnings are only tracked for ten years (standard loan repayment).

I also question how earnings are calculated. For example, because I max out both a 403b and a 457b retirement account, my earnings are taxed as if I earned half of what I actually do. Does my double savings opportunity affect the calculation?

Re: The Economist's US college rankings

#38

Post by BookmarkSaver »

stagger87 wrote: post #27
et2o wrote: post #26

This is hilariously bad. Schools that produce academics as opposed to those who go into high-paying professions are penalized, when in fact (at many of the top institutions) producing academics is one of the main goals.

This is not bad. It is simply one of many possible ways to rank colleges.

But it is a serious flaw in the methodology. It says nothing about the quality of education or the boost in earnings that a college will provide, if you are specifically going to a school to do so. At the very least they need to somehow account for individuals that go to the top schools but aren't looking to go into lucrative careers.

CalTech is a common example being cited here. It is basically impossible to deny realistically that going to CalTech is a great way to make a lot of money. But it is in the bottom percentages of these rankings, likely because so many CalTech alumns go into academia.

The metric is supposedly showing which colleges can potentially increase your earnings the most, but it just isn't doing that. It's a great idea, it just hasn't been implemented fully.

Re: The Economist's US college rankings

#39

Post by jpatel3 »

_gwlb wrote: post #4

This tool would be a lot more useful if it allowed for filtering by chosen course of study. For example, University of Washington has a median earnings that is slightly below expected, but I can pretty much guarantee that their computer science graduates are earning well above median.

Here is another way to see the expectation of the students by state - https://tuvalabs.com/benf/datasets/s/fdcbab1d15e64518a3cc3a5...

Re: The Economist's US college rankings

#40

Post by _delirium »

digikata wrote: post #19
_gwlb wrote: post #4

This tool would be a lot more useful if it allowed for filtering by chosen course of study. For example, University of Washington has a median earnings that is slightly below expected, but I can pretty much guarantee that their computer science graduates are earning well above median.

That was my thought too, to make a simplistic example if a university had both say an engineering school and an art school, it might presumably do worse than a university with only an engineering school. So this metric might favor smaller, focused schools which happen to concentrate on education areas with high median salaries...

> So this metric might favor smaller, focused schools which happen to concentrate on education areas with high median salaries...

I don't think that part's necessarily true. If a school focuses on an area with high median salaries, the model will take that into account in the predicted salaries, so the school will have to have even higher actual salaries than typical for the field (and its input demographics, SAT scores, etc.) to get a positive value-add. See Caltech for an example of a STEM-focused school that does badly by this measure: from its SAT scores, demographics, and heavy concentration of STEM majors, the regression analysis predicts that it should produce graduates with a median salary of $82k. But the actual median is $74k, so its value-add is taken to be -$8k.

Some of the schools that do well are in areas with poor salaries, but score highly because they do better than you'd expect (or than the model would expect, anyway) for that area and student demographics. Otis College of Art and Design has a predicted salary of $29k from the regression analysis, but actual median is $42k, so implied value-add +$13k.

Post reply on HN