MIT to no longer consider SAT subject tests in admissions decisions • Hacker News Board

Re: MIT to no longer consider SAT subject tests in admissions decisions

#311

Post by zaptheimpaler » Fri, Mar 20, 2020, 10:25 PM UTC

A comment section of people who didn't read the article lol. They explicitly said they consider SAT to be a predictive metric of success at MIT. This is not critical of SAT, only SAT subject tests.

While they weren't explicit about why SAT subject tests won't be accepted, this may be a clue:

> No: in fairness to all applicants, we won’t consider them for anyone. We think it would be unfair to consider scores only from those who have scored well and therefore choose to send them to us.

Seems reasonable to expect tests which are optional to suffer from heavy selection bias as they described. I'd expect optional tests to also skew towards the rich because there is a cost to each test.

Re: MIT to no longer consider SAT subject tests in admissions decisions

#312

Post by salty_biscuits » Fri, Mar 20, 2020, 10:27 PM UTC

MichaelDickens wrote: post #155

jzoch wrote: post #132

Earlier quoted context omitted.

Eh its all easily game-able. The SAT (and ACT) is a joke. So are the subject tests. The only subject test that wasn't laughable are the language ones (since you can't "intuit" spanish as easily as math or physics or chemistry). Its not bad if you are a good test taker (a dumb skill but a useful one) or wealthy. My friends were tutored idiots and did well (2200+ SAT, 700+ subject tests) and i was a good test taker (sa…

If the SAT were a joke, you wouldn't expect scores to follow a bell curve with only ~0.3% of people getting a perfect score, but they do.

If you had a multiple choice quiz with the actual questions hidden you would get a normal distribution in results. Anything where you sum lots of small random effects.

Re: MIT to no longer consider SAT subject tests in admissions decisions

#313

Post by DiogenesKynikos » Fri, Mar 20, 2020, 10:35 PM UTC

jimmyvalmer wrote: post #86

DiogenesKynikos wrote: post #28

Earlier quoted context omitted.

There's been a similar debate over subject GREs in my field. The argument has been made that subject GREs are racist/sexist, because of the differing distributions of test scores in different demographic groups. There is a strong social pressure to accept this argument, despite its obvious absurdity, leading most people in the field to do so (and if they disagreed, they probably wouldn't tell you). The problem is tha…

MIT's grad schools jettisoned GRE consideration a while ago. As you say, the GRE quant is trivial by MIT's standards, and the GRE verbal largely irrelevant. It's fairly easy to spot the "heavy-hitters" just from recommendations, undergraduate publication record, thesis, and GPA. In short, MIT and its elite cohort (CMU, Caltech, etc.) only accept undergrads who found their way into their professor's research groups --…

GPA is difficult, because there's no uniform standard. A 3.9 GPA from Caltech is much more impressive than a 3.9 GPA from Harvard (where there's serious grade inflation), but then again, you have to look carefully at which specific courses went into that GPA. But how do you compare those numbers to a grade of 15/20 from a French university, or a 1.1 from a German university, or some other country whose system you don't know?

Undergrad research (including a published first-author paper) is almost a requirement now for entry into many top programs. But you have to know the research group the student worked with, and you get applications from all over the world.

The subject GRE is a uniform quantitative baseline. It isn't everything (or even most of everything), but it is a somewhat objective standard that can at least flag applicants who might have real difficulty in a top program.

The thing is, I've heard "equity" arguments for disregarding not only the subject GRE, but also published research ("Not everyone has the opportunity to do research as an undergrad") and letters of reference ("Letter writers suffer from implicit bias"). The "equity" advocates seem to want to throw out every piece of relevant information. Of course, that makes it easier to hit whatever targets "equity" requires.

Re: MIT to no longer consider SAT subject tests in admissions decisions

#314

Post by im3w1l » Fri, Mar 20, 2020, 10:40 PM UTC

salty_biscuits wrote: post #312

MichaelDickens wrote: post #155

Earlier quoted context omitted.

If the SAT were a joke, you wouldn't expect scores to follow a bell curve with only ~0.3% of people getting a perfect score, but they do.

If you had a multiple choice quiz with the actual questions hidden you would get a normal distribution in results. Anything where you sum lots of small random effects.

If the test was not measuring anything at all, then you would expect repeat takers to have completely unrelated scores on first and last test.

Re: MIT to no longer consider SAT subject tests in admissions decisions

#315

Post by quartzite » Fri, Mar 20, 2020, 10:40 PM UTC

jschwartzi wrote: post #252

quartzite wrote: post #210

Earlier quoted context omitted.

That's irrelevant, what matters is that a high IQ society has a vested interest in measuring IQ as a proxy for general intelligence and standardized testing results should reliably correlate to IQ and general intelligence.

But is IQ really a proxy for general intelligence?

Yes, and it's the best proxy we have.

Re: MIT to no longer consider SAT subject tests in admissions decisions

#316

Post by disgruntledphd2 » Fri, Mar 20, 2020, 10:43 PM UTC

ebrenes wrote: post #235

disgruntledphd2 wrote: post #211

Earlier quoted context omitted.

This isn't a good argument, because the SAT (and other tests of the same kind) are normalised to have this property. The SAT may not be good tests, but that fact has nothing to do with the presence or absence of a normal distribution.

The SATs have a top score, if it were easy to game you'd find many people would clump at the top. That would make it impossible to normalize that distribution into a bell-shape because they're all clustered into the same bucket with no way of spreading them. OP's point was that the perfect scores remain consistently outside of people's grasp despite the variety of resources available to prep for the exam. I only once…

That is correct.

So the SATs are (almost certainly) 3-pl (actually 2) IRT models. Essentially, it's a multivariate generalised linear mixed model to estimate both question difficulty and participant ability.

Normally, they'll estimate the abilities on the logistic scale, and use the percentile to back transform to a standard normal.

Most people don't cluster at the top because they are a good proxy for g, which is an imaginary statistical construct that we use to explain differences in school outcomes.

So I had a long digression here about the usefulness of penalties for guessing, but it turns out the SATs don't do that anymore, so wth?

(ETS invented IRT, that's why I'm pretty sure).

Re: MIT to no longer consider SAT subject tests in admissions decisions

#317

Post by freepor » Fri, Mar 20, 2020, 10:43 PM UTC

JMTQp8lwXL wrote: post #306

quartzite wrote: post #205

Earlier quoted context omitted.

This is exactly correct. Those who fight for fair admissions on the basis of intellectual ability and academic potential should hold up the SAT and standardized testing as a cornerstone of the admissions process. Standardized testing significantly levels the playing field for students across income brackets. Returns on study investment quickly diminish, and reaching a plateau on returns doesn't require much investmen…

> Standardized testing significantly levels the playing field for students across income brackets. There's a correlation between parental income and SAT scores. “[S]tudents from families earning more than $200,000 a year average a combined score of 1,714, while students from families earning under $20,000 a year average a combined score of 1,326.” https://www.cnbc.com/2019/10/03/rich-students-get-better-sat... An "eq…

But there’s a much greater correlation between parental income and other measures you could use to evaluate applicants. Even height has parental income correlation, you can‘t eliminate it completely.

Re: MIT to no longer consider SAT subject tests in admissions decisions

#318

Post by graeme » Fri, Mar 20, 2020, 10:44 PM UTC

_hardwaregeek wrote: post #292

quartzite wrote: post #205

Earlier quoted context omitted.

This is exactly correct. Those who fight for fair admissions on the basis of intellectual ability and academic potential should hold up the SAT and standardized testing as a cornerstone of the admissions process. Standardized testing significantly levels the playing field for students across income brackets. Returns on study investment quickly diminish, and reaching a plateau on returns doesn't require much investmen…

This is not entirely true. SAT scores can be used as ways to admit more privileged students, as they tend to have access to more test prep along with privileges such as extra time: > In 2010 three College Board researchers analyzed data from more than 150,000 students who took the SAT, and they found that the demographics of the two “discrepant” groups differed substantially. The students with the inflated SAT scores…

Does that study control for school quality? If grades are normalized for a local school population, then good grades at one school can be worse than mediocre grades at another school.

This was exactly the purpose for which tests like the SAT were created. If that factor wasn’t controlled for, then the quote above is misleading.

Re: MIT to no longer consider SAT subject tests in admissions decisions

#319

Post by Hydraulix989 » Fri, Mar 20, 2020, 10:45 PM UTC

jschwartzi wrote: post #252

quartzite wrote: post #210

Earlier quoted context omitted.

That's irrelevant, what matters is that a high IQ society has a vested interest in measuring IQ as a proxy for general intelligence and standardized testing results should reliably correlate to IQ and general intelligence.

But is IQ really a proxy for general intelligence?

Yes. https://en.wikipedia.org/wiki/G_factor_(psychometrics)

An individual's performance on one type of cognitive task tends to be comparable to that person's performance on other kinds of cognitive tasks.

Re: MIT to no longer consider SAT subject tests in admissions decisions

#320

Post by chongli » Fri, Mar 20, 2020, 10:45 PM UTC

ebrenes wrote: post #235

disgruntledphd2 wrote: post #211

Earlier quoted context omitted.

This isn't a good argument, because the SAT (and other tests of the same kind) are normalised to have this property. The SAT may not be good tests, but that fact has nothing to do with the presence or absence of a normal distribution.

The SATs have a top score, if it were easy to game you'd find many people would clump at the top. That would make it impossible to normalize that distribution into a bell-shape because they're all clustered into the same bucket with no way of spreading them. OP's point was that the perfect scores remain consistently outside of people's grasp despite the variety of resources available to prep for the exam. I only once…

A lot of the reason the SATs are able to maintain a spread that lets them normalize the distribution is that they fill the tests with stupid tricks that fool people into wasting time. If you do a ton of prep then you learn to spot these tricks and then a lot of the questions become really easy and you can knock them off quickly.

A poor smart student may have mastery of the material but their score will suffer if they don't know the tricks.