Why did the A-level algorithm say no? • Hacker News Board

Re: Why did the A-level algorithm say no?

#81

Post by macspoofing » Mon, Aug 17, 2020, 12:31 AM UTC

stordoff wrote: post #73

macspoofing wrote: post #34

Earlier quoted context omitted.

>shouldn't students have the right to get the exam grades they actually deserve? Look, nothing I've seen here changes the base fact that if you were a good students getting good grades, you are going to be OK. It also doesn't change the base fact that mark inflation exits. That some schools will grade higher than others for the same academic performance. So no, you don't have a 'right' to a mark you haven't earned. I…

> Let's say you had weak course marks and you were hoping to use the test to boost your final score to get into the school of your choice. Then covid happened and you weren't able to. Well... take that as important life lesson. Don't slack off until the end and hope that you can make up your grades with the exams. I'm not sure why what you mean by "weak course marks" - something like A-Level Maths is assessed entirel…

>Alternatively, if you _did_ work hard, but it's assumed that your predicted grades have been inflated, you face getting knocked down through no fault of your own.

Mark inflation is a real thing by the way. You can ignore it and assume all teacher-assigned markers across the country are roughly the same, but then you will punish those pupils who come from school where the standards are higher and higher grades are harder to get. Is that fair?

>I also query the notion that you can reliably know which schools get predicted grades wrong.

That's actually not very hard to do. There was a minor controversy a few back in Canada where a number universities were found out to have correlated the marks students got in university with the marks they got from the specific high school they came from. They had that kind of historical data on every local high-school. They then normalized those marks. People got offended but without provincial standardized testing and grading, you're just going to punish schools with highest academic standards.

>Teachers, and teaching methods, change every year - the people making the predictions are not necessarily the same year to year.

I agree it changes. I disagree it changes every year, especially in aggregate. A low performing academic school this year is going to be a low performing academic school next year. Maybe in 5 years it will be different, but year over year, the changes are going to be inconsequential.

Re: Why did the A-level algorithm say no?

#82

Post by Diggsey » Mon, Aug 17, 2020, 12:31 AM UTC

There's no good answer here, the unfairness is an inevitable side effect of lockdown, along with people dying due to stopped cancer treatments and a host of other issues.

Using teacher's predicted grades is stupid: teachers use predictions in whatever way they think will best motivate their students. If the student is lazy, they'll under-predict to shock them into revising more. If the student is not very confident, they'll over-predict to encourage them.

Unless the students are actually allowed to take an exam, all methods are unfair. Using mock results is at least tangentially related to the students' abilities.

Re: Why did the A-level algorithm say no?

#83

Post by jdietrich » Mon, Aug 17, 2020, 12:52 AM UTC

corin_ wrote: post #31

mjburgess wrote: post #28

Earlier quoted context omitted.

> it's hard not to be sceptical that this wasn't intentional and class-motivated. It's incredibly easy "not to be sceptical". The teacher's predicted grades are, on average, unprecedented . And represent many times greater than any previous yoy increase. This is far less true at top-performing schools where teachers are predicting far more consistent grades (by the very nature of being a top performing school). Teach…

I completely agree that the system put in place is poor and that it causing an accidental bias is very feasible. But if you're going to put this system in place, with months of planning, you should make damn sure that it isn't massively favouring the children who are already advantaged in life. Which it statistically is, whether intentional or not. They screwed it up so spectacularly that the education secretary had…

Independent schools got higher grades because of a quirk in the algorithm. If there weren't enough students taking a particular course at a particular centre for the rank/previous results method to be reasonably accurate, the algorithm defaulted to partially or wholly relying on predicted grades.

Independent schools generally have fewer students and a wider range of courses and predicted grades were inflated across the board, so they ended up with better grades than larger colleges and academies.

I'm not sure how you could have avoided this bias without creating a lot of really weird outcomes.

Re: Why did the A-level algorithm say no?

#84

Post by jdietrich » Mon, Aug 17, 2020, 1:03 AM UTC

stoneman24 wrote: post #53

mjburgess wrote: post #28

Earlier quoted context omitted.

> it's hard not to be sceptical that this wasn't intentional and class-motivated. It's incredibly easy "not to be sceptical". The teacher's predicted grades are, on average, unprecedented . And represent many times greater than any previous yoy increase. This is far less true at top-performing schools where teachers are predicting far more consistent grades (by the very nature of being a top performing school). Teach…

Given the extent of some candidates differences and the impact to their life (denied entrance to offered Uni courses for that year) there’s some very serious outrage. If you were part of good class in a school with a poor record, then you’re stuffed. It’s a unprecedented situation and the only guide that you have are teacher grades.

>Given the extent of some candidates differences and the impact to their life (denied entrance to offered Uni courses for that year) there’s some very serious outrage.

It's not as bad as you might think. A large number of students are rejecting their offers with the intention of re-applying in 2021 and there has been a substantial reduction in the number of foreign students, so there are an unprecedented number of surplus places available through clearing. Even if your grades are substantially worse than you might have hoped, your chances of getting a place on a decent course are very good.

Re: Why did the A-level algorithm say no?

#85

Post by throwaway45349 » Mon, Aug 17, 2020, 1:06 AM UTC

switch007 wrote: post #67

russholmes wrote: post #38

Earlier quoted context omitted.

Small classes are excluded, so as you say, biased towards private schools. My niece was hit by this but luckily still did well enough to get a place. She was attending a large school in a deprived area. Her coursework was A*, given predicted AAA but was downgraded to ABB. The minister of education's reason for downgrading the grades was so that pupils did not end up in jobs they were not capable of doing. Delicious i…

> The minister of education's reason for downgrading the grades was so that pupils did not end up in jobs they were not capable of doing. Delicious irony. Fake news, sorry. https://fullfact.org/education/gavin-williamson-fake-quote/

Colloquially, worth remembering: "exams don't get you jobs, they just prevent you from getting jobs."

Re: Why did the A-level algorithm say no?

#86

Post by jdietrich » Mon, Aug 17, 2020, 1:10 AM UTC

weego wrote: post #14

macspoofing wrote: post #12

Earlier quoted context omitted.

>If what is written here is true ... So is it? The guy is mining theoretical examples, but I wonder if in practice anyone has been hurt by this. And if they have (let's say a non-failing student was given a failing grade), why not handle those cases within a manual appeals process? After all, the exams weren't written, so no matter what mechanism you choose their will be students that get higher markers than they oth…

Of course children have been hurt by this is practice, there are numerous cases of children getting down graded from their predicted results so badly they're having University offer places withdrawn, and it's not by a point or two, we're talking straight A students getting demotd to Ds across the board because their school has a history of bad results, which are irrelevant to any given student right now. Oxford have…

Disappointment on results day is par for the course. Grades awarded this year were higher than last year across the board, so it can't be argued that the system was unreasonably harsh. The centre-assessed grades provided by schools and colleges were unrealistically ambitious, so if they were taken at face value we would have seen an unprecedented level of grade inflation.

As always, students who are unhappy with their results have the option of (re)-sitting their exams and re-applying for university; given the substantial uncertainty over whether universities will be able to offer anything like a normal level of teaching in the next academic year, deferring might be a sensible option regardless of results.

Re: Why did the A-level algorithm say no?

#87

Post by mantap » Mon, Aug 17, 2020, 1:45 AM UTC

mjburgess wrote: post #52

kitd wrote: post #47

Earlier quoted context omitted.

Teachers predict grades all the time. It's part of how student progress is tracked. We are constantly being told what level our kids are at and predicted level at exam time. Part of the problem is that teachers often push the "prediction" slightly higher as a means of motivating the student. Now those predictions are being used as the real deal. I am pretty convinced that most teachers with more than superficial expe…

Well as a point of fact, they didnt. Their requested, "accurate" grades represent a 20% increase in general attainment. That has never happened in the history of modern education, and occurs only when, eg., civil wars end. As a comparative reference, we'd expect an at-most, 3% increase in IQ in the general population.

This is desirable and it's easy to explain why. For any student there will be a range of grades they could expect to achieve. The fairest grade to assign is the best case grade in that range (not the midpoint), because there is no evidence that they would not have studied hard and achieved that grade in the exam. If every student is given their best case grade then you will indeed see an overall rise, this simply reflects a lack of knowledge of who would have studied hard and who would not have.

Re: Why did the A-level algorithm say no?

#88

Post by rossjudson » Mon, Aug 17, 2020, 2:32 AM UTC

Twirrim wrote: post #46

Jabbles wrote: post #3

Do teachers recommend a grade, or a score? If they only recommend grades, how did the algorithm pick which students to change the grades for?

Here's a write-up of the situation, shared on Facebook publicly, from a friend involved in careers at a UK Sixth Form college (where students take their A-Levels): ################### I've spent the last two days dealing with very upset young people who have had their plans ripped up. Some will still be able to go to university, but not the one they planned on and have been forced into clearing, some won't be able to…

You've described a system where, if an exceptional kid comes from a lower-performing school, there is nothing they can do to gain entrance to the top universities.

I wonder if the effects are visible when comparing allocation of spots vs past years. Or is that what they are optimizing for? To achieve the same allocation across feeder schools as the year before?

Re: Why did the A-level algorithm say no?

#89

Post by sukilot » Mon, Aug 17, 2020, 2:39 AM UTC

jdietrich wrote: post #83

corin_ wrote: post #31

Earlier quoted context omitted.

I completely agree that the system put in place is poor and that it causing an accidental bias is very feasible. But if you're going to put this system in place, with months of planning, you should make damn sure that it isn't massively favouring the children who are already advantaged in life. Which it statistically is, whether intentional or not. They screwed it up so spectacularly that the education secretary had…

Independent schools got higher grades because of a quirk in the algorithm. If there weren't enough students taking a particular course at a particular centre for the rank/previous results method to be reasonably accurate, the algorithm defaulted to partially or wholly relying on predicted grades. Independent schools generally have fewer students and a wider range of courses and predicted grades were inflated across t…

Well, wealthy students who opt out of government secondary schools could be given lowest priority at government colleges, and if that offends someone they can expand government funded colleges so that all could attend.

Re: Why did the A-level algorithm say no?

#90

Post by sukilot » Mon, Aug 17, 2020, 2:47 AM UTC

ppod wrote: post #76

mjburgess wrote: post #39

Earlier quoted context omitted.

Right, and what was the previous yoy disparity? If the previous YoY was also 5%/2%, then it is just repeating the trend. It isnt the job of statisticians to "fix" social inequality by BSing some grades. Its their job to estimate likely exam results based on previous performance. There's absolutely no grounds on which to assume that growth in A*/A across these demographics would be the same; and of course, very many r…

You're making very strong statements over a huge assumption that you have no data to back up. This is not like previous years. The students sat no exam, they were ranked within their school, and then assigned grades based on the previous performance of their school. It is reverse affirmative action.

They sat for mocks.