What the world can learn from the UK's A-level grading fiasco • Hacker News Board

Re: What the world can learn from the UK's A-level grading fiasco

#11

Post by meheleventyone » Wed, Sep 02, 2020, 9:37 AM UTC

tonyedgecombe wrote: post #8

jonplackett wrote: post #2

The problem was not using an algorithm, it was that they used a crap algorithm that could award students grades that were higher than those even available on the paper that they sat, and made it impossible to get a high grade if no-one fromm your college got one before. I'd love to see the details of the algorithm itself. Surely it can't have been too difficult to put some guard rails in there to prevent it moving an…

and made it impossible to get a high grade if no-one fromm your college got one before Well that is unlikely. If you are going to handle it algorithmically then what else would you expect. Clearly the teachers can't be relied on because now everybody is getting an A*, the grades are effectively useless as an indicator. I was disappointed although not surprised that the government caved on this.

Personally I’d expect assessment of an individual to be about their performance not their schools past performance. Any system not predicated on that is doomed to fail in assessing students performance as this one obviously did.

Re: What the world can learn from the UK's A-level grading fiasco

#12

Post by agent008t » Wed, Sep 02, 2020, 9:37 AM UTC

How did they deal with this problem in other countries, say Ireland or Japan?

Re: What the world can learn from the UK's A-level grading fiasco

#13

Post by MaxBarraclough » Wed, Sep 02, 2020, 9:38 AM UTC

londons_explore wrote: post #4

jonplackett wrote: post #2

The problem was not using an algorithm, it was that they used a crap algorithm that could award students grades that were higher than those even available on the paper that they sat, and made it impossible to get a high grade if no-one fromm your college got one before. I'd love to see the details of the algorithm itself. Surely it can't have been too difficult to put some guard rails in there to prevent it moving an…

The algorithm was the same one used for decades, and previously published. It pretty much says: > You get your exam grade. If that isn't available, you get a grade set by averaging your peers from the same school, with the set of peers decided by a ranking set by your teacher. If that isn't available, you get a grade chosen by your raking in class selected from the distribution of your schools past performance. There…

There's more to it. The system being used will lead to grade-inflation, [0] as there are two grade-estimation systems in use and the student gets to keep the better of the two.

The two systems used are teachers' grade predictions, and the grade-prediction algorithm.

We discussed this two weeks ago at https://news.ycombinator.com/item?id=24191882

[0] https://www.bbc.co.uk/news/uk-53810655

Re: What the world can learn from the UK's A-level grading fiasco

#14

Post by lordnacho » Wed, Sep 02, 2020, 9:39 AM UTC

It's hard to see how the algo could do anything but disappoint. If you don't have the exam information, what information have you got left? Just results from previous years, and a guess as to where you might be in the class, from a teacher who doesn't want to disappoint anyone.

But people have since forever seen exams as a way to stand out from the crowd, individually. If you went to a bad school, exams were a chance to show some extra effort. If you were a slacker, you'd be found out.

Without the specific exam results, all you've got is the reputation of the school, and chances are variation within a school is greater than between.

Re: What the world can learn from the UK's A-level grading fiasco

#15

Post by chippy » Wed, Sep 02, 2020, 9:39 AM UTC

tonyedgecombe wrote: post #8

jonplackett wrote: post #2

The problem was not using an algorithm, it was that they used a crap algorithm that could award students grades that were higher than those even available on the paper that they sat, and made it impossible to get a high grade if no-one fromm your college got one before. I'd love to see the details of the algorithm itself. Surely it can't have been too difficult to put some guard rails in there to prevent it moving an…

and made it impossible to get a high grade if no-one fromm your college got one before Well that is unlikely. If you are going to handle it algorithmically then what else would you expect. Clearly the teachers can't be relied on because now everybody is getting an A*, the grades are effectively useless as an indicator. I was disappointed although not surprised that the government caved on this.

Two reasons: The results indicate that the algorithm was flawed: "Ofqual figures show 39.1% of 700,000 teacher assessments were lowered by at least one grade"

Secondly, many students and teachers complained. This second reason is the thing that "the world can learn" that the article focuses on.

Re: What the world can learn from the UK's A-level grading fiasco

#16

Post by MaxBarraclough » Wed, Sep 02, 2020, 9:40 AM UTC

nicoburns wrote: post #7

londons_explore wrote: post #4

Earlier quoted context omitted.

The algorithm was the same one used for decades, and previously published. It pretty much says: > You get your exam grade. If that isn't available, you get a grade set by averaging your peers from the same school, with the set of peers decided by a ranking set by your teacher. If that isn't available, you get a grade chosen by your raking in class selected from the distribution of your schools past performance. There…

> There isn't really any fairer way that doesn't lead to grade inflation when there is an element of dishonesty/optimism on the part of teachers. IMO it's much better just to swallow the grade inflation for a year. It will naturally reset itself next year (when exams will be sat again), and the negative effect from it are pretty minimal. The algorithm is based on the assumption that performance of successive school y…

> IMO it's much better just to swallow the grade inflation for a year

You're not alone, this is the thinking they're using. [0]

> the negative effect from it are pretty minimal

That doesn't sound right. Top-flight students will be disadvantaged.

[0] https://www.bbc.co.uk/news/uk-53810655

Re: What the world can learn from the UK's A-level grading fiasco

#17

Post by [deleted] » Wed, Sep 02, 2020, 9:41 AM UTC

feintruled wrote: post #9

jonplackett wrote: post #2

The problem was not using an algorithm, it was that they used a crap algorithm that could award students grades that were higher than those even available on the paper that they sat, and made it impossible to get a high grade if no-one fromm your college got one before. I'd love to see the details of the algorithm itself. Surely it can't have been too difficult to put some guard rails in there to prevent it moving an…

And don't forget the totally illogical situation of awarding someone a higher grade for advanced maths then they got for basic maths. I had a passing thought of a different system - using basically the same calculation of a school's predicted grades, allocate each school a 'budget' of marks they can distribute as they see fit, based on whatever criteria they pick. Just to be clear, I don't think that's a particularly…

[deleted]

Re: What the world can learn from the UK's A-level grading fiasco

#18

Post by makomk » Wed, Sep 02, 2020, 9:42 AM UTC

In reality, I think this fiasco has more to do with politics than algorithms. One of the big problems is that the alternatives had downsides which were largely glossed over for political reasons - for example the university growth cap isn't just an arbitrary creation, it exists to stop more prestigious universities siphoning off a disproportionate share of students and leaving less prestigious ones in deep trouble and this seems likely to happen now.

Another big problem is that private schools seem to have made more realistic predictions of their students' results than state schools, as demonstrated by the fact that just taking many of their predictions as-is only caused slightly more grade inflation than in the state school results where 40% were downgraded - and this was portrayed in the press as proof that it was an attack on state schools because their results were disproportionately affected by the algorithmic downgrading.

A third problem is that what it was OK to be concerned about varied depending on which political side it benefitted. For example, there was a completely false and made up claim on social media and in publications that should know better that the Education Secretary had said taking the results would cause students to be promoted into jobs that they weren't competent to do: https://fullfact.org/education/gavin-williamson-fake-quote/ I can't imagine that it would've gone down any better if he claimed students would get onto courses they weren't good enough for, yet a week later after GCSEs went the other way and used predicted results that exact claim was uncritically regurgitated by the media.

Re: What the world can learn from the UK's A-level grading fiasco

#19

Post by tonyedgecombe » Wed, Sep 02, 2020, 9:44 AM UTC

chippy wrote: post #15

tonyedgecombe wrote: post #8

Earlier quoted context omitted.

and made it impossible to get a high grade if no-one fromm your college got one before Well that is unlikely. If you are going to handle it algorithmically then what else would you expect. Clearly the teachers can't be relied on because now everybody is getting an A*, the grades are effectively useless as an indicator. I was disappointed although not surprised that the government caved on this.

Two reasons: The results indicate that the algorithm was flawed: "Ofqual figures show 39.1% of 700,000 teacher assessments were lowered by at least one grade" Secondly, many students and teachers complained. This second reason is the thing that "the world can learn" that the article focuses on.

Which can be explained by teachers assessments being too optimistic. Now we have reverted to that you can see the results, grade inflation.

Of course students complained, I would if I didn't get the grades I was hoping for. That doesn't mean the government has to cave in and pretend everybody is a winner.

Re: What the world can learn from the UK's A-level grading fiasco

#20

Post by londons_explore » Wed, Sep 02, 2020, 9:45 AM UTC

nicoburns wrote: post #7

londons_explore wrote: post #4

Earlier quoted context omitted.

The algorithm was the same one used for decades, and previously published. It pretty much says: > You get your exam grade. If that isn't available, you get a grade set by averaging your peers from the same school, with the set of peers decided by a ranking set by your teacher. If that isn't available, you get a grade chosen by your raking in class selected from the distribution of your schools past performance. There…

> There isn't really any fairer way that doesn't lead to grade inflation when there is an element of dishonesty/optimism on the part of teachers. IMO it's much better just to swallow the grade inflation for a year. It will naturally reset itself next year (when exams will be sat again), and the negative effect from it are pretty minimal. The algorithm is based on the assumption that performance of successive school y…

> IMO it's much better just to swallow the grade inflation for a year.

When hiring, I'll be crossing off any qualifications earned in 2020. As well as the results being 'fake', the actual learning was in most cases not being done because the students knew they could chill at home and not learn. Sure, some students were studying, but as an employer I don't want to take that risk.

Would you want your doctor to be the one who messed around on TikTok for the whole year of his final exams?