Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did the A-level algorithm say no?

bbc.co.uk

Post reply on HN
Print view View original

31–40 of 143 posts

Re: Why did the A-level algorithm say no?

#31

Post by corin_ »

mjburgess wrote: post #28
corin_ wrote: post #23

Two things worth noting that the article doesn't go into... 1) "The year-on-year rise in proportion of students achieving A or A+ grades was much higher at independent schools than state comprehensives" https://www.theguardian.com/education/2020/aug/13/england-a-... Essentially, students lucky enough to be in private education were more likely to see their grades go up / stay the same, compared to students receiving…

> it's hard not to be sceptical that this wasn't intentional and class-motivated. It's incredibly easy "not to be sceptical". The teacher's predicted grades are, on average, unprecedented . And represent many times greater than any previous yoy increase. This is far less true at top-performing schools where teachers are predicting far more consistent grades (by the very nature of being a top performing school). Teach…

I completely agree that the system put in place is poor and that it causing an accidental bias is very feasible.

But if you're going to put this system in place, with months of planning, you should make damn sure that it isn't massively favouring the children who are already advantaged in life. Which it statistically is, whether intentional or not.

They screwed it up so spectacularly that the education secretary had to make concessions to it two days before the results, then a day after the results Ofqual made a statement about who could appeal that contradicted the education secretary, before withdrawing that statement a few hours later saying they were still deciding.

So sure it could be entirely down to incompetence not malice. But given either is possible, and this government's track record of cronyism I don't see why it's unreasonable to be sceptical.

Re: Why did the A-level algorithm say no?

#32

Post by mjburgess »

corin_ wrote: post #31
mjburgess wrote: post #28

Earlier quoted context omitted.

> it's hard not to be sceptical that this wasn't intentional and class-motivated. It's incredibly easy "not to be sceptical". The teacher's predicted grades are, on average, unprecedented . And represent many times greater than any previous yoy increase. This is far less true at top-performing schools where teachers are predicting far more consistent grades (by the very nature of being a top performing school). Teach…

I completely agree that the system put in place is poor and that it causing an accidental bias is very feasible. But if you're going to put this system in place, with months of planning, you should make damn sure that it isn't massively favouring the children who are already advantaged in life. Which it statistically is, whether intentional or not. They screwed it up so spectacularly that the education secretary had…

It doesnt "favour" any children.

Those children were already going to get the highest results.

There is no incompetence here. The predicted grades are a good estimate of what the results would have been.

The issue is that all teachers significantly over-predicted their students grades, but less so at high-performing schools. (Why? because the grades are more consistent at those schools, so there's less room for error).

Re: Why did the A-level algorithm say no?

#33

Post by pjc50 »

macspoofing wrote: post #12
jonp888 wrote: post #8

Earlier quoted context omitted.

If what is written here is true then it's way beyond 'unfair' and on towards mathemtically nonsensicle: https://twitter.com/alexhern/status/1294200553661227008 It is claimed that in any class where the chance of someone failing is judged to be > 0%, even if it is a tiny fraction of a percentage point, then the algorithm requires that someone in the class MUST be given a failing grade, even if it the weakest student i…

>If what is written here is true ... So is it? The guy is mining theoretical examples, but I wonder if in practice anyone has been hurt by this. And if they have (let's say a non-failing student was given a failing grade), why not handle those cases within a manual appeals process? After all, the exams weren't written, so no matter what mechanism you choose their will be students that get higher markers than they oth…

Twitter provides examples; A level art downgraded from C to U (hard fail). https://twitter.com/hannahMulley1/status/1294571475509415937...

Re: Why did the A-level algorithm say no?

#34

Post by macspoofing »

ngcazz wrote: post #24
macspoofing wrote: post #15

A big pile of 'meh'. They are trying to make a statistical guess based on priors because the pandemic prevented the exams from being written. They are going to get things wrong. If they have an appeals process for things that are completely out line (e.g. a passing student getting a failing grade), then that's the best you can hope for given the circumstances. In the big picture, the good students are going to get go…

Even if you weren't completely mistaken in your assumptions, shouldn't students have the right to get the exam grades they actually deserve? I was at best average in secondary school and I had to kick it up a few notch for the national exams (Portuguese A-level equivalent). Under your "meh" style, I'd have lost at least a year before I'd be able to enroll in uni, because whatever!, despite getting scores above the 95…

>shouldn't students have the right to get the exam grades they actually deserve?

Look, nothing I've seen here changes the base fact that if you were a good students getting good grades, you are going to be OK.

It also doesn't change the base fact that mark inflation exits. That some schools will grade higher than others for the same academic performance.

So no, you don't have a 'right' to a mark you haven't earned. If you had weak course marks in a school that inflates marks, you don't get to the spot of another student that worked hard during the term and was in a school that did not inflate marks.

I have yet to see a good argument that this model is bad (and it may be), or percentage of students have gotten a grade that is out of line with expectations. Also, I do support an appeals process for the cases where the algorithm just got it completely wrong.

>I was at best average in secondary school and I had to kick it up a few notch for the national exams (Portuguese A-level equivalent)

If you were a bad students, or a student on some margin, you were always going to rely on luck to get through. What if the the set of questions on the standardized test just so happen to coincide with areas you were weak in? I was in situations where I had a weak grasp on course material, but the exam just so happened to have questions in areas that I was strong in and I got a higher mark than I probably deserved. It happens. But again, things just average themselves out over time and students will end up where they 'deserve' to be.

>Under your "meh" style, I'd have lost at least a year before I'd be able to enroll in un

I'll play along. Let's say you had weak course marks and you were hoping to use the test to boost your final score to get into the school of your choice. Then covid happened and you weren't able to. Well... take that as important life lesson. Don't slack off until the end and hope that you can make up your grades with the exams.

Re: Why did the A-level algorithm say no?

#35

Post by corin_ »

mjburgess wrote: post #32
corin_ wrote: post #31

Earlier quoted context omitted.

I completely agree that the system put in place is poor and that it causing an accidental bias is very feasible. But if you're going to put this system in place, with months of planning, you should make damn sure that it isn't massively favouring the children who are already advantaged in life. Which it statistically is, whether intentional or not. They screwed it up so spectacularly that the education secretary had…

It doesnt "favour" any children. Those children were already going to get the highest results. There is no incompetence here. The predicted grades are a good estimate of what the results would have been. The issue is that all teachers significantly over-predicted their students grades, but less so at high-performing schools. (Why? because the grades are more consistent at those schools, so there's less room for error…

Independant schools (private schools) received 4.7% more A/A* grades than they did in 2019. The most common type of non-private school, Secondary Comprehensive, received only 2% more A/A* grades than in 2019.

The algorithm didn't keep the gap between paid and unpaid education, it expanded it.

(Source for these numbers and more in my first comment.)

I was lucky enough to not only go to two private schools, but the second one was ranked #1 in England for exam ranks 2 of the 3 years I was there (and helped me get good grades despite being extremely unmotivated and lazy). But at least my privilege was to be given a good education that made learning / passing exams (the latter more) easy, not that the government decided my school deserved an artificial grades boost compared to state schools.

Re: Why did the A-level algorithm say no?

#36

Post by macspoofing »

pjc50 wrote: post #29
macspoofing wrote: post #27

Earlier quoted context omitted.

Because mark inflation is a thing, and schools are different. There are schools that will give out A+ and others a B, for the same academic performance. School reputations don't change from year to year. So a school that inflates marks one year, will more than likely inflate them the next year. Are you suggesting that this should not be taken into consideration? That you should simply trust the relative weight of gra…

Per other thread, A levels are marked outside the school by the qualifications body.

Right. A levels were not written so a statistical model was created to approximate A level results based on prior performance of applicants correlated with their school and course marks. Yeah, I get all that. It changes nothing about my argument.

Re: Why did the A-level algorithm say no?

#37

Post by naruciakk »

Couldn't the UK just have a regular exam, just with the social distancing measures implemented? The exams were held in many European countries with these measures and nothing bad happened and the children has received a fair examination for the University

Re: Why did the A-level algorithm say no?

#38

Post by russholmes »

viburnum wrote: post #20
macspoofing wrote: post #15

A big pile of 'meh'. They are trying to make a statistical guess based on priors because the pandemic prevented the exams from being written. They are going to get things wrong. If they have an appeals process for things that are completely out line (e.g. a passing student getting a failing grade), then that's the best you can hope for given the circumstances. In the big picture, the good students are going to get go…

Nope, they excluded small schools (expensive private schools) from the algorithm.

Small classes are excluded, so as you say, biased towards private schools. My niece was hit by this but luckily still did well enough to get a place. She was attending a large school in a deprived area. Her coursework was A*, given predicted AAA but was downgraded to ABB.

The minister of education's reason for downgrading the grades was so that pupils did not end up in jobs they were not capable of doing. Delicious irony.

Re: Why did the A-level algorithm say no?

#39

Post by mjburgess »

corin_ wrote: post #35
mjburgess wrote: post #32

Earlier quoted context omitted.

It doesnt "favour" any children. Those children were already going to get the highest results. There is no incompetence here. The predicted grades are a good estimate of what the results would have been. The issue is that all teachers significantly over-predicted their students grades, but less so at high-performing schools. (Why? because the grades are more consistent at those schools, so there's less room for error…

Independant schools (private schools) received 4.7% more A/A* grades than they did in 2019. The most common type of non-private school, Secondary Comprehensive, received only 2% more A/A* grades than in 2019. The algorithm didn't keep the gap between paid and unpaid education, it expanded it. (Source for these numbers and more in my first comment.) I was lucky enough to not only go to two private schools, but the sec…

Right, and what was the previous yoy disparity?

If the previous YoY was also 5%/2%, then it is just repeating the trend.

It isnt the job of statisticians to "fix" social inequality by BSing some grades. Its their job to estimate likely exam results based on previous performance.

There's absolutely no grounds on which to assume that growth in A*/A across these demographics would be the same; and of course, very many reasons to presume they'd be different.

Post reply on HN