Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did the A-level algorithm say no?

bbc.co.uk

Post reply on HN
Print view View original

71–80 of 143 posts

Re: Why did the A-level algorithm say no?

#71

Post by BayesianDice »

weego wrote: post #14
macspoofing wrote: post #12

Earlier quoted context omitted.

>If what is written here is true ... So is it? The guy is mining theoretical examples, but I wonder if in practice anyone has been hurt by this. And if they have (let's say a non-failing student was given a failing grade), why not handle those cases within a manual appeals process? After all, the exams weren't written, so no matter what mechanism you choose their will be students that get higher markers than they oth…

Of course children have been hurt by this is practice, there are numerous cases of children getting down graded from their predicted results so badly they're having University offer places withdrawn, and it's not by a point or two, we're talking straight A students getting demotd to Ds across the board because their school has a history of bad results, which are irrelevant to any given student right now. Oxford have…

To clarify this - as far as I've seen, so far, 3 of the 45 Oxford colleges/PPHs (Worcester, Wadham, and St Edmund Hall) are accepting all offer-holders regardless of results.

(Oxford, along with Cambridge but unlike many universities, has the results of interviews and, for many subjects, admissions tests to assess to whom to offer a place. They make offers expecting that the vast majority of them will be met, rather than over-offering and expecting A-level results to prune the numbers significantly. In previous incarnations of the admissions system, candidates who passed the entrance exam and interview routinely got an offer of two Es, i.e. a formality. So, providing places to all offer-holders would not seem an unreasonable course of action in the circumstances.)

Re: Why did the A-level algorithm say no?

#72

Post by stordoff »

oh_sigh wrote: post #22
weego wrote: post #13

Earlier quoted context omitted.

It's very clearly unfair, I'm not sure what your example has to do with anything. No students performance should be a sum based on how previous students have done at their school. That's literally reinforcing the postcode divide in education quality that were supposed to turn a blind eye to. My home town school has historically refused to even put children in for exams if they didn't get at least a C grade in their m…

My high school had one person 3 years prior to my senior year get into an Ivy, and then before that it was no one in recent memory. In my senior class, 6 of us got into Ivies. I can only imagine what my life would be like if we only got the "fair" allotment of zero Ivy spots in a system like this.

Similar happened to me (UK) - my school went from performing under the national average for the preceding three years to being notably above for my year group, and no one could remember anyone going to Oxbridge, but there were two of us in that year (the school barely even knew about the extra exams we had to take - required for one, optional for the other - so the prep. was more or less entirely self-taught).

Re: Why did the A-level algorithm say no?

#73

Post by stordoff »

macspoofing wrote: post #34
ngcazz wrote: post #24

Earlier quoted context omitted.

Even if you weren't completely mistaken in your assumptions, shouldn't students have the right to get the exam grades they actually deserve? I was at best average in secondary school and I had to kick it up a few notch for the national exams (Portuguese A-level equivalent). Under your "meh" style, I'd have lost at least a year before I'd be able to enroll in uni, because whatever!, despite getting scores above the 95…

>shouldn't students have the right to get the exam grades they actually deserve? Look, nothing I've seen here changes the base fact that if you were a good students getting good grades, you are going to be OK. It also doesn't change the base fact that mark inflation exits. That some schools will grade higher than others for the same academic performance. So no, you don't have a 'right' to a mark you haven't earned. I…

> Let's say you had weak course marks and you were hoping to use the test to boost your final score to get into the school of your choice. Then covid happened and you weren't able to. Well... take that as important life lesson. Don't slack off until the end and hope that you can make up your grades with the exams.

I'm not sure why what you mean by "weak course marks" - something like A-Level Maths is assessed entirely in exams at the end of the two year period. There's no formal assessment prior to that.

> If you had weak course marks in a school that inflates marks, you don't get to the spot of another student that worked hard during the term and was in a school that did not inflate marks.

Alternatively, if you _did_ work hard, but it's assumed that your predicted grades have been inflated, you face getting knocked down through no fault of your own. I also query the notion that you can reliably know which schools get predicted grades wrong. Teachers, and teaching methods, change every year - the people making the predictions are not necessarily the same year to year.

Re: Why did the A-level algorithm say no?

#75

Post by stordoff »

macspoofing wrote: post #15

A big pile of 'meh'. They are trying to make a statistical guess based on priors because the pandemic prevented the exams from being written. They are going to get things wrong. If they have an appeals process for things that are completely out line (e.g. a passing student getting a failing grade), then that's the best you can hope for given the circumstances. In the big picture, the good students are going to get go…

> In the big picture, the good students are going to get good markers, average students are going to get average markets, and bad students will get bad markets. Again, it is what it is.

In my year, I and one other student had to go significantly beyond the support provided to get the grades we needed. I studied one A-Level entirely outside school as they couldn't make the timetable work, and then added another to be done in a year (typically a two year course) because my school had given me bad advice about which were required. If it were this year, and my school were known for overpmarking, I'd face getting downgraded for something I had no control over. I was predicted, and needed, the top grades (AAA) in specific subjects, and even dropping to AAB would mean not getting into my university of choice. It can be a kick in the teeth for anyone who has put the extra effort in.

Re: Why did the A-level algorithm say no?

#76

Post by ppod »

mjburgess wrote: post #39
corin_ wrote: post #35

Earlier quoted context omitted.

Independant schools (private schools) received 4.7% more A/A* grades than they did in 2019. The most common type of non-private school, Secondary Comprehensive, received only 2% more A/A* grades than in 2019. The algorithm didn't keep the gap between paid and unpaid education, it expanded it. (Source for these numbers and more in my first comment.) I was lucky enough to not only go to two private schools, but the sec…

Right, and what was the previous yoy disparity? If the previous YoY was also 5%/2%, then it is just repeating the trend. It isnt the job of statisticians to "fix" social inequality by BSing some grades. Its their job to estimate likely exam results based on previous performance. There's absolutely no grounds on which to assume that growth in A*/A across these demographics would be the same; and of course, very many r…

You're making very strong statements over a huge assumption that you have no data to back up. This is not like previous years. The students sat no exam, they were ranked within their school, and then assigned grades based on the previous performance of their school. It is reverse affirmative action.

Re: Why did the A-level algorithm say no?

#77

Post by ppod »

mjburgess wrote: post #54
corin_ wrote: post #49

Earlier quoted context omitted.

I can't find that data easily, if I'm bored tomorrow I might try to do the research myself - all the data is available but raw not top level overview at https://www.compare-school-performance.service.gov.uk/downlo... It's only been a couple of days, we'll get more data journalists breaking it down better soon. You know what would be really helpful? If the algorithm was released to be publicly reviewed. Oh, but the go…

I agree. It should be published. However more important than the algorithm is just the media reporting on what ofqual actually say. I've barely read any of their actual analysis in news on the issue. Key facts: students are always ranked and graded on a curve. Student populations have a fixed number of grades awarded every year, etc. The system is specifically designed in recognition that grades are positional goods,…

>Student populations have a fixed number of grades awarded every year, etc.

Yes, for the national cohort. This is different, the predicted results were weighted on a per-school basis. For example, a particular child taking, say, a history exam, in a school which had previously had no pupils that achieved A* in history over the past four years had probably no chance of getting an A*, even if that was their teachers' prediction. Purely on the basis of which school they went to. A weak cohort (compared to previous years) at a good school would be upweighted.

Re: Why did the A-level algorithm say no?

#78

Post by ppod »

lhnz wrote: post #56
mjburgess wrote: post #48

Earlier quoted context omitted.

I meant all teachers across school types, not all teachers individually. In both state and private schools, teachers over-predicted. But higher-performing schools have higher and more uniform results, so there's less room to actually over predict -- so there's comparatively less reduction. That's just an in-built concequence of what it means to be a high performing school

Sure. It makes sense. I don't believe in the media conspiracy that the Tories created an algorithm that systematically awards higher grades to rich kids because of inherent classism or even by accident. But I do really disagree with the insistence that all teachers in comprehensive schools were conspiring to give their kids higher grades than they deserved -- this view was frequently repeated on social media. Even if…

Just to be clear about what actually happened, the results were weighted on a per-school basis, based on results from the previous four years. The disparity in weighting is not solely down to effects at the top of the distribution.

Re: Why did the A-level algorithm say no?

#79

Post by ppod »

mjburgess wrote: post #52
kitd wrote: post #47

Earlier quoted context omitted.

Teachers predict grades all the time. It's part of how student progress is tracked. We are constantly being told what level our kids are at and predicted level at exam time. Part of the problem is that teachers often push the "prediction" slightly higher as a means of motivating the student. Now those predictions are being used as the real deal. I am pretty convinced that most teachers with more than superficial expe…

Well as a point of fact, they didnt. Their requested, "accurate" grades represent a 20% increase in general attainment. That has never happened in the history of modern education, and occurs only when, eg., civil wars end. As a comparative reference, we'd expect an at-most, 3% increase in IQ in the general population.

I have no problem with the idea of down-weighting the entire national cohort to correct for this. The problem is that they actually weighted the results on a per-school basis.

Re: Why did the A-level algorithm say no?

#80

Post by peteretep »

ppod wrote: post #76
mjburgess wrote: post #39

Earlier quoted context omitted.

Right, and what was the previous yoy disparity? If the previous YoY was also 5%/2%, then it is just repeating the trend. It isnt the job of statisticians to "fix" social inequality by BSing some grades. Its their job to estimate likely exam results based on previous performance. There's absolutely no grounds on which to assume that growth in A*/A across these demographics would be the same; and of course, very many r…

You're making very strong statements over a huge assumption that you have no data to back up. This is not like previous years. The students sat no exam, they were ranked within their school, and then assigned grades based on the previous performance of their school. It is reverse affirmative action.

What are the strong statements you think he’s making?
Post reply on HN