We went from Impossible to Unreliable. I like the direction as a techie.
But not sure as a sociologist or an anthropologist.
Positional preferences, order effects, prompt sensitivity undermine AI judgments
21–30 of 87 posts
Re: Positional preferences, order effects, prompt sensitivity undermine AI judgments
#22[flagged]
Re: Positional preferences, order effects, prompt sensitivity undermine AI judgments
#23I’d argue real judges are unreliable as well. The real question for me is: are they less reliable than human judges? Probably yes. But I favor a relative measurement to humans than a plain statement like that.
> The real question for me is: are they less reliable than human judges?
I'd caution that it's never just about ratios: We must also ask whether the "shape" of their performance is knowable and desirable. A chess robot's win-rate may be wonderful, but we are unthinkingly confident a human wouldn't "lose" by disqualification for ripping off an opponent's finger.
Would we accept a "judge" that is fairer on average... but gives ~5% lighter sentences to people with a certain color shirt, or sometimes issues the death-penalty for shoplifting? Especially when we cannot diagnose the problem or be sure we fixed it? (Maybe, but hopefully not without a lot of debate over the risks!)
In contrast, there's a huge body of... of stuff regarding human errors, resources we deploy so pervasively it can escape our awareness: Your brain is a simulation and diagnostic tool for other brains, battle-tested (sometimes literally) over millions of years; we intuit many kinds of problems or confounding factors to look for, often because we've made them ourselves; and thousands of years of cultural practice for detection, guardrails, and error-compensating actions. Only a small minority of that toolkit can be reused for "AI."
Re: Positional preferences, order effects, prompt sensitivity undermine AI judgments
#24[flagged]
Re: Positional preferences, order effects, prompt sensitivity undermine AI judgments
#25Re: Positional preferences, order effects, prompt sensitivity undermine AI judgments
#26I’d argue real judges are unreliable as well. The real question for me is: are they less reliable than human judges? Probably yes. But I favor a relative measurement to humans than a plain statement like that.
I think the main difference is an AI judge may provide three different rulings if you just ask it the same thing three times. A human judge is much less likely to be so "flip-floppy". You can observe this using any of the present-day LLM's - ask it an architectural/design question, provide it with your thoughts, reasoning, constraints, etc... and see what it tells you. Then... click the "Retry" button and see how sim…
Humans flip-flop all the time. This is a major reason why the Meyers-Briggs Type Indicator does such a poor job of assigning the same person the same Meyers-Briggs type on successive tests.
It can be difficult to observe this fact in practice because, unlike for an LLM, you can't just ask a human the exact same question three times in five seconds and get three different answers, because unlike an LLM we have memory. But, as someone who works with human-labeled data, it's something I have to contend with on a daily basis. For the things I'm working on, if you give the same annotator the same thing to label two different times spaced far enough apart for them to forget that they have seen this thing before, the chance of them making the same call both times is only about 75%. If I do that with a prompted LLM anotator, I'm used to seeing more like 85%, and for some models it can be possible to get even better consistency than that with the right conditions and enough time spent fussing with the prompt.
I still prefer the human labels when I can afford them because LLM labeling has plenty of other problems. But being more flip-floppy than humans is not one that I have been able to empirically observe.
Re: Positional preferences, order effects, prompt sensitivity undermine AI judgments
#27Fully agree, I've found that LLMs aren't good at tasks that require evaluation. Think about it, if they were good at evaluation, you could remove all humans in the loop and have recursively self improving AGI. Nice to see an article that makes a more concrete case.
Humans aren't good at validation either. We need tools, experiments, labs. Unproven ideas are a dime a dozen. Remember the hoopla about room temperature superconductivity? The real source of validation is external consequences.
Re: Positional preferences, order effects, prompt sensitivity undermine AI judgments
#28I’d argue real judges are unreliable as well. The real question for me is: are they less reliable than human judges? Probably yes. But I favor a relative measurement to humans than a plain statement like that.
Can we stop with the "AI being unreliable like people" because it is demonstrably false at best and cult like thought termination at the worst.
Re: Positional preferences, order effects, prompt sensitivity undermine AI judgments
#29[flagged]
Comments like this break the site guidelines, and not just a little. Can you please review https://news.ycombinator.com/newsguidelines.html and take the intended spirit of this site more to heart? Note these:
"Please don't fulminate."
"Don't be curmudgeonly. Thoughtful criticism is fine, but please don't be rigidly or generically negative."
"Please don't sneer, including at the rest of the community."
"When disagreeing, please reply to the argument instead of calling names. 'That is idiotic; 1 + 1 is 2, not 3' can be shortened to '1 + 1 is 2, not 3."
There's plenty of LLM skepticism on HN and that's fine, but like all comments here, it needs to be thoughtful.
(We detached this comment from https://news.ycombinator.com/item?id=44074957)
Re: Positional preferences, order effects, prompt sensitivity undermine AI judgments
#30[flagged]
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
https://news.ycombinator.com/newsguidelines.html
(We detached this comment from https://news.ycombinator.com/item?id=44074957)