Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Irrelevant facts about cats added to math problems increase LLM errors by 300%

science.org

Post reply on HN
Print view View original

181–190 of 270 posts

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#181

Post by imtringued »

krisoft wrote: post #140
diamond559 wrote: post #72

Earlier quoted context omitted.

Yeah you're right, if that human is 5 years old or has crippling ADHD.

You can argue until the cows come home. The point is that they claim without evidence that humans are not suspectible to this kind of distraction. If they want to estabilish this as a fact there is a trivialy easy experiment they can conduct. “Someone on hacker news strongly feels it is true, and is willing to argue the case with witty comments.” is not how scientific knowledge is estabilished. We either have done th…

The answer is three apples.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#182

Post by [deleted] »

userbinator wrote: post #164

This looks like it'll be useful for CAPTCHA purposes. According to the researchers, “the triggers are not contextual so humans ignore them when instructed to solve the problem”—but AIs do not. Not all humans, unfortunately: https://en.wikipedia.org/wiki/Age_of_the_captain

[deleted]

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#183

Post by qcnguy »

EGreg wrote: post #144
acc_297 wrote: post #80

There is more than one comment here asserting that the authors should have done a parallel comparison study against humans on the same question bank as if the study authors had set out to investigate whether humans or LLMs reason better in this situation. The authors do include the claim that humans would immediately disregard this information and maybe some would and some wouldn't that could be debated and seemingly…

Why are some people always trying to defend LLMs and say either “humans are also like this” or “this has always been a problem even before AIs” Listen, LLMs are different than humans. They are modeling things. Most RLHF makes them try to make sense of whatever you’re saying as much as you can. So they’re not going to disregard cats, OK? You can train LLMs to be extremely unhuman-like. Why anthropomorphize them?

It's because most use cases for AI involve replacing people. So if a person would suffer a problem and an AI does too it doesn't matter, it would just be a Nirvana fallacy to refuse the AI because it has the same problems as the previous people did.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#184

Post by portaouflop »

ants_everywhere wrote: post #173
orbital-decay wrote: post #168

Earlier quoted context omitted.

How far back do I have to look, and what definition do you use? Because I can start with theorem provers and chess engines of the 1950s. Nothing in that list is based on humans, even remotely. Only neural networks were a vague form of biomimicry early on and currently have academic biomimicry approaches, of which all suck because they poorly map to available semiconductor manufacturing processes. Attention is mislead…

> How far back do I have to look The earliest reference I know off the top of my head is Aristotle, which would be the 4th century BCE > I can start with theorem provers If you're going to talk about theorem provers, you may want to include the medieval theory of obligations and their game-semantic-like nature. Or the Socratic notion of a dialogue in which arguments are arrived at via a back and forth. Or you may wan…

You can compare them to humans but it’s kind of boring. Maybe more interesting if you are an “ai” researcher

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#185

Post by awanderingmind »

userbinator wrote: post #164

This looks like it'll be useful for CAPTCHA purposes. According to the researchers, “the triggers are not contextual so humans ignore them when instructed to solve the problem”—but AIs do not. Not all humans, unfortunately: https://en.wikipedia.org/wiki/Age_of_the_captain

Cool example in that link, thanks!

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#187

Post by Ekaros »

empath75 wrote: post #100
acc_297 wrote: post #80

There is more than one comment here asserting that the authors should have done a parallel comparison study against humans on the same question bank as if the study authors had set out to investigate whether humans or LLMs reason better in this situation. The authors do include the claim that humans would immediately disregard this information and maybe some would and some wouldn't that could be debated and seemingly…

I generally will respond to stuff like this with "people do this, too", but this result given their specific examples is genuinely surprising to me, and doesn't match at all my experience with using LLMs in practice, where it does frequently ignore irrelevant data in providing a helpful response. I do think that people think far too much about 'happy path' deployments of AI when there are so many ways it can go wrong…

When I think lot of use cases LLMs are planned for. I think not happy paths are critical. There is not insignificant number of people who would ramble about other things to customer support person if given opportunity. Or lack capability to only state needed and not add extra context.

There might be happy path when you isolated to one or a few things. But not in general use cases...

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#189

Post by [deleted] »

awanderingmind wrote: post #185
userbinator wrote: post #164

This looks like it'll be useful for CAPTCHA purposes. According to the researchers, “the triggers are not contextual so humans ignore them when instructed to solve the problem”—but AIs do not. Not all humans, unfortunately: https://en.wikipedia.org/wiki/Age_of_the_captain

Cool example in that link, thanks!

[deleted]

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#190

Post by janalsncm »

ants_everywhere wrote: post #165
orbital-decay wrote: post #154

Earlier quoted context omitted.

>The whole point of AI is to create an intelligence modeled on humans and to compare it to humans. According to who? Everyone who's anyone is trying to create highly autonomous systems that do useful work. That's completely unrelated to modeling them on humans or comparing them to humans.

Go back and look at the history of AI, including current papers from the most advanced research teams. Nearly every component is based on humans - neural net - long/short term memory - attention - reasoning - activation function - learning - hallucination - evolutionary algorithm If you're just consuming an AI to build a React app then you don't have to care. If you are building an artificial intelligence then in pra…

Those terms sound similar to biological concepts but they’re very different.

Neural networks are not like brains. They don’t grow new neurons. A “neuron” in an artificial neural net is represented with a single floating point number. Sometimes even quantized down to a 4 bit int. Their degrees of freedom are highly limited compared to a brain. Most importantly, the brain does not do back propagation like an ANN does.

LSTMs have about as much to do with brain memory as RAM does.

Attention is a specific mathematical operation applied to matrices.

Activation functions are interesting because originally they were more biologically inspired and people used sigmoid. Now people tend to use simpler ones like ReLU or its leaky cousin. Turns out what’s important is creating nonlinearities.

Hallucinations in LLMs have to do with the fact that they’re statistical models not grounded in reality.

Evolutionary algorithms, I will give you that one although they’re way less common than backprop.

Post reply on HN