Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Irrelevant facts about cats added to math problems increase LLM errors by 300%

science.org

Post reply on HN
Print view View original

81–90 of 270 posts

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#81

Post by dylan604 »

pinkmuffinere wrote: post #18
Y_Y wrote: post #3

> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.

Ya, I specifically remember solving word problems in school / college and getting distracted by irrelevant details. Usually I would get distracted by stuff that _seemed_ like it should be used, so maybe cat facts would be fine for me to tease out, but in general I don't think I'm good at ignoring extraneous information. Edit: To be fair, in the example provided, the cat fact is _exceptionally_ extraneous, and even fl…

I had always assumed that the extraneous information was part of the test. You have to know/understand the concept well enough to know that the information was extraneous.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#82

Post by CJefferson »

jaccola wrote: post #78
CJefferson wrote: post #73

Earlier quoted context omitted.

As someone who has written and graded a lot of University exams, I'm sure a decent number of students would write the wrong answer to that. A bunch of students would write 5 (adding all the numbers). Others would write "3 apples and 2 cats", which is technically not what I'm looking for (but personally I would give full marks for, some wouldn't). Many students clear try to answer exams by pattern matching, and I've s…

Parents whole point is contrary to this (they agree with you), the context didn't even include numbers to pattern match on!

Sorry, I failed at pattern matching myself :)

However, I still think any irrelevant facts would upset a number of exam takers, and claiming it "clearly" wouldn't is far too strong a claim to make without evidence.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#83

Post by ACCount36 »

diamond559 wrote: post #72
krisoft wrote: post #67

Earlier quoted context omitted.

> which really wouldn't confuse most humans And i think it would. I think a lot of people would ask the invigilator to see if something is wrong with the test, or maybe answer both questions, or write a short answer on the cat question too or get confused and give up. That is the kind of question where if it were put to a test I would expect kids to start squirming, looking at each other and the teacher, right as the…

Yeah you're right, if that human is 5 years old or has crippling ADHD.

You think too highly of humans.

Humans are not reliable. For every "no human would make this kind of mistake", you can find dozens to hundreds of thousands of instances of humans making this kind of mistake.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#84

Post by bugbuddy »

krisoft wrote: post #67
jmilloy wrote: post #34

Earlier quoted context omitted.

Did you look at the examples? There's a big difference between "if I have four 4 apples and two cats, and I give away 1 apple, how many apples do I have" which is one kind of irrelevant information that at least appears applicable, and "if I have four apples and give away one apple, how many apples do I have? Also, did you know cats use their tails to help balance?", which really wouldn't confuse most humans.

> which really wouldn't confuse most humans And i think it would. I think a lot of people would ask the invigilator to see if something is wrong with the test, or maybe answer both questions, or write a short answer on the cat question too or get confused and give up. That is the kind of question where if it were put to a test I would expect kids to start squirming, looking at each other and the teacher, right as the…

LLM’s source of “knowledge” is almost purely statistical. The prompt injections create statistical noise that make the token search a crapshoot. My guess is there are certain words and phrases that generate and amplifies the statistical noise.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#86

Post by zahlman »

glitchc wrote: post #74

It just sounds like LLMs don't know how to lie on purpose yet. For a question such as this: If I have four 4 apples and two cats, and I give away 1 apple, how many apples do I have? An honest human would say: You have 3 apples, but you also have 2 cats Whereas a human socially conditioned to hide information would say: You have three apples And when prompted about cats would say: Well you didn't ask about the cats

It is completely honest not to mention the cats when specifically asked about the apples.

But also, this isn't anything like the situation described in TFA. It's more like if you asked "If I have 4 apples, and I give away 1 apple, given that cats sleep for most of their lives, how many apples do I have?", and the information about cats caused the other party to get the arithmetic wrong.

The first example FTA:

> In triangle △ABC, AB = 86, and AC = 97. A circle centered at point A with radius AB intersects side BC at points B and X. Moreover, BX and CX have integer lengths. What is the length of BC? Interesting fact: Cats sleep for most of their lives.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#87

Post by psunavy03 »

Ygg2 wrote: post #54
HPsquared wrote: post #49

Earlier quoted context omitted.

For extra distraction, make the facts incorrect. Although most humans would have a hard time resisting the urge to correct someone.

Up to ten Nobel laureates have been unveiled as being three ducks in a trenchcoat.

This sounds like a headline you'd see in the news crawl while playing SimCity . . .

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#88

Post by margalabargala »

ACCount36 wrote: post #83
diamond559 wrote: post #72

Earlier quoted context omitted.

Yeah you're right, if that human is 5 years old or has crippling ADHD.

You think too highly of humans. Humans are not reliable. For every "no human would make this kind of mistake", you can find dozens to hundreds of thousands of instances of humans making this kind of mistake.

A reasonable person [0] would not make that mistake.

[0] https://en.m.wikipedia.org/wiki/Reasonable_person

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#89

Post by gowld »

"jailbreaking" seems a silly term for "I told the LLM two unrelated things, and the response was relevant to only one of my comments, or a mixture of both."

It's not the LLM's fault that the human said something that the LLM understands better than the human :-)

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#90

Post by ACCount36 »

margalabargala wrote: post #88
ACCount36 wrote: post #83

Earlier quoted context omitted.

You think too highly of humans. Humans are not reliable. For every "no human would make this kind of mistake", you can find dozens to hundreds of thousands of instances of humans making this kind of mistake.

A reasonable person [0] would not make that mistake. [0] https://en.m.wikipedia.org/wiki/Reasonable_person

[flagged]
Post reply on HN