Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Irrelevant facts about cats added to math problems increase LLM errors by 300%

science.org

Post reply on HN
Print view View original

61–70 of 270 posts

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#61

Post by pythonaut_16 »

pessimizer wrote: post #30

"Irrelevant" facts about cats are the most interesting part of a math problem, because they don't belong there. The math problem was also "irrelevant" to the information about cats, but at least its purpose was obvious because it was shaped like a math problem (except for the interesting barnacle attached to its rear.) Any person encountering any of these questions worded this way on a test would find the psychology…

On the other hand, this is helpful to know as a user of LLMs because it suggests that LLMs are bad at isolating the math problem from the cat fact. That means providing irrelevant context may be harmful to getting back a good answer in other domains as well.

Ideally you'd want the LLM to solve the math problem correctly and then comment on the cat fact or ask why it was included.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#62

Post by nomel »

IAmNotACellist wrote: post #56

This doesn't seem noteworthy. It's called a context window for a reason--because the input is considered context. You could train an LLM to consider the context potentially adversarial or irrelevant, and this phenomenon would go away, at the expense of the LLM sometimes considering real context to be irrelevant. To me, this observation sounds as trite as: "randomly pressing a button while inputting a formula on your…

This should be more of a problem for agents, with less bound context.

But, I would claim it’s a problem for a common use case if LLM of “here’s my all my code, add this feature and fix this”. How much of that code is irrelevant to the problem? Probably most of it.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#63

Post by brazzy »

pinkmuffinere wrote: post #18
Y_Y wrote: post #3

> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.

Ya, I specifically remember solving word problems in school / college and getting distracted by irrelevant details. Usually I would get distracted by stuff that _seemed_ like it should be used, so maybe cat facts would be fine for me to tease out, but in general I don't think I'm good at ignoring extraneous information. Edit: To be fair, in the example provided, the cat fact is _exceptionally_ extraneous, and even fl…

It's a well-known problem for humans as well: https://en.wikipedia.org/wiki/Age_of_the_captain

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#64

Post by HPsquared »

Ygg2 wrote: post #54
HPsquared wrote: post #49

Earlier quoted context omitted.

For extra distraction, make the facts incorrect. Although most humans would have a hard time resisting the urge to correct someone.

Up to ten Nobel laureates have been unveiled as being three ducks in a trenchcoat.

That's still technically true

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#65

Post by wagwang »

jmilloy wrote: post #34
Y_Y wrote: post #3

> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.

Did you look at the examples? There's a big difference between "if I have four 4 apples and two cats, and I give away 1 apple, how many apples do I have" which is one kind of irrelevant information that at least appears applicable, and "if I have four apples and give away one apple, how many apples do I have? Also, did you know cats use their tails to help balance?", which really wouldn't confuse most humans.

Yes, especially interview questions that include a stupid "real life example" that is usually irrelevant to the question.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#66

Post by layer8 »

Y_Y wrote: post #3

> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.

It would have been interesting to see how a human control group performs, but it also seems highly unlikely that it would triple their error rate.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#67

Post by krisoft »

jmilloy wrote: post #34
Y_Y wrote: post #3

> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.

Did you look at the examples? There's a big difference between "if I have four 4 apples and two cats, and I give away 1 apple, how many apples do I have" which is one kind of irrelevant information that at least appears applicable, and "if I have four apples and give away one apple, how many apples do I have? Also, did you know cats use their tails to help balance?", which really wouldn't confuse most humans.

> which really wouldn't confuse most humans

And i think it would. I think a lot of people would ask the invigilator to see if something is wrong with the test, or maybe answer both questions, or write a short answer on the cat question too or get confused and give up.

That is the kind of question where if it were put to a test I would expect kids to start squirming, looking at each other and the teacher, right as they reach that one.

I’m not sure how big this effect is, but it would be very surprising if there is no effect and unsuspecting, and unwarned people perform the same on the “normal” and the “distractions” test. Especially if the information is phrased as a question like in your example.

I heard it from teachers that students get distracted if they add irrelevant details to word problems. This is obviously anecdotal, but the teachers who I chatted about this thought it is because people are trained through their whole education that all elements of world problems must be used. So when they add extra bits people’s minds desperately try to use it.

But the point is not that i’m right. Maybe i’m totaly wrong. The point is that if the paper want to state as a fact one way or an other they should have performed an experiment. Or cite prior research. Or avoided stating an unsubstantiated opinion about human behaviour and stick to describing the AI.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#68

Post by layer8 »

electricboots wrote: post #14

Funny, I was using chatGPT to have a conversation with a friend that doesn't speak English the other day. At the end of one of my messages, I appended 'how is your cat?', which was completely dropped from the translated output. I guess I'm doing it wrong?

They already adjusted ChatGPT to that study. Unrelated trailing cat content is now ignored.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#69

Post by cwillu »

Xss3 wrote: post #41
Y_Y wrote: post #3

> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.

Read the article before commenting next time and you wont end up looking like a typical redditor.

“Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that". ”

--https://news.ycombinator.com/newsguidelines.html

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#70

Post by patall »

Xss3 wrote: post #42
patall wrote: post #32

I am ambivalent about these kinds of 'attack'. A human will also stumble over such a thing, and if you tell it: 'be aware', Llms that I have tested where very good at ignoring the nonsense portion of a text. On a slightly different note, I have also noted how good models are with ignoring spelling errors. In one hobby forum I frequent, one guy intentionally writes every single word with at least one spelling error (o…

Humans do not stumble over this. Did you read the article? They present a normal maths problem then add a random cat fact to the end or the start. Humans dont struggle with that...

Print out only the text and hand it, without any context, to a random other human and look what happens. I highly doubt that more than 25% will answer the question, and not because they are incapable of answering it.

What you forget is that you have context. Like: 'Look, LLMs are not able to answer this question!'. While you post the text without any context to the LLM.

Post reply on HN