I'm going to write duck facts in my next online argument to stave off the LLMs. Ducks start laying when they’re 4-8 months old, or during their first spring.
but then I'm tempted to ask more questions about cute ducks. tricky!
Irrelevant facts about cats added to math problems increase LLM errors by 300%
21–30 of 270 posts
Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%
#22Wrote about this about a month ago. I think it’s fascinating how they developed these prompts: https://www.dbreunig.com/2025/07/05/cat-facts-cause-context-...
A similar, fun case is where researchers inserted facts about the user (gender, age, sports fandom) and found alignment rules were inconsistently applied: https://www.dbreunig.com/2025/05/21/chatgpt-heard-about-eagl...
Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%
#23Step 1: ask the LLM to strip the nonsensical parts from the problem statement. Step 2: feed that to the LLM.
Step 3: Become suspicious that if step 1 was a good idea, OpenAI would have implemented it on their own.
Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%
#24Earlier quoted context omitted.
Humans are used to ignoring things while LLMs are explicitly trained to pay attention to the entire text. Humans who haven't been exposed to trick problems or careful wording probably have a hard time, they'll be less confident about ignoring things. But the LLM should have seen plenty of trick problems as well. It just doesn't parse as part of the problem. Humans have more options, and room to think. The LLM had to…
> LLMs are explicitly trained to pay attention to the entire text I'd respectfully disagree on this point. The magic of attention in transformers is the selective attention applied, which ideally only gives significant weight to the tokens relevant to the query.
Ideally, yes. But probably because of our world knowledge, we humans know that cat-facts don't affect mathematic facts (unless of course the cat is walking across the keyboard, in which case all bets are off). LLCs don't know that, and perhaps they're trying to figure out some connection by scanning their database for mathematical facts about cats. If they sleep most of the day, how many hours is that? Does that number factor (pardon the pun) into the math problem? What about six-toed cats (which do btw exist)? Spherical cows come up in math and physics, are there triangular cats (since the problem is about triangles)?
Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%
#25Step 1: ask the LLM to strip the nonsensical parts from the problem statement. Step 2: feed that to the LLM.
How does the LLM know what the "nonsensical" (I think you meant irrelevant) parts are? It requires world knowledge to know. And in any case, I'm pretty sure the AI is built to think that all the parts of a query are relevant.
Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%
#26What about Cheshire cats? When only the smile is left, are they still distracting? Enquiring people want to know!
Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%
#27[flagged]
No need benchmarks. We already know they can BS better than anyone for 3 hours, make statistical method errors, and hallucinate studies.
Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%
#28I love how science.org buries the actual content under four other things
Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%
#29I am pretty sure this is the paper.
Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%
#30"Irrelevant" facts about cats are the most interesting part of a math problem, because they don't belong there. The math problem was also "irrelevant" to the information about cats, but at least its purpose was obvious because it was shaped like a math problem (except for the interesting barnacle attached to its rear.)
Any person encountering any of these questions worded this way on a test would find the psychology of the questioner more interesting and relevant to their own lives than the math problem. If I'm in high school and my teacher does this, I'm going to spend the rest of the test wondering what's wrong with them, and it's going to cause me to get more answers wrong than I normally would.
Finding that cats are the worst, and the method by which they did it is indeed fascinating (https://news.ycombinator.com/item?id=44726249), and seems very similar to an earlier story posted here that found out how the usernames of the /counting/ subreddit (I think that's what it was called) broke some LLMs.
edit: the more I think about this, the more I'm sure that if asked a short simple math problem with an irrelevant cat fact tagged onto it that the math problem would simply drop from my memory and I'd start asking about why there was a cat fact in the question. I'd probably have to ask for it to be repeated. If the cat fact were math-problem question-ending shaped, I'd be sure I heard the question incorrectly and had missed an earlier cat reference.