41–50 of 270 posts
> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.
I am ambivalent about these kinds of 'attack'. A human will also stumble over such a thing, and if you tell it: 'be aware', Llms that I have tested where very good at ignoring the nonsense portion of a text. On a slightly different note, I have also noted how good models are with ignoring spelling errors. In one hobby forum I frequent, one guy intentionally writes every single word with at least one spelling error (o…
They present a normal maths problem then add a random cat fact to the end or the start. Humans dont struggle with that...
Step 1: ask the LLM to strip the nonsensical parts from the problem statement. Step 2: feed that to the LLM.
Wrote about this about a month ago. I think it’s fascinating how they developed these prompts: https://www.dbreunig.com/2025/07/05/cat-facts-cause-context-...
A similar, fun case is where researchers inserted facts about the user (gender, age, sports fandom) and found alignment rules were inconsistently applied: https://www.dbreunig.com/2025/05/21/chatgpt-heard-about-eagl...
> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.
Did you look at the examples? There's a big difference between "if I have four 4 apples and two cats, and I give away 1 apple, how many apples do I have" which is one kind of irrelevant information that at least appears applicable, and "if I have four apples and give away one apple, how many apples do I have? Also, did you know cats use their tails to help balance?", which really wouldn't confuse most humans.
Maybe I'm totally wrong about that, but they really should have tested humans too, without that context this result seems lacking.
I'm going to write duck facts in my next online argument to stave off the LLMs. Ducks start laying when they’re 4-8 months old, or during their first spring.
Edit: a quick re-search shows they’ve differentiated a bit. But why are cats just the lowest common denominator? As someone who is allergic to them any cat reference immediately falls flat (personal problem, I know).
I'm going to write duck facts in my next online argument to stave off the LLMs. Ducks start laying when they’re 4-8 months old, or during their first spring.
> Now, if I asked you, presumably a human, to solve that math problem, you’d likely have no issue ignoring the totally unrelated aside at the end there I'm not so sure that is true. Good math students could ignore the cat fact, but I bet if you run this experimental in non-AP math classes you'll see an effect.
I think this would be true if the irrelevant information was within the question, but in this case it is tacked on to the end. Usually when irrelevant information trips up students, it is because it seems like part of the problem. When it's stuck on the end and preceded by "Random fact," as in this study, I don't think it would trip up the students. The only case where it might is if the student is reading the proble…
It would be easier to ignore if it were before the problem.