31–40 of 270 posts
Step 1: ask the LLM to strip the nonsensical parts from the problem statement. Step 2: feed that to the LLM.
How does the LLM know what the "nonsensical" (I think you meant irrelevant) parts are? It requires world knowledge to know. And in any case, I'm pretty sure the AI is built to think that all the parts of a query are relevant.
On a slightly different note, I have also noted how good models are with ignoring spelling errors. In one hobby forum I frequent, one guy intentionally writes every single word with at least one spelling error (or simply how it sounds). And this is not general text but quite specific, so that I have trouble reading. Llms (phind.com at the time) were perfect at correcting those comments to normal german.
> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.
> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.
Step 1: ask the LLM to strip the nonsensical parts from the problem statement. Step 2: feed that to the LLM.
Step 3: Become suspicious that if step 1 was a good idea, OpenAI would have implemented it on their own.
I am ambivalent about these kinds of 'attack'. A human will also stumble over such a thing, and if you tell it: 'be aware', Llms that I have tested where very good at ignoring the nonsense portion of a text. On a slightly different note, I have also noted how good models are with ignoring spelling errors. In one hobby forum I frequent, one guy intentionally writes every single word with at least one spelling error (o…
Earlier quoted context omitted.
Humans are used to ignoring things while LLMs are explicitly trained to pay attention to the entire text. Humans who haven't been exposed to trick problems or careful wording probably have a hard time, they'll be less confident about ignoring things. But the LLM should have seen plenty of trick problems as well. It just doesn't parse as part of the problem. Humans have more options, and room to think. The LLM had to…
> LLMs are explicitly trained to pay attention to the entire text I'd respectfully disagree on this point. The magic of attention in transformers is the selective attention applied, which ideally only gives significant weight to the tokens relevant to the query.
The model architecture is actually already known to have effects on some tasks. In particular, SSMs are worse than transformers at retrieving specific information from the context window [1], which e.g. reduces their performance on multiple choice benchmarks. Which is a performance difference that isn't reflected in their language modeling ability (perplexity).
> Now, if I asked you, presumably a human, to solve that math problem, you’d likely have no issue ignoring the totally unrelated aside at the end there I'm not so sure that is true. Good math students could ignore the cat fact, but I bet if you run this experimental in non-AP math classes you'll see an effect.
I think this would be true if the irrelevant information was within the question, but in this case it is tacked on to the end. Usually when irrelevant information trips up students, it is because it seems like part of the problem. When it's stuck on the end and preceded by "Random fact," as in this study, I don't think it would trip up the students. The only case where it might is if the student is reading the proble…
> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irrelevant information, even when it isn't about cats. I would have insisted on a human control group to compare the results with.