Or do we really understand it? Many humans ignore negation and condition too.
Language models still struggle with the concept of negation
31–40 of 172 posts
Re: Language models still struggle with the concept of negation
#32Does this increase anyone else's sense that LLMs are actually modeling something similar to the way humans think? It certainly does for me. I find that negations/conjugations of negations are surprisingly difficult for to parse. Beyond two negations at most, I can feel myself having to switch into "code" mode, where I'm explicitly casing out the booleans, to understand them. Obviously the examples given in the articl…
This specific example doesn’t, but I think there’s a fair amount of brain processing similar to what LLMs do. I just don’t think it’s the part of our mental experience that we regard as making decisions; it’s more like the part that is active when absent-mindedly daydreaming.
Re: Language models still struggle with the concept of negation
#33This is an old complaint about language models that GPT seems to have mostly overcome
This article smells like Google PR.
Re: Language models still struggle with the concept of negation
#34This is one of my favorite test cases: Prompt: Here is a riddle. It is a common riddle but with some changes that make it more difficult. You are an alien that will live for at least 10000 years. You have no sense of temperature. you can not feel hot or cold. you have eyes and can see. you are in a house. downstairs are 3 light switches that control 3 light bulbs that are upstairs. Each light bulb will last for exact…
It's telling that it didn't pick up on the fact that the whole "once you look you can't touch the switches anymore" isn't in this version of the riddle. I mean the obvious strategy in this case is turn on the first switch, go upstairs, look at the bulbs. Go back downstairs and try the second switch. You've not got them all mapped out in about 30 seconds.
Using GTP4 I asked if there was a way to do it in less than 3 years, but it couldn't figure this out even if I told it you can look and use the switches as much as you want. Instead it suggested turning on a switch for 10 minutes, then using your "excellent alien vision" determine which 3 year lifespan bulb has 10 minutes of wear on it.
Makes me think GPT4 doesn't really have better reasoning, it just looks like better reasoning because it's been fed way more data.
Re: Language models still struggle with the concept of negation
#35As these LLMs can not give an answer only can give what the answer would sound like, there's basically nothing to see here. The shape of the answer is still the same.
Re: Language models still struggle with the concept of negation
#36This is one of my favorite test cases: Prompt: Here is a riddle. It is a common riddle but with some changes that make it more difficult. You are an alien that will live for at least 10000 years. You have no sense of temperature. you can not feel hot or cold. you have eyes and can see. you are in a house. downstairs are 3 light switches that control 3 light bulbs that are upstairs. Each light bulb will last for exact…
It's telling that it didn't pick up on the fact that the whole "once you look you can't touch the switches anymore" isn't in this version of the riddle. I mean the obvious strategy in this case is turn on the first switch, go upstairs, look at the bulbs. Go back downstairs and try the second switch. You've not got them all mapped out in about 30 seconds. Using GTP4 I asked if there was a way to do it in less than 3 y…
Variants of common riddles remain the final frontier. I'm trying to cross the river in a canoe with a carrot, cabbage, and cucumber...
Re: Language models still struggle with the concept of negation
#37Re: Language models still struggle with the concept of negation
#38Earlier quoted context omitted.
While the answer landed on bats, ChatGPT kind of trips itself up in its explanations: 1. "Most winged creatures either have paws (like bats) or lay eggs (like birds and insects)" - but bats don't have paws. 2. "However, if we consider the term "wings" more broadly" - why do we need to consider the term "wings" more broadly? Nobody is arguing there is some definition of "wing" that would exclude bats. Bat wings evolve…
> Bats have wings and no paws. Can hind limbs of bats be classified as paws? English is not my native language, so I'm just being curious.
No, not really. Paws kind of implies something that's fairly fleshy and soft and which bats don't have. Claws would be appropriate description of what bats have.
Re: Language models still struggle with the concept of negation
#39The title of the article is kinda the answer at the same time. Chatbots don't know what stuff is. They have no ability to gain knowledge out of learned text, just counting occurences of words in texts and giving them a weight, depending on the relationship in that text. They are just putting combinations of text together.
And the concept of negating something related to something else kinda needs an understanding of the topic at hand.
Re: Language models still struggle with the concept of negation
#40Do you Don't Not Dislike Not Strong Bad?
TF, it hurts my brain.