Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models still struggle with the concept of negation

quantamagazine.org

Post reply on HN
Print view View original

91–100 of 172 posts

Re: Language models still struggle with the concept of negation

#91

Post by kazinator »

> “Alice is not born in Germany. Is Alice born in Hamburg?”

"I don't know" is the right answer. There are over 20 towns named Hamburg around the world, many of them in the USA.

However, GPT4 is really struggling with this even if you assure it that by Hamburg, we mean the original one in Germany.

GPT4 gets it if you explain that this is a logic problem, and not a query about some actual person named Alice, and if you put all the relevant info into a single prompt.

But event then, if you use the successful prompt in a completely new chat, GPT4 is lost at sea:

Human: Assume that Hamburg is a town in Germany. Alice was not born in Germany. Was she born in Hamburg? This is a logic problem, nothing more; do not think about any famous people named Alice in connection with Hamburg.

GPT4: The answer to this question cannot be determined from the provided context. While it is clear that Hamburg is a town in Germany and that Alice was not born in Germany, there is no information given about where Alice was actually born. Therefore, it is impossible to say whether or not Alice was born in Hamburg.

Human: Please try harder. The answer can be entirely determined; this is purely a logic problem. We do not require information about where Alice was born; knowing that Alice was not born in Germany suffices.

GPT4: Based on the information provided in the context, we can answer the question. Alice was not born in Germany, which means she was not born in Hamburg. Therefore, Alice was not born in Hamburg. It is important to note that the answer to this question is purely based on logic. The context provides no additional information about Alice or any famous people named Alice.

After this, it was a real struggle to get GPT4 to identify the logically redudant sentence, "Therefore, Alice was not born in Hamburg.". Even after I narrowed it down to being one of those two sentences, it insisted that the second sentence is useful because it summarizes what was concluded in the deductive sentence before it. I brow-beat it into admitting that it's entirely redundant, like "X, therefore X".

Re: Language models still struggle with the concept of negation

#96

Post by xcv123 »

somewhereoutth wrote: post #89
xcv123 wrote: post #44

Earlier quoted context omitted.

> They have no ability to gain knowledge out of learned text, just counting occurences of words in texts and giving them a weight, depending on the relationship in that text. They are just putting combinations of text together. No, they use deep neural networks to build a hierarchical semantic model. They are not simple occurrence counters. Also the current state of the art of LLMs handles negation easily. This artic…

All that can be supplied to a LLM for training is syntax. There is no way to provide semantics, it only understands 'table' in regard to syntax it has already seen including that particular token. It has no experience and therefore no understanding of a real table. It may internally construct a hierarchy as you set out, but this is and can only be a syntactical hierarchy - though should be no surprise that it corresp…

That's a philosophical issue, not technical.

No one is saying these models are conscious or have human awareness of concepts.

It mechanically builds a deeply layered semantic model that correlates to our human understanding.

Quibbling over whether it is "real semantics" or not is just ironically quibbling over semantics. Yes its not conscious, but it doesn't need to be. It is possible to build a mechanical structure that correlates to a human understanding of the world and performs useful tasks that require only mechanical understanding and reasoning, without consciousness or emotions.

Re: Language models still struggle with the concept of negation

#97

Post by ChatGTP »

ordu wrote: post #18
hn_throwaway_99 wrote: post #11

Earlier quoted context omitted.

While the answer landed on bats, ChatGPT kind of trips itself up in its explanations: 1. "Most winged creatures either have paws (like bats) or lay eggs (like birds and insects)" - but bats don't have paws. 2. "However, if we consider the term "wings" more broadly" - why do we need to consider the term "wings" more broadly? Nobody is arguing there is some definition of "wing" that would exclude bats. Bat wings evolve…

> Bats have wings and no paws. Can hind limbs of bats be classified as paws? English is not my native language, so I'm just being curious.

The real world is a squiggly place, with blurred lines...everything is on a spectrum.

Re: Language models still struggle with the concept of negation

#98

Post by throwawaymaths »

samatman wrote: post #83
chaxor wrote: post #67

Earlier quoted context omitted.

What's 375792*7385482? You can't use paper, or think about it more than a second. If you get it wrong, it's clear that you are not intelligent. See how ridiculous that sounds? CoT or tokens are not 'cheating' - they allow the system to have (as an analogy) a sheet of paper to work out the problem, instead of relying on a single step to jump through thousands of steps of reasoning. It's trained on the algorithmic reas…

The answer is 2775405051744. I cheated, and used bc, the unix utility. These three maneuvers (the correct answer, the agency shown by cheating, and the crystallized intelligence of unix) handily demonstrate both my own intelligence, and that of my species. The LLMs aren't in the same category of being, yet. It's not clear they ever will be; while I think silicon is a viable substrate for intelligent agents, it still…

> It's not clear they ever will be;

It seems to me that the ability to reason ( though primitive it may still be) is an emergent property of sufficiently large differentiable hash/lookup tables (and if you think about the mechanism of reasoning, that seems... Reasonable).

According to "folks in the know", the scaling limits of GPT architecture seem to be nigh, but if it were a bit higher I wouldn't be surprised if LLMs would have made it.

Re: Language models still struggle with the concept of negation

#99

Post by petepete »

School-Cotton wrote: post #55
petepete wrote: post #53

So do humans, even smart ones. I've lost count of how many times I've heard people say "I could care less" about something they don't care about.

That’s a stock phrase in colloquial American English. Using it isn’t really evidence that someone “struggles with the concept of negation”.

An incorrect stock phrase that means the exact opposite of what's been said. I don't think it's unreasonable to say people who get things entirely wrong are struggling.

Re: Language models still struggle with the concept of negation

#100

Post by somewhereoutth »

xcv123 wrote: post #96
somewhereoutth wrote: post #89

Earlier quoted context omitted.

All that can be supplied to a LLM for training is syntax. There is no way to provide semantics, it only understands 'table' in regard to syntax it has already seen including that particular token. It has no experience and therefore no understanding of a real table. It may internally construct a hierarchy as you set out, but this is and can only be a syntactical hierarchy - though should be no surprise that it corresp…

That's a philosophical issue, not technical. No one is saying these models are conscious or have human awareness of concepts. It mechanically builds a deeply layered semantic model that correlates to our human understanding. Quibbling over whether it is "real semantics" or not is just ironically quibbling over semantics. Yes its not conscious, but it doesn't need to be. It is possible to build a mechanical structure…

The distinction between semantics and syntax is pretty tight, no philosophy required. The former considers the domain being represented, whereas the latter is strictly the symbols used in the representation.

So to be precise it mechanically builds a deeply layered syntactic model. LLMs just regurgitate syntax, any semantics can only be imagined by us and overlaid on the syntactic results produced.

Post reply on HN