Language models still struggle with the concept of negation • Hacker News Board

Re: Language models still struggle with the concept of negation

#101

Post by pmoriarty » Sat, May 13, 2023, 11:28 AM UTC

chaxor wrote: post #72

u_name wrote: post #39

The title of the article is kinda the answer at the same time. Chatbots don't know what stuff is. They have no ability to gain knowledge out of learned text, just counting occurences of words in texts and giving them a weight, depending on the relationship in that text. They are just putting combinations of text together. And the concept of negating something related to something else kinda needs an understanding of…

It's a bit bizarre (but also very intriguing!) that you would bring up some of the older techniques of NLP, such as TFIDF, Latent Semantic indexing, GloVe, etc (at least that's what I assume you mean when you mention the severely outdated cooccurence type of models) when these clearly don't use any of that. Transformers have been hyped like crazy lately due to all of these advances, so why being up cooccurence unless…

"Transformers are GNNs (somewhat trivial ones, as they are sheaf NNs, but nonetheless) and GNNs are dynamic programmers, which has been shown via category theory (Velolickovic etc al)."

In which paper was this demonstrated?

Re: Language models still struggle with the concept of negation

#102

Post by hef19898 » Sat, May 13, 2023, 11:30 AM UTC

Jackson__ wrote: post #76

williamcotton wrote: post #5

What animals don’t have paws or lay eggs, but have wings? Work it out, step by step. ChatGPT (GPT-4): Sure, let's go through this step by step: First, you asked for animals that don't have paws. This would exclude mammals such as dogs, cats, bears, and so on, as these species have paws. Second, you specified animals that don't lay eggs. This eliminates a wide range of animals including all birds, most reptiles, and s…

and here's using bard, arriving at an almost correct conclusion: What animals don’t have paws or lay eggs, but have wings? Work it out, step by step. Here are the steps on how to find animals that don't have paws or lay eggs, but have wings: 1.Define the terms. What do we mean by "paws"? What do we mean by "eggs"? What do we mean by "wings"? 2. Identify animals that meet the criteria. There are a few animals that mee…

Regarding point 1., was Bard trained on J. Peaterson speaches or what?

Re: Language models still struggle with the concept of negation

#103

Post by School-Cotton » Sat, May 13, 2023, 11:31 AM UTC

petepete wrote: post #99

School-Cotton wrote: post #55

Earlier quoted context omitted.

That’s a stock phrase in colloquial American English. Using it isn’t really evidence that someone “struggles with the concept of negation”.

An incorrect stock phrase that means the exact opposite of what's been said. I don't think it's unreasonable to say people who get things entirely wrong are struggling.

The whole point of stock phrases, idioms, expressions, etc. is that their meaning is “non-compositional”; i.e., they mean something different from the literal combination of words that make them up. Including even extreme cases where they can mean the opposite, like this one. That doesn’t mean they’re “wrong”, it’s just a quirk of human language.

Re: Language models still struggle with the concept of negation

#104

Post by pmoriarty » Sat, May 13, 2023, 11:32 AM UTC

somewhereoutth wrote: post #100

xcv123 wrote: post #96

Earlier quoted context omitted.

That's a philosophical issue, not technical. No one is saying these models are conscious or have human awareness of concepts. It mechanically builds a deeply layered semantic model that correlates to our human understanding. Quibbling over whether it is "real semantics" or not is just ironically quibbling over semantics. Yes its not conscious, but it doesn't need to be. It is possible to build a mechanical structure…

The distinction between semantics and syntax is pretty tight, no philosophy required. The former considers the domain being represented, whereas the latter is strictly the symbols used in the representation. So to be precise it mechanically builds a deeply layered syntactic model. LLMs just regurgitate syntax, any semantics can only be imagined by us and overlaid on the syntactic results produced.

Does a definition of a word in a dictionary provide the syntax or the semantics of the word it defined?

Re: Language models still struggle with the concept of negation

#105

Post by nathan_compton » Sat, May 13, 2023, 11:38 AM UTC

famouswaffles wrote: post #43

tedunangst wrote: post #36

Earlier quoted context omitted.

Variants of common riddles remain the final frontier. I'm trying to cross the river in a canoe with a carrot, cabbage, and cucumber...

Variants of common riddles actually can be solved with GPT-4, but you have to rewrite it so it doesn't look like the riddle from memory(sometimes, it's as easy as changing names to something completely different). Turns out Language models trust their memory quite a bit. Slightly related - they won't actually use the results of tools if it differs a lot from what it expects the output to be - https://vgel.me/posts/to…

"Language models trust their memory quite a bit."

All they have is memory, either in the weights or the input prompt. To the extent that these models appear to reason, it is precisely in the ability to successfully substitute information from the prompt into reasoning patterns in the training data. It shouldn't be any surprise that this fails when patterns in the prompt strongly condition the model to reproduce particular patterns of reasoning (eg, many words in the riddle indicate a well known riddle, but the details are different).

I know the impulse to anthropomorphize is almost impossibly seductive, but I find that the best way to understand and use these models is to remember: they are giant conditional probability distributions for the next token.

Re: Language models still struggle with the concept of negation

#106

Post by nathan_compton » Sat, May 13, 2023, 11:40 AM UTC

lakis wrote: post #46

source99 wrote: post #16

This is one of my favorite test cases: Prompt: Here is a riddle. It is a common riddle but with some changes that make it more difficult. You are an alien that will live for at least 10000 years. You have no sense of temperature. you can not feel hot or cold. you have eyes and can see. you are in a house. downstairs are 3 light switches that control 3 light bulbs that are upstairs. Each light bulb will last for exact…

The reply is funny. I told ChatGPT that it can do multiple trips and that it can see both switches and light bulbs at the same time. ChatGPT keeps insisting that you have to turn on a switch, wait a few minutes , flip switch to off, turn other switch to on, then go and look and feel the switches. The classic solution to the riddle. But if you can see both switches and light bulbs, you turn on one switch, you see whic…

"feel the switches"?

Re: Language models still struggle with the concept of negation

#107

Post by belter » Sat, May 13, 2023, 11:51 AM UTC

samatman wrote: post #83

chaxor wrote: post #67

Earlier quoted context omitted.

What's 375792*7385482? You can't use paper, or think about it more than a second. If you get it wrong, it's clear that you are not intelligent. See how ridiculous that sounds? CoT or tokens are not 'cheating' - they allow the system to have (as an analogy) a sheet of paper to work out the problem, instead of relying on a single step to jump through thousands of steps of reasoning. It's trained on the algorithmic reas…

The answer is 2775405051744. I cheated, and used bc, the unix utility. These three maneuvers (the correct answer, the agency shown by cheating, and the crystallized intelligence of unix) handily demonstrate both my own intelligence, and that of my species. The LLMs aren't in the same category of being, yet. It's not clear they ever will be; while I think silicon is a viable substrate for intelligent agents, it still…

Until an AI decides to take control of a robot, physically sequestrate a human, and under the threat of physical pain make the human perform the tasks the AI can't...

Re: Language models still struggle with the concept of negation

#108

Post by williamcotton » Sat, May 13, 2023, 12:21 PM UTC

jiggawatts wrote: post #58

QuadrupleA wrote: post #47

Earlier quoted context omitted.

Then, if that's not accurate, take that entire process and repeat that 5 times, looking for the most common answer.

And then make sure to train GPT to help you trade stocks, because you’ll need those millions to pay your OpenAI bill.

Too true… I can’t think of a time prior to this where I was spending $200/month researching the use of a new tool.

Re: Language models still struggle with the concept of negation

#109

Post by [deleted] » Sat, May 13, 2023, 12:26 PM UTC

nathan_compton wrote: post #105

famouswaffles wrote: post #43

Earlier quoted context omitted.

Variants of common riddles actually can be solved with GPT-4, but you have to rewrite it so it doesn't look like the riddle from memory(sometimes, it's as easy as changing names to something completely different). Turns out Language models trust their memory quite a bit. Slightly related - they won't actually use the results of tools if it differs a lot from what it expects the output to be - https://vgel.me/posts/to…

"Language models trust their memory quite a bit." All they have is memory, either in the weights or the input prompt. To the extent that these models appear to reason, it is precisely in the ability to successfully substitute information from the prompt into reasoning patterns in the training data. It shouldn't be any surprise that this fails when patterns in the prompt strongly condition the model to reproduce parti…

[deleted]

Re: Language models still struggle with the concept of negation

#110

Post by williamcotton » Sat, May 13, 2023, 12:28 PM UTC

hammyhavoc wrote: post #70

jiggawatts wrote: post #10

Earlier quoted context omitted.

Interesting that it contradicted itself by saying bats have paws and don’t have paws, but arrived at the correct conclusion anyway.

"A broken clock is right twice per day" as the saying goes.

That aphorism doesn’t apply here.

What is meant by that saying is, “well the first time we ran the machine it worked just fine, so we got all excited, but then it consistently failed after that”, with the lesson being, increase your sample size before you pass judgement.

LLMs almost suffer from the opposite. Because they are correct so often the danger is in assuming the response is always correct.

Maybe, “even a Swiss made watch needs to be checked against the town bells”?