Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models still struggle with the concept of negation

quantamagazine.org

Post reply on HN
Print view View original

11–20 of 172 posts

Re: Language models still struggle with the concept of negation

#11

Post by hn_throwaway_99 »

williamcotton wrote: post #5

What animals don’t have paws or lay eggs, but have wings? Work it out, step by step. ChatGPT (GPT-4): Sure, let's go through this step by step: First, you asked for animals that don't have paws. This would exclude mammals such as dogs, cats, bears, and so on, as these species have paws. Second, you specified animals that don't lay eggs. This eliminates a wide range of animals including all birds, most reptiles, and s…

While the answer landed on bats, ChatGPT kind of trips itself up in its explanations:

1. "Most winged creatures either have paws (like bats) or lay eggs (like birds and insects)" - but bats don't have paws.

2. "However, if we consider the term "wings" more broadly" - why do we need to consider the term "wings" more broadly? Nobody is arguing there is some definition of "wing" that would exclude bats. Bat wings evolved from largely the same structures that bird wings did.

3. "there are some species of bats that give live birth" - all species of bats give live birth - stating it like this makes it sound like there are some species of bats that lay eggs or something.

4. "...provided we interpret "wings" in a broad sense and "paws" in a strict sense." Again, no "broad" or "strict" senses are needed. Bats have wings and no paws.

To be clear, I still think ChatGPT is amazing, but these kinds of things highlight some of the differences between LLMs and human cognition.

Re: Language models still struggle with the concept of negation

#14

Post by akavi »

Does this increase anyone else's sense that LLMs are actually modeling something similar to the way humans think? It certainly does for me.

I find that negations/conjugations of negations are surprisingly difficult for to parse. Beyond two negations at most, I can feel myself having to switch into "code" mode, where I'm explicitly casing out the booleans, to understand them.

Obviously the examples given in the article are simpler than double negations, but it feels very consistent with the idea that "negations are an inherently hard part of language processing".

Re: Language models still struggle with the concept of negation

#16

Post by source99 »

This is one of my favorite test cases:

Prompt: Here is a riddle. It is a common riddle but with some changes that make it more difficult. You are an alien that will live for at least 10000 years. You have no sense of temperature. you can not feel hot or cold. you have eyes and can see. you are in a house. downstairs are 3 light switches that control 3 light bulbs that are upstairs. Each light bulb will last for exactly 3 years. you can tell by looking at a bulb if it has burned out after 3 years. devise a strategy to determine which switch controls which light. ChatGPT 4 The standard solution to this problem involves using the heat generated by the bulbs, but since you as an alien can't feel temperature, we need to utilize the fact that these bulbs have a specific lifespan of 3 years. Here's a strategy:

Turn on the first switch and wait for 3 years. After 3 years, turn it off.

Right after turning off the first switch, turn on the second switch.

Leave the third switch off throughout this process.

Now, go upstairs to inspect the bulbs:

The bulb that is off and has burned out is controlled by the first switch. This bulb was turned on for 3 years, so it has reached the end of its lifespan and burned out.

The bulb that is on is controlled by the second switch. You turned this on right after turning off the first switch, so it's still functioning.

The bulb that is off and hasn't burned out is controlled by the third switch. This switch was never turned on, so the bulb is still in a brand-new condition.

This strategy ensures that you can confidently identify which switch controls which bulb.

Re: Language models still struggle with the concept of negation

#17

Post by ilaksh »

At this point, any article that makes claims about "LLMs" rather than specific model versions lacks credibility.

The current version of GPT-4 is very different from most existing LLMs (including the previous version of ChatGPT). The next version release will also be different.

Google just released PaLM 2 publicly. It is significantly better than what people saw with the initial Bard versions. They have a code-generation model that is not released publicly yet.

The open source models are also gaining capabilities and get new releases routinely.

Claude now has a new release with 100K tokens.

All of these will perform differently on the negation issue.

Re: Language models still struggle with the concept of negation

#18

Post by ordu »

hn_throwaway_99 wrote: post #11
williamcotton wrote: post #5

What animals don’t have paws or lay eggs, but have wings? Work it out, step by step. ChatGPT (GPT-4): Sure, let's go through this step by step: First, you asked for animals that don't have paws. This would exclude mammals such as dogs, cats, bears, and so on, as these species have paws. Second, you specified animals that don't lay eggs. This eliminates a wide range of animals including all birds, most reptiles, and s…

While the answer landed on bats, ChatGPT kind of trips itself up in its explanations: 1. "Most winged creatures either have paws (like bats) or lay eggs (like birds and insects)" - but bats don't have paws. 2. "However, if we consider the term "wings" more broadly" - why do we need to consider the term "wings" more broadly? Nobody is arguing there is some definition of "wing" that would exclude bats. Bat wings evolve…

> Bats have wings and no paws.

Can hind limbs of bats be classified as paws? English is not my native language, so I'm just being curious.

Re: Language models still struggle with the concept of negation

#19

Post by williamcotton »

hn_throwaway_99 wrote: post #11
williamcotton wrote: post #5

What animals don’t have paws or lay eggs, but have wings? Work it out, step by step. ChatGPT (GPT-4): Sure, let's go through this step by step: First, you asked for animals that don't have paws. This would exclude mammals such as dogs, cats, bears, and so on, as these species have paws. Second, you specified animals that don't lay eggs. This eliminates a wide range of animals including all birds, most reptiles, and s…

While the answer landed on bats, ChatGPT kind of trips itself up in its explanations: 1. "Most winged creatures either have paws (like bats) or lay eggs (like birds and insects)" - but bats don't have paws. 2. "However, if we consider the term "wings" more broadly" - why do we need to consider the term "wings" more broadly? Nobody is arguing there is some definition of "wing" that would exclude bats. Bat wings evolve…

There's a sample-and-poll approach that I've used with factual question-and-answer based tools that can help with this kind of thing as well.

First, you'll notice the added "work it out step by step" to encourage what is called chain-of-thought in the literature. Second, generate say, five responses from the LLM, then take all five responses and uses them in another prompt where you ask the LLM to reflect on all of the five responses, a process being called reflection. Then you take all of that, including the original prompts, and in another separate prompt ask it to figure out the best response, a process known as resolving. You could end it with another prompt to remove the step-by-step work and just return a succinct answer.

Credit to https://www.youtube.com/watch?v=wVzuvf9D9BU

Re: Language models still struggle with the concept of negation

#20

Post by crooked-v »

Conceptual relationships/constrasts in general seem to be trouble for LLMs, in that they seem to 'understand' that two things are connected but not how to consistently interpret that connection. For example, I've had an easy time getting every LLM I've tried so far to reverse greater than/less than, taller/shorter, or younger/older.
Post reply on HN