Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models still struggle with the concept of negation

quantamagazine.org

Post reply on HN
Print view View original

81–90 of 172 posts

Re: Language models still struggle with the concept of negation

#81

Post by mirekrusin »

source99 wrote: post #16

This is one of my favorite test cases: Prompt: Here is a riddle. It is a common riddle but with some changes that make it more difficult. You are an alien that will live for at least 10000 years. You have no sense of temperature. you can not feel hot or cold. you have eyes and can see. you are in a house. downstairs are 3 light switches that control 3 light bulbs that are upstairs. Each light bulb will last for exact…

Arguably LLMs are handicapped by tuning that always expects some answer without questioning the question.

Re: Language models still struggle with the concept of negation

#82

Post by blahedo »

Even from a purely human natural-language-linguistics standpoint, negation is hard! It's often hard to figure out how it distributes ("I could not go" may have the "not" scoping over either the "could" or the "go" depending on inflection and context), and its meaning is often not especially compositional in an obvious way: "red" might mean something like "the set of all red things" and "table" could be "the set of all tables" and so "red table" might be seen as the intersection of the two sets. For the majority of adjectives X, an "X Thing" is also a "Thing" (but not every Thing is an X Thing). [0] But a "Not Thing" is, to a first approximation, the set complement of "Thing". Which is tricky. If you've ever learned a foreign language to at least an intermediate level, you'll know that clauses that involve any but the most basic negation are a magnitude more complex to build and interpret than ones that don't.

Coming at it from the neural network side, I'll also point out that at the simplest level of neural networks, operations like AND and OR and basic NOT are doable, with a single neuron, but the seemingly-simple XOR just can't be done without an extra layer. (The formal description of the problem is that XOR is not "linearly separable", which I will handwave as "not a simple composition of its parts".) This is not precisely the same thing as the negation problem for LLMs, but it feels like it has the same basic flavour.

[0] An interesting exception is certain adjectives like "former", as a "former teacher" is generally not a "teacher", and semantic models that can handle this are more complicated!

Re: Language models still struggle with the concept of negation

#83

Post by samatman »

chaxor wrote: post #67
benreesman wrote: post #66

Earlier quoted context omitted.

And if you don’t force it with the “step-by-step” cheat code? I know what both 3.5-turbo and 4 say on my account, but I think I might be in some cost-effective inference holdout group because it’s always so dumb.

What's 375792*7385482? You can't use paper, or think about it more than a second. If you get it wrong, it's clear that you are not intelligent. See how ridiculous that sounds? CoT or tokens are not 'cheating' - they allow the system to have (as an analogy) a sheet of paper to work out the problem, instead of relying on a single step to jump through thousands of steps of reasoning. It's trained on the algorithmic reas…

The answer is 2775405051744.

I cheated, and used bc, the unix utility.

These three maneuvers (the correct answer, the agency shown by cheating, and the crystallized intelligence of unix) handily demonstrate both my own intelligence, and that of my species.

The LLMs aren't in the same category of being, yet. It's not clear they ever will be; while I think silicon is a viable substrate for intelligent agents, it still strikes me as doubtful they'll be any more descended from the current machine learning systems as they are from Symbolics Lisp.

Re: Language models still struggle with the concept of negation

#84

Post by maxbond »

foobarbecue wrote: post #62
byyyy wrote: post #29

Earlier quoted context omitted.

> but bats don't have paws. Bat paw: https://upload.wikimedia.org/wikipedia/commons/e/e1/Bat_in_a... ChatGPT was totally in tune with this concept. The appendage on the elbow of the wing can be a paw or not a paw but it certainly evolved from paws. The reasoning was spot on and amazingly nuanced.

What? No. You're rationalizing what is clearly an error. Truth still exists, and bats do not have paws, any more than humans do.

(Add a grain of salt because I searched for about 15 minutes here.)

Doing some digging, paws are "soft foot-like parts of a mammal, generally a quadruped, that has claws" (Wikipedia). Bats do appear to have claws, but not really with the same structure as a paw; they don't seem to be mounted above a soft, fleshy pad. Rather, they're at the top of the wing, sitting on bone. They're also very different in function, when I've had the fortune to see a bat they seemed to use the claw to climb in a similar way to how climbers use ice axes.

My impression is that bat's claws are the anatomical equivalent of a thumb, not a hand/paw, and that the entire paw or hand of bat's ancestors evolved into their wings (sort of like if your fingers became very long and webbed and then you were able to fly with them).

So, my conclusion is that no, the claws of a bat are not a paw.

ETA: I neglected to consider their hind claws, but I don't think those are padded either.

https://i.pinimg.com/originals/1e/8d/09/1e8d0923d5caf2dbab68...

That certainly has claws but I wouldn't describe that as being soft or as being similar to a paw. And they're pretty clearly made for hanging, climbing, and grasping prey/food, and not for walking.

Re: Language models still struggle with the concept of negation

#85

Post by maxbond »

livinglist wrote: post #37
marcianx wrote: post #30

Earlier quoted context omitted.

Actually, no species of bats lay eggs - they are mammals, and not the egg-laying kind (platypus or echidna).

I don’t know why I always have assumed bats lay eggs…

I think people think of bats and snakes as being neighboring categories for some reason. Maybe because of old superstitions and such, that they're both servants of evil or something. Maybe that was the source of your misconception.

Re: Language models still struggle with the concept of negation

#86

Post by HPsquared »

diceduckmonk wrote: post #73
hn_throwaway_99 wrote: post #11

Earlier quoted context omitted.

While the answer landed on bats, ChatGPT kind of trips itself up in its explanations: 1. "Most winged creatures either have paws (like bats) or lay eggs (like birds and insects)" - but bats don't have paws. 2. "However, if we consider the term "wings" more broadly" - why do we need to consider the term "wings" more broadly? Nobody is arguing there is some definition of "wing" that would exclude bats. Bat wings evolve…

> "there are some species of bats that give live birth" - all species of bats give live birth ChatGPT is being more logical to be honest. It’s true, some species of bats give live birth. To assume all bats give live birth might be committing the fallacy of induction.

Wiki: "Bats are mammals of the order Chiroptera" ... (funnily enough - chiro = hands, ptera = wings, hand-wings, makes sense)

Chiroptera fits under Mammalia in the phylogenenetric tree.

Re: Language models still struggle with the concept of negation

#87

Post by HPsquared »

Jackson__ wrote: post #76
williamcotton wrote: post #5

What animals don’t have paws or lay eggs, but have wings? Work it out, step by step. ChatGPT (GPT-4): Sure, let's go through this step by step: First, you asked for animals that don't have paws. This would exclude mammals such as dogs, cats, bears, and so on, as these species have paws. Second, you specified animals that don't lay eggs. This eliminates a wide range of animals including all birds, most reptiles, and s…

and here's using bard, arriving at an almost correct conclusion: What animals don’t have paws or lay eggs, but have wings? Work it out, step by step. Here are the steps on how to find animals that don't have paws or lay eggs, but have wings: 1.Define the terms. What do we mean by "paws"? What do we mean by "eggs"? What do we mean by "wings"? 2. Identify animals that meet the criteria. There are a few animals that mee…

I believed it for a minute when it said butterflies don't lay eggs, literally had to do a search.

Re: Language models still struggle with the concept of negation

#88

Post by gitrog »

HPsquared wrote: post #86
diceduckmonk wrote: post #73

Earlier quoted context omitted.

> "there are some species of bats that give live birth" - all species of bats give live birth ChatGPT is being more logical to be honest. It’s true, some species of bats give live birth. To assume all bats give live birth might be committing the fallacy of induction.

Wiki: "Bats are mammals of the order Chiroptera" ... (funnily enough - chiro = hands, ptera = wings, hand-wings, makes sense) Chiroptera fits under Mammalia in the phylogenenetric tree.

Just need to mention that Echidnas and Platypuses are also mammals and they lay eggs.

Re: Language models still struggle with the concept of negation

#89

Post by somewhereoutth »

xcv123 wrote: post #44
u_name wrote: post #39

The title of the article is kinda the answer at the same time. Chatbots don't know what stuff is. They have no ability to gain knowledge out of learned text, just counting occurences of words in texts and giving them a weight, depending on the relationship in that text. They are just putting combinations of text together. And the concept of negating something related to something else kinda needs an understanding of…

> They have no ability to gain knowledge out of learned text, just counting occurences of words in texts and giving them a weight, depending on the relationship in that text. They are just putting combinations of text together. No, they use deep neural networks to build a hierarchical semantic model. They are not simple occurrence counters. Also the current state of the art of LLMs handles negation easily. This artic…

All that can be supplied to a LLM for training is syntax. There is no way to provide semantics, it only understands 'table' in regard to syntax it has already seen including that particular token. It has no experience and therefore no understanding of a real table.

It may internally construct a hierarchy as you set out, but this is and can only be a syntactical hierarchy - though should be no surprise that it corresponds to our usual semantic hierarchy. But whereas our syntax proceeds from our semantics, its syntax proceeds only from our syntax that we've fed it.

Re: Language models still struggle with the concept of negation

#90

Post by HPsquared »

gitrog wrote: post #88
HPsquared wrote: post #86

Earlier quoted context omitted.

Wiki: "Bats are mammals of the order Chiroptera" ... (funnily enough - chiro = hands, ptera = wings, hand-wings, makes sense) Chiroptera fits under Mammalia in the phylogenenetric tree.

Just need to mention that Echidnas and Platypuses are also mammals and they lay eggs.

I should have said bats fit under "Placentalia" - placental mammals.
Post reply on HN