Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models still struggle with the concept of negation

quantamagazine.org

Post reply on HN
Print view View original

141–150 of 172 posts

Re: Language models still struggle with the concept of negation

#141

Post by byyyy »

foobarbecue wrote: post #139
maxbond wrote: post #128

Earlier quoted context omitted.

Well, if you're looking for my opinion, I don't think they're trolling. But I do think both of you were (intentionally or unintentionally) goading each other from a disagreement to an argument with your rhetoric (eg, where you said, "let's assume that's a reasonable source", or when they said "you obviously sourced this it just didn't match your point so you discarded it" - that brings up the temperature of the conve…

[flagged]

Bats are quadrupeds. When they walk they walk like a pterodactyl with the elbow of the wing. They don't stand on their hind legs. See the vampire bat. Additionally the "broad" definition from Merriam Websters covers it.

These aren't pedantic details we are just nitpicking on either. ChatGPT's response considered the nuance the definition encompasses, which to keep on topic is thoroughly impressive and relevant to the overall conversation.

Off-topic:

The person you queried mentioned "intentional goading." Which is what you're continuing to do with that Calvin and Hobbes link. I think this, the flag and accusing me of trolling is taking it too far. A little minor goading is ok during a debate, (I actually don't completely agree with the absolutist politeness policies of HN). While I returned the goad (which I shouldn't have), ultimately I didn't really have a problem with it. I think, now though, it has escalated now past the point of no return. I'll be exiting this thread because of this. Farewell.

Re: Language models still struggle with the concept of negation

#142

Post by byyyy »

maxbond wrote: post #128
foobarbecue wrote: post #118

Earlier quoted context omitted.

I think we're getting totally taken for a ride... byyyy can't actually be serious right...? Anyway I'm enjoying imagining bats with little paws, it's an amusing image.

Well, if you're looking for my opinion, I don't think they're trolling. But I do think both of you were (intentionally or unintentionally) goading each other from a disagreement to an argument with your rhetoric (eg, where you said, "let's assume that's a reasonable source", or when they said "you obviously sourced this it just didn't match your point so you discarded it" - that brings up the temperature of the conve…

It does bring up the temp, I apologize for including you in that. I appreciate your reply.

Re: Language models still struggle with the concept of negation

#143

Post by usaar333 »

kazinator wrote: post #91

> “Alice is not born in Germany. Is Alice born in Hamburg?” "I don't know" is the right answer. There are over 20 towns named Hamburg around the world, many of them in the USA. However, GPT4 is really struggling with this even if you assure it that by Hamburg, we mean the original one in Germany. GPT4 gets it if you explain that this is a logic problem, and not a query about some actual person named Alice, and if you…

hmm, worked here:

> No, Alice was not born in Hamburg. If Hamburg is a town in Germany, and Alice was not born in Germany, then it logically follows that Alice was not born in Hamburg.

Re: Language models still struggle with the concept of negation

#144

Post by nathan_compton »

famouswaffles wrote: post #136
nathan_compton wrote: post #134

Earlier quoted context omitted.

Code is often just a sequence of steps (sometimes with comments to indicate goals). As such, it is just another form of patterns of reasoning. Many chains of thought that you would utilize in code are useful skeletons to think about other things. I don't see how this undermines my point.

If code transfers to reasoning tasks that don't have anything to do with code then what is being "substituted" ? Ideas and concepts ? Code and MMLU don't share similar "reasoning patterns" unless you're being extremely vague. In the, "they both require reasoning" sense.

Structure. GPT has seen lots of logical constructions/arguments for things. These are either explicitly in code (in documentation) or are implicitly in code (code is often a linear sequence of steps building to, for example, a return value). ChatGPT learns patterns like this. A prompt may condition the generator to produce something like one of these patterns with elements from the prompt substituted into the generated text. This works relatively often, but fails exactly in the case where the prompt so strongly indicates a pattern that won't work for the prompt given.

I won't say these models can't reason per se, but they can only reason using their memories and the prompt. There is nothing else for them to compute on.

In a hand wavy kind of way, when ChatGPT fails at a riddle phrased in a way as to make it seem similar to a common riddle, you're seeing overfitting. But given the quantity of data these models consume, its hard to imagine how to test for overfitting because the training data contains things similar to almost anything you can imagine. Because of that I'm still very suspicious of claims that they "reason" in any strong sense of the word.

But if you try very hard you can find "held out" data and when you test on it, GPT4 stops looking so smart:

https://teddit.net/r/singularity/comments/121tc48/gpt4_fails...

That said, I've been very impressed by GPT4 as a productivity tool.

Re: Language models still struggle with the concept of negation

#145

Post by iacomus »

Jackson__ wrote: post #76
williamcotton wrote: post #5

What animals don’t have paws or lay eggs, but have wings? Work it out, step by step. ChatGPT (GPT-4): Sure, let's go through this step by step: First, you asked for animals that don't have paws. This would exclude mammals such as dogs, cats, bears, and so on, as these species have paws. Second, you specified animals that don't lay eggs. This eliminates a wide range of animals including all birds, most reptiles, and s…

and here's using bard, arriving at an almost correct conclusion: What animals don’t have paws or lay eggs, but have wings? Work it out, step by step. Here are the steps on how to find animals that don't have paws or lay eggs, but have wings: 1.Define the terms. What do we mean by "paws"? What do we mean by "eggs"? What do we mean by "wings"? 2. Identify animals that meet the criteria. There are a few animals that mee…

The response I got from Bard:

The only animals that fit the description of having wings, no paws, and no eggs are bats. Bats are mammals, which means they give birth to live young. They also have wings, which are made of skin stretched between their elongated fingers. Bats do not have paws, as their feet are adapted for grasping branches and other objects.

Here is a step-by-step process for determining which animals fit the description:

1. Identify all animals that have wings. 2. Eliminate all animals that lay eggs. 3. Eliminate all animals that have paws. 4. The remaining animals are bats.

Here is a list of animals that have wings, no paws, and no eggs:

- Bats

It is important to note that there are some animals that may appear to fit the description, but do not. For example, some insects have wings, but they do not have paws and they lay eggs. Additionally, some birds have wings, but they do have paws and they lay eggs.

Re: Language models still struggle with the concept of negation

#146

Post by famouswaffles »

nathan_compton wrote: post #144
famouswaffles wrote: post #136

Earlier quoted context omitted.

If code transfers to reasoning tasks that don't have anything to do with code then what is being "substituted" ? Ideas and concepts ? Code and MMLU don't share similar "reasoning patterns" unless you're being extremely vague. In the, "they both require reasoning" sense.

Structure. GPT has seen lots of logical constructions/arguments for things. These are either explicitly in code (in documentation) or are implicitly in code (code is often a linear sequence of steps building to, for example, a return value). ChatGPT learns patterns like this. A prompt may condition the generator to produce something like one of these patterns with elements from the prompt substituted into the generat…

>but they can only reason using their memories and the prompt.

Eh no.

https://arxiv.org/abs/2212.10559

>But if you try very hard you can find "held out" data and when you test on it, GPT4 stops looking so smart:

This can be done to anybody. This can be done to you. It's not a gotcha. Nobody is saying GPTs don't/can't memorize.

Re: Language models still struggle with the concept of negation

#147

Post by xcv123 »

somewhereoutth wrote: post #100
xcv123 wrote: post #96

Earlier quoted context omitted.

That's a philosophical issue, not technical. No one is saying these models are conscious or have human awareness of concepts. It mechanically builds a deeply layered semantic model that correlates to our human understanding. Quibbling over whether it is "real semantics" or not is just ironically quibbling over semantics. Yes its not conscious, but it doesn't need to be. It is possible to build a mechanical structure…

The distinction between semantics and syntax is pretty tight, no philosophy required. The former considers the domain being represented, whereas the latter is strictly the symbols used in the representation. So to be precise it mechanically builds a deeply layered syntactic model. LLMs just regurgitate syntax, any semantics can only be imagined by us and overlaid on the syntactic results produced.

You are disagreeing philosophically about what constitutes "true" semantics. That is not a technical argument.

If you are right about this, then you should edit or delete this wikipedia article and publish a paper to inform all NLP researchers that there is no such thing as a semantic similarity metric because NLP models cannot understand "true" semantics.

https://en.wikipedia.org/wiki/Semantic_similarity

Re: Language models still struggle with the concept of negation

#148

Post by gregsadetsky »

IanCal wrote: post #68
gregsadetsky wrote: post #8

This reminded me of https://github.com/elsamuko/Shirt-without-Stripes Whose days may be soon over as gpt4 gets this overall right..? As noted by many folks here

Gpt4 + klarna plugin does that just fine. Gpt4 decided to search for plain shirts.

Just tried GPT4 + Klarna and got 4 suggestions that were all along the lines of "adidas Women's Adicolor Classics 3-Stripes T-shirt - Black" ... :-)

I'll open an issue...

Re: Language models still struggle with the concept of negation

#149

Post by [deleted] »

famouswaffles wrote: post #146
nathan_compton wrote: post #144

Earlier quoted context omitted.

Structure. GPT has seen lots of logical constructions/arguments for things. These are either explicitly in code (in documentation) or are implicitly in code (code is often a linear sequence of steps building to, for example, a return value). ChatGPT learns patterns like this. A prompt may condition the generator to produce something like one of these patterns with elements from the prompt substituted into the generat…

>but they can only reason using their memories and the prompt. Eh no. https://arxiv.org/abs/2212.10559 >But if you try very hard you can find "held out" data and when you test on it, GPT4 stops looking so smart: This can be done to anybody. This can be done to you. It's not a gotcha. Nobody is saying GPTs don't/can't memorize.

[deleted]

Re: Language models still struggle with the concept of negation

#150

Post by twic »

livinglist wrote: post #37
marcianx wrote: post #30

Earlier quoted context omitted.

Actually, no species of bats lay eggs - they are mammals, and not the egg-laying kind (platypus or echidna).

I don’t know why I always have assumed bats lay eggs…

Have you considered getting a job as a large language model?
Post reply on HN