Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models still struggle with the concept of negation

quantamagazine.org

Post reply on HN
Print view View original

151–160 of 172 posts

Re: Language models still struggle with the concept of negation

#151

Post by hn_throwaway_99 »

byyyy wrote: post #111
maxbond wrote: post #84

Earlier quoted context omitted.

(Add a grain of salt because I searched for about 15 minutes here.) Doing some digging, paws are "soft foot-like parts of a mammal, generally a quadruped, that has claws" (Wikipedia). Bats do appear to have claws, but not really with the same structure as a paw; they don't seem to be mounted above a soft, fleshy pad. Rather, they're at the top of the wing, sitting on bone. They're also very different in function, whe…

https://www.vocabulary.com/dictionary/paw As long as it has claws according to vocabulary dot com it is a paw.

[flagged]

Re: Language models still struggle with the concept of negation

#152

Post by maxbond »

hn_throwaway_99 wrote: post #151
byyyy wrote: post #111

Earlier quoted context omitted.

https://www.vocabulary.com/dictionary/paw As long as it has claws according to vocabulary dot com it is a paw.

[flagged]

This subthread is a flamewar that died down. Please don't rekindle it. The point has been made to death, and making it again but more insulting makes it more difficult for people who might disagree to consider the perspective, because they have to filter out their feelings about being insulted - a weighty task - in order to begin considering it, let alone accepting it.

Re: Language models still struggle with the concept of negation

#153

Post by blazespin »

kazinator wrote: post #91

> “Alice is not born in Germany. Is Alice born in Hamburg?” "I don't know" is the right answer. There are over 20 towns named Hamburg around the world, many of them in the USA. However, GPT4 is really struggling with this even if you assure it that by Hamburg, we mean the original one in Germany. GPT4 gets it if you explain that this is a logic problem, and not a query about some actual person named Alice, and if you…

GPT4 struggles with things for sure, but this isn't one of them. You're just projecting what you think should be the right answer. If it worries you that much, try adding "Provide alternative responses to this question. Each response should try to answer using a different approach to solving the problem." to your prompts.

If you want an example of something GPT4 has a hard time with, try this:

there is no 'e' in the color red, right? Yes, that is correct. The word "red" does not contain the letter 'e'.

Re: Language models still struggle with the concept of negation

#154

Post by chaxor »

pmoriarty wrote: post #101
chaxor wrote: post #72

Earlier quoted context omitted.

It's a bit bizarre (but also very intriguing!) that you would bring up some of the older techniques of NLP, such as TFIDF, Latent Semantic indexing, GloVe, etc (at least that's what I assume you mean when you mention the severely outdated cooccurence type of models) when these clearly don't use any of that. Transformers have been hyped like crazy lately due to all of these advances, so why being up cooccurence unless…

"Transformers are GNNs (somewhat trivial ones, as they are sheaf NNs, but nonetheless) and GNNs are dynamic programmers, which has been shown via category theory (Velolickovic etc al)." In which paper was this demonstrated?

It's literally the title. Peter Velockovic is probably the most prolific author of this era regarding LLMs, and does not yet realize it. https://arxiv.org/pdf/2203.15544.pdf

Re: Language models still struggle with the concept of negation

#155

Post by chaxor »

samatman wrote: post #83
chaxor wrote: post #67

Earlier quoted context omitted.

What's 375792*7385482? You can't use paper, or think about it more than a second. If you get it wrong, it's clear that you are not intelligent. See how ridiculous that sounds? CoT or tokens are not 'cheating' - they allow the system to have (as an analogy) a sheet of paper to work out the problem, instead of relying on a single step to jump through thousands of steps of reasoning. It's trained on the algorithmic reas…

The answer is 2775405051744. I cheated, and used bc, the unix utility. These three maneuvers (the correct answer, the agency shown by cheating, and the crystallized intelligence of unix) handily demonstrate both my own intelligence, and that of my species. The LLMs aren't in the same category of being, yet. It's not clear they ever will be; while I think silicon is a viable substrate for intelligent agents, it still…

Are you sure? Have you read through enough literature on GNNs? Transformers are GNNs with attention message passing and positional embeddings. GNNs have also been *mathematically proven * to be dynamic programmers and to *align with algorithmic reasoning*. So tell me again how LLMs aren't capable of performing reasoning to be intelligent.

Re: Language models still struggle with the concept of negation

#156

Post by byyyy »

hn_throwaway_99 wrote: post #151
byyyy wrote: post #111

Earlier quoted context omitted.

https://www.vocabulary.com/dictionary/paw As long as it has claws according to vocabulary dot com it is a paw.

[flagged]

I'm ok with your language. I found the "competent in English" rather immature and juvenile, but it's not a big deal and really minor.

I also understand that you might not realize how heated this sub thread was as the other guy deleted half his posts and flagged one of mine (therefore possibly making it invisible to you). So adding a little heat to something that looks tame isn't a big deal. I won't continue the flame war but I will continue the discussion.

It's not my interpretation. It's about the interpretation by Merriam Webster. I believe not only is it your interpretation that is incorrect when compared to official sources, but you didn't fully read the official definition either.

The eagle is not a quadruped. The definition in Merriam Webster says it has to be on a quadruped for it to be a paw. Hence your example is completely irrelevant. Please correct your mistake.

Re: Language models still struggle with the concept of negation

#157

Post by chaxor »

benreesman wrote: post #74
chaxor wrote: post #67

Earlier quoted context omitted.

What's 375792*7385482? You can't use paper, or think about it more than a second. If you get it wrong, it's clear that you are not intelligent. See how ridiculous that sounds? CoT or tokens are not 'cheating' - they allow the system to have (as an analogy) a sheet of paper to work out the problem, instead of relying on a single step to jump through thousands of steps of reasoning. It's trained on the algorithmic reas…

I know I’m bad at multiplying six and seven digit integers in my head, and I don’t care at all if that makes me more or less intelligent (whatever the fuck that means) than a wrist watch. What does that possibly have to do with GPT-4 shitting the bed if you draw the logic puzzle out of a deck of cards instead of cherry-picking and hinting it?

My point is that there are already mathematical proofs in the literature that show that LLMs literally are doing algorithmic reasoning. This has been proven by category theory, in the area of GNNs (for which transformers are a trivial subset). Sorry if I'm going over too much basics, as this is obvious and well understood by anyone actually in the field, to the point of being excessively boring.

Re: Language models still struggle with the concept of negation

#158

Post by maxbond »

byyyy wrote: post #156
hn_throwaway_99 wrote: post #151

Earlier quoted context omitted.

[flagged]

I'm ok with your language. I found the "competent in English" rather immature and juvenile, but it's not a big deal and really minor. I also understand that you might not realize how heated this sub thread was as the other guy deleted half his posts and flagged one of mine (therefore possibly making it invisible to you). So adding a little heat to something that looks tame isn't a big deal. I won't continue the flame…

I'm presuming you're new here since your account is new & I seem to detect some misunderstandings about how HN works. If I presume too much, my apologies.

Are you aware of the "showdead" function in your settings? I don't think they deleted any comments, they were flagged. If you turn on showdead, you'll be able to see them again. (You can't delete a comment someone's responded to, or after 2 hours.)

I don't know if that other commenter flagged your post, but I don't think it matters. I didn't flag any posts here, but both of you guys were getting flagged. In situations like that, I try to interpret the flags as a request from the overall community not to have that sort of discussion on the forum. The sort of argument you were engaged with (and I recognize this isn't fully on you here, it takes two to tango) is against the guidelines of the site and generally frowned upon by the community, so there's no reason to suspect this other commenter was flagging you. At least 3 people were flagging both of your posts.

I understand that you feel pointed language is appropriate in a debate, and that HM'd guidelines here are patronizing. That's all well and good. The objection of the community isn't a moralizing one. It's a practical one; the issue is that flamewars displace the type of discussion this community is seeking to have and erode the good will on which the forum turns.

(And I do agree that bats crawling on the ground is quadrupedalism, for what it's worth. I'm not sure if that makes them a quadruped or not since that isn't their primary move of locomotion, indeed my understanding is that it's a vulnerable position for them and that it's quite deadly [since they can't evade predators].)

Re: Language models still struggle with the concept of negation

#159

Post by kazinator »

blazespin wrote: post #153
kazinator wrote: post #91

> “Alice is not born in Germany. Is Alice born in Hamburg?” "I don't know" is the right answer. There are over 20 towns named Hamburg around the world, many of them in the USA. However, GPT4 is really struggling with this even if you assure it that by Hamburg, we mean the original one in Germany. GPT4 gets it if you explain that this is a logic problem, and not a query about some actual person named Alice, and if you…

GPT4 struggles with things for sure, but this isn't one of them. You're just projecting what you think should be the right answer. If it worries you that much, try adding "Provide alternative responses to this question. Each response should try to answer using a different approach to solving the problem." to your prompts. If you want an example of something GPT4 has a hard time with, try this: there is no 'e' in the…

I don't see what you're talking about. In this situation, there's exactly one correct answer. There's no projecting or other psychobabble involved.

Re: Language models still struggle with the concept of negation

#160

Post by petepete »

School-Cotton wrote: post #135
petepete wrote: post #132

Earlier quoted context omitted.

It does. "I couldn't care less" is a common term and "I could care less" is a misunderstanding. I'd hope more people would realise they're saying they _do care_ about something they're claiming to not care about. It sounds stupid to a native English speaker.

You’re just repeating the same point over and over rather than actually giving any argument. > It sounds stupid to a native English speaker You might be a native English speaker, but you’re not a native speaker of American English.

> not a native speaker of American English

I could care less.

Post reply on HN