Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models still struggle with the concept of negation

quantamagazine.org

Post reply on HN
Print view View original

111–120 of 172 posts

Re: Language models still struggle with the concept of negation

#111

Post by byyyy »

maxbond wrote: post #84
foobarbecue wrote: post #62

Earlier quoted context omitted.

What? No. You're rationalizing what is clearly an error. Truth still exists, and bats do not have paws, any more than humans do.

(Add a grain of salt because I searched for about 15 minutes here.) Doing some digging, paws are "soft foot-like parts of a mammal, generally a quadruped, that has claws" (Wikipedia). Bats do appear to have claws, but not really with the same structure as a paw; they don't seem to be mounted above a soft, fleshy pad. Rather, they're at the top of the wing, sitting on bone. They're also very different in function, whe…

https://www.vocabulary.com/dictionary/paw

As long as it has claws according to vocabulary dot com it is a paw.

Re: Language models still struggle with the concept of negation

#112

Post by williamcotton »

benreesman wrote: post #66
williamcotton wrote: post #5

What animals don’t have paws or lay eggs, but have wings? Work it out, step by step. ChatGPT (GPT-4): Sure, let's go through this step by step: First, you asked for animals that don't have paws. This would exclude mammals such as dogs, cats, bears, and so on, as these species have paws. Second, you specified animals that don't lay eggs. This eliminates a wide range of animals including all birds, most reptiles, and s…

And if you don’t force it with the “step-by-step” cheat code? I know what both 3.5-turbo and 4 say on my account, but I think I might be in some cost-effective inference holdout group because it’s always so dumb.

Ok, so calling CoT a cheat code is quite loaded, is it not?

You might get more use out of these tools if you learn how to use them and these natural language instructions are how you use these tools.

It’s like calling the reverse gear on your car a cheat code!

Re: Language models still struggle with the concept of negation

#113

Post by [deleted] »

replygirl wrote: post #23
hn_throwaway_99 wrote: post #11

Earlier quoted context omitted.

While the answer landed on bats, ChatGPT kind of trips itself up in its explanations: 1. "Most winged creatures either have paws (like bats) or lay eggs (like birds and insects)" - but bats don't have paws. 2. "However, if we consider the term "wings" more broadly" - why do we need to consider the term "wings" more broadly? Nobody is arguing there is some definition of "wing" that would exclude bats. Bat wings evolve…

there are some google results suggesting bats have front paws, some using "paws" colloquially and some suggesting their pads can qualify as paws. these results are given authority because they show up in the ostensibly fact-based q&a section, which, in conflict with actual taxonomical literature, may be enough to indicate to gpt that it's necessary to define terms

[deleted]

Re: Language models still struggle with the concept of negation

#114

Post by famouswaffles »

nathan_compton wrote: post #105
famouswaffles wrote: post #43

Earlier quoted context omitted.

Variants of common riddles actually can be solved with GPT-4, but you have to rewrite it so it doesn't look like the riddle from memory(sometimes, it's as easy as changing names to something completely different). Turns out Language models trust their memory quite a bit. Slightly related - they won't actually use the results of tools if it differs a lot from what it expects the output to be - https://vgel.me/posts/to…

"Language models trust their memory quite a bit." All they have is memory, either in the weights or the input prompt. To the extent that these models appear to reason, it is precisely in the ability to successfully substitute information from the prompt into reasoning patterns in the training data. It shouldn't be any surprise that this fails when patterns in the prompt strongly condition the model to reproduce parti…

LLMs trained on code reason better. Perform better on reasoning benchmarks even if the benchmarks have nothing to do with code. You're wrong.

https://arxiv.org/abs/2210.07128

Re: Language models still struggle with the concept of negation

#115

Post by r-w »

throwawaymaths wrote: post #98
samatman wrote: post #83

Earlier quoted context omitted.

The answer is 2775405051744. I cheated, and used bc, the unix utility. These three maneuvers (the correct answer, the agency shown by cheating, and the crystallized intelligence of unix) handily demonstrate both my own intelligence, and that of my species. The LLMs aren't in the same category of being, yet. It's not clear they ever will be; while I think silicon is a viable substrate for intelligent agents, it still…

> It's not clear they ever will be; It seems to me that the ability to reason ( though primitive it may still be) is an emergent property of sufficiently large differentiable hash/lookup tables (and if you think about the mechanism of reasoning, that seems... Reasonable). According to "folks in the know", the scaling limits of GPT architecture seem to be nigh, but if it were a bit higher I wouldn't be surprised if LL…

I think you’re conflating reasoning with inference. And in any case, GP wasn’t using reasoning, but rather resourcefulness/invention. I’m not sure an LLM can pull that off beyond coming up with vague, derivative fiction.

Re: Language models still struggle with the concept of negation

#116

Post by williamcotton »

QuadrupleA wrote: post #47
williamcotton wrote: post #19

Earlier quoted context omitted.

There's a sample-and-poll approach that I've used with factual question-and-answer based tools that can help with this kind of thing as well. First, you'll notice the added "work it out step by step" to encourage what is called chain-of-thought in the literature. Second, generate say, five responses from the LLM, then take all five responses and uses them in another prompt where you ask the LLM to reflect on all of t…

Then, if that's not accurate, take that entire process and repeat that 5 times, looking for the most common answer.

And 5 of those, and the best answer…

But it does seem like a single call to an LLM is not the correct approach.

I’ve gotten the best results from attempting to mirror how I would operate in response to a prompt. Much of this is colored by decades of philosophical readings, which in my case means a decent amount of analytical philosophy along with latter Wittgenstein and other post linguistic turn thinkers.

So when I break things down, either in some kind of approach inspired by Frege, Wittgenstein or unadorned self-reflection, I can see that the unconscious mind seems to be doing something recursive before my conscious mind is ready to start fumbling over the word.

It seems like an LLM is just part of a larger cognitive architecture that will use these metacognitive approaches.

It’s also very obvious that mammalian brains are much more efficient than GPUs.

Re: Language models still struggle with the concept of negation

#117

Post by foobarbecue »

byyyy wrote: post #111
maxbond wrote: post #84

Earlier quoted context omitted.

(Add a grain of salt because I searched for about 15 minutes here.) Doing some digging, paws are "soft foot-like parts of a mammal, generally a quadruped, that has claws" (Wikipedia). Bats do appear to have claws, but not really with the same structure as a paw; they don't seem to be mounted above a soft, fleshy pad. Rather, they're at the top of the wing, sitting on bone. They're also very different in function, whe…

https://www.vocabulary.com/dictionary/paw As long as it has claws according to vocabulary dot com it is a paw.

[flagged]

Re: Language models still struggle with the concept of negation

#118

Post by foobarbecue »

maxbond wrote: post #84
foobarbecue wrote: post #62

Earlier quoted context omitted.

What? No. You're rationalizing what is clearly an error. Truth still exists, and bats do not have paws, any more than humans do.

(Add a grain of salt because I searched for about 15 minutes here.) Doing some digging, paws are "soft foot-like parts of a mammal, generally a quadruped, that has claws" (Wikipedia). Bats do appear to have claws, but not really with the same structure as a paw; they don't seem to be mounted above a soft, fleshy pad. Rather, they're at the top of the wing, sitting on bone. They're also very different in function, whe…

I think we're getting totally taken for a ride... byyyy can't actually be serious right...?

Anyway I'm enjoying imagining bats with little paws, it's an amusing image.

Re: Language models still struggle with the concept of negation

#120

Post by williamcotton »

oh_sigh wrote: post #60
marcianx wrote: post #30

Earlier quoted context omitted.

Actually, no species of bats lay eggs - they are mammals, and not the egg-laying kind (platypus or echidna).

Sorry, yes, I know this, I was making fun of a computer's output.

I commend your dry, unadorned humor!
Post reply on HN