Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models still struggle with the concept of negation

quantamagazine.org

Post reply on HN
Print view View original

121–130 of 172 posts

Re: Language models still struggle with the concept of negation

#121

Post by byyyy »

foobarbecue wrote: post #118
maxbond wrote: post #84

Earlier quoted context omitted.

(Add a grain of salt because I searched for about 15 minutes here.) Doing some digging, paws are "soft foot-like parts of a mammal, generally a quadruped, that has claws" (Wikipedia). Bats do appear to have claws, but not really with the same structure as a paw; they don't seem to be mounted above a soft, fleshy pad. Rather, they're at the top of the wing, sitting on bone. They're also very different in function, whe…

I think we're getting totally taken for a ride... byyyy can't actually be serious right...? Anyway I'm enjoying imagining bats with little paws, it's an amusing image.

I'm serious.

https://www.merriam-webster.com/dictionary/paw

Look up the definition of a paw. I think you have a sort of mixed up definition of what it is. Possibly a different definition used among your "bat expert" friends.

And I quote (again):

   Paw: the foot of a quadruped (such as a lion or dog) that has claws
   broadly : the foot of an animal
Both the broad and technical definition of paw fits what the bat has. Bats have paws.

The thing you are referring to is likely are these soft pads that are often on the paws, sometimes called paw pads.

Re: Language models still struggle with the concept of negation

#123

Post by byyyy »

foobarbecue wrote: post #122
byyyy wrote: post #119

Earlier quoted context omitted.

[flagged]

[flagged]

Birds don't have paws because they aren't quadrupeds according to websters.

Bats walk on fours. They are quadrupeds therefore they have paws according to Merriam Webster.

Why don't you address the point I brought up? I already completely understand your definition no need to reiterate it. However, there is a clear disconnect between your definition of paw and the definition from Merriam Webster. Please address it.

Re: Language models still struggle with the concept of negation

#124

Post by washadjeffmad »

jiggawatts wrote: post #58
QuadrupleA wrote: post #47

Earlier quoted context omitted.

Then, if that's not accurate, take that entire process and repeat that 5 times, looking for the most common answer.

And then make sure to train GPT to help you trade stocks, because you’ll need those millions to pay your OpenAI bill.

Don't not do that, but also, a 30B with 70-80% of the reasoning of the latest GPT can be run on a home PC.

Given that most people have computers and RAM is dirt cheap right now, I'm already maxing out all of my devices.

Re: Language models still struggle with the concept of negation

#125

Post by byyyy »

byyyy wrote: post #121
foobarbecue wrote: post #118

Earlier quoted context omitted.

I think we're getting totally taken for a ride... byyyy can't actually be serious right...? Anyway I'm enjoying imagining bats with little paws, it's an amusing image.

I'm serious. https://www.merriam-webster.com/dictionary/paw Look up the definition of a paw. I think you have a sort of mixed up definition of what it is. Possibly a different definition used among your "bat expert" friends. And I quote (again): Paw: the foot of a quadruped (such as a lion or dog) that has claws broadly : the foot of an animal Both the broad and technical definition of paw fits what the bat has. Bats…

Also, I don't know if you noticed.

But this nuance in the definition of the word paw was addressed by chatGPT.

Re: Language models still struggle with the concept of negation

#126

Post by mercer »

samatman wrote: post #83
chaxor wrote: post #67

Earlier quoted context omitted.

What's 375792*7385482? You can't use paper, or think about it more than a second. If you get it wrong, it's clear that you are not intelligent. See how ridiculous that sounds? CoT or tokens are not 'cheating' - they allow the system to have (as an analogy) a sheet of paper to work out the problem, instead of relying on a single step to jump through thousands of steps of reasoning. It's trained on the algorithmic reas…

The answer is 2775405051744. I cheated, and used bc, the unix utility. These three maneuvers (the correct answer, the agency shown by cheating, and the crystallized intelligence of unix) handily demonstrate both my own intelligence, and that of my species. The LLMs aren't in the same category of being, yet. It's not clear they ever will be; while I think silicon is a viable substrate for intelligent agents, it still…

with a bit of glue code I can allow gpt4 to use those tools. and decide to do so. cheating is part of 'intelligence'.

Re: Language models still struggle with the concept of negation

#127

Post by somewhereoutth »

pmoriarty wrote: post #104
somewhereoutth wrote: post #100

Earlier quoted context omitted.

The distinction between semantics and syntax is pretty tight, no philosophy required. The former considers the domain being represented, whereas the latter is strictly the symbols used in the representation. So to be precise it mechanically builds a deeply layered syntactic model. LLMs just regurgitate syntax, any semantics can only be imagined by us and overlaid on the syntactic results produced.

Does a definition of a word in a dictionary provide the syntax or the semantics of the word it defined?

In common use by humans, it provides semantics, in terms of other previously understood semantics. However, if there is no semantic understanding by the reader, then all it provides is a syntactic rewrite rule for each word.

Re: Language models still struggle with the concept of negation

#128

Post by maxbond »

foobarbecue wrote: post #118
maxbond wrote: post #84

Earlier quoted context omitted.

(Add a grain of salt because I searched for about 15 minutes here.) Doing some digging, paws are "soft foot-like parts of a mammal, generally a quadruped, that has claws" (Wikipedia). Bats do appear to have claws, but not really with the same structure as a paw; they don't seem to be mounted above a soft, fleshy pad. Rather, they're at the top of the wing, sitting on bone. They're also very different in function, whe…

I think we're getting totally taken for a ride... byyyy can't actually be serious right...? Anyway I'm enjoying imagining bats with little paws, it's an amusing image.

Well, if you're looking for my opinion, I don't think they're trolling. But I do think both of you were (intentionally or unintentionally) goading each other from a disagreement to an argument with your rhetoric (eg, where you said, "let's assume that's a reasonable source", or when they said "you obviously sourced this it just didn't match your point so you discarded it" - that brings up the temperature of the conversation).

Re: Language models still struggle with the concept of negation

#129

Post by bluepoint »

mirekrusin wrote: post #81
source99 wrote: post #16

This is one of my favorite test cases: Prompt: Here is a riddle. It is a common riddle but with some changes that make it more difficult. You are an alien that will live for at least 10000 years. You have no sense of temperature. you can not feel hot or cold. you have eyes and can see. you are in a house. downstairs are 3 light switches that control 3 light bulbs that are upstairs. Each light bulb will last for exact…

Arguably LLMs are handicapped by tuning that always expects some answer without questioning the question.

Actually, this is a really good point. Asking a question means that you have an internal model or the world and you realize that there are some data that is way off. Is it even possible at all to train an LLM to ask a question? This also gives me an idea for the future where most online text is generated by AI: we will play the game of responding questions with questions and then we know we are human!

Re: Language models still struggle with the concept of negation

#130

Post by throwawaymaths »

r-w wrote: post #115
throwawaymaths wrote: post #98

Earlier quoted context omitted.

> It's not clear they ever will be; It seems to me that the ability to reason ( though primitive it may still be) is an emergent property of sufficiently large differentiable hash/lookup tables (and if you think about the mechanism of reasoning, that seems... Reasonable). According to "folks in the know", the scaling limits of GPT architecture seem to be nigh, but if it were a bit higher I wouldn't be surprised if LL…

I think you’re conflating reasoning with inference. And in any case, GP wasn’t using reasoning, but rather resourcefulness/invention. I’m not sure an LLM can pull that off beyond coming up with vague, derivative fiction.

Gp said "state of being" so I don't think the either of us can know for sure what was meant by that.
Post reply on HN