Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bag of words, have mercy on us

experimental-history.com

Post reply on HN
Print view View original

341–350 of 362 posts

Re: Bag of words, have mercy on us

#341

Post by fc417fc802 »

emp17344 wrote: post #321
throw4847285 wrote: post #255

Earlier quoted context omitted.

Then what are non-human animals doing?

Excellent point. See also the failure of Sapir-Whorf to prove that language determines thought. I think we have plenty of evidence that, while language can influence thought, it is not thought itself. Many people invested in AI are happy to throw out decades of linguistic evidence that language and thought are separate.

I was with you up to the last sentence. By what reasoning do you claim that LLMs only consist of words? The input and output are words but all the stuff in the middle - where the magic happens - does not appear to be quite that simple.

Re: Bag of words, have mercy on us

#342

Post by fc417fc802 »

jibal wrote: post #335
panarky wrote: post #152

Earlier quoted context omitted.

When someone says "AIs aren't really thinking" because AIs don't think like people do, what I hear is "Airplanes aren't really flying" because airplanes don't fly like birds do.

That's a fallacy of denial of the antecedent. You are inferring from the fact that airplanes really fly that AIs really think, but it's not a logically valid inference.

Observing a common (potential) failure mode is not equivalent to asserting a logical inference. It is only a fallacy if you "P, therefore C" which GP is not (at least to my eye) doing.

Re: Bag of words, have mercy on us

#343

Post by lostmsu »

fc417fc802 wrote: post #339
lostmsu wrote: post #293

Earlier quoted context omitted.

> But raising kids, I can clearly see that intelligence isn't just solved by LLMs Funny, I have the opposite experience. Like early LLMs kids tend to give specific answers to the questions they don't understand or don't really know or remember the answer to. Kids also loop (give the same reply repeatedly to different prompts), enter highly emotional states where their output is garbled (everyone loves that one), etc.…

Same failure modes, but not a general solution to intelligence.

What makes you think so?

Re: Bag of words, have mercy on us

#344

Post by stonogo »

XorNot wrote: post #151
stonogo wrote: post #71

Earlier quoted context omitted.

Beta was superior in everything but run length, and it lost because it was more expensive than VHS without being sufficiently superior to justify the cost.

Yes but that's not a minor issue: Beta's run length was so limited that it couldn't record or hold a full length movie or even a long TV show.

Only for the first year or two of release. Early VHS capacities were better, but by the late 70s or early 80s there wasn't a meaningful difference.

Re: Bag of words, have mercy on us

#345

Post by d-lisp »

balamatom wrote: post #325
d-lisp wrote: post #218

Earlier quoted context omitted.

I doubt words are involved when we e.g. solve a mathematical problem. To me, solving problems happens in a logico/aesthetical space which may be the same as when you are intellectually affected by a work of art. I don't remember myself being able to translate directly into words what I feel for a great movie or piece of music, even if in the late I can translate this "complex mental entity" into words, exactly like I…

My wordmangling and mathsolving happen in that sort of logico/aesthetical space, too! The twist about words in particular is they are distinctly articulable symbols, i.e. you can sound 'em out - and thus, presumably, have a reasonable expectation for bearers of the same language to comprehend if not what you meant then at least some vaguely predictable meaning-cloud associated with the given speech act. That's unlike…

What is your point exactly in regard to what I said earlier, how would you rephrase what you just said as a philosophical/epistemological statement ?

> It's extremely common

I cannot conceive this ? I am lacking the empirical knowledge you seem to have. (I don't understand your "archetypical case", I can't relate to it). I'd love a reexplanation of your point here, as your intent is unclear to me.

I didn't understand also the "introspection modulators" part :(, (a well known practice ?? I must be living on another planet haha...).

edit: or maybe that's a metaphor for "language" ??

Re: Bag of words, have mercy on us

#346

Post by fc417fc802 »

lostmsu wrote: post #343
fc417fc802 wrote: post #339

Earlier quoted context omitted.

Same failure modes, but not a general solution to intelligence.

What makes you think so?

The lack of results so far. The observable behavior exhibits many uncanny similarities but also clearly has missing pieces.

As the person you initially responded to said, observing children growing up should make it obvious.

Or if we shift to stating the obvious, there's the minor detail that the vast majority of architectures lack the ability to learn during inference. That's one of the basic things that biological systems are capable of.

Re: Bag of words, have mercy on us

#347

Post by atomicthumbs »

lostmsu wrote: post #297
meheleventyone wrote: post #171

Earlier quoted context omitted.

This really shows how imprecise a term 'thinking' is here. In this sense any predictive probabilistic blackbox model could be termed 'thinking'. Particularly when juxtaposed against something as concrete as flight that we have modelled extremely accurately.

Yes, to a degree. A very low degree at that.

AIs think like a rock flies.

Re: Bag of words, have mercy on us

#348

Post by b112 »

danielbln wrote: post #155
b112 wrote: post #119

Earlier quoted context omitted.

When you see a dog, or describe the entity, do you discuss the genetic makeup or the bone structure? No, you describe the bark. The end result is what counts. Training or not, it's just spewing predictive, relational text.

So do we, but that's helpful.

" Training or not, it's just spewing predictive, relational text."

If you're responding to that, "so do we" is not accurate.

We're not spewing predictive, relational text. We're communicating, after thought, and the output is meant to communicate something specifically.

With AI, it's not trying to communicate an idea. It's just spewing predictive text. There's no thought to it. At all.

Re: Bag of words, have mercy on us

#349

Post by ptidhomme »

marcosdumay wrote: post #340
ptidhomme wrote: post #310

Earlier quoted context omitted.

When you ask an LLM a question about cars, it needs an inner representation of what a car is (how imperfect it may be) to answer your question. A model of "language" as you want to define it would output a grammatically correct wall of text that goes nowhere.

A map of how concepts relate in language is not a model of the world, except on the extremely limited sense that languages are part or the world. And yeah, that wasn't clear before people created those machines that can speak but can't think. But it should be completely obvious to anybody that interacts with them for a small while.

"How concepts relate" is called a model. That it uses language to be interacted with is irrelevant to the fact that it's a model of of a worldly concept.

What of multi modal models according to you ? Are they "models of eyesight", "models of sound", or pixels or wavelengths... C'mon.

Re: Bag of words, have mercy on us

#350

Post by habinero »

ACCount37 wrote: post #185
habinero wrote: post #177

Earlier quoted context omitted.

Yeah. I have a half-cynical/half-serious pet theory that a decent fraction of humanity has a broken theory of mind and thinks everyone has the same thought patterns they do. If it talks like me, it thinks like me. Whenever the comment section takes a long hit and goes "but what is thinking, really " I get slightly more cynical about it lol

Why not? By now, it's pretty clear that LLMs implement abstract thinking - as do humans. They don't think exactly like humans do - but they sure copy a lot of human thinking, and end up closer to it than just about anything that's not a human.

It isn't clear because they do none of that lol. They don't think.

It can kinda sorta look like thinking if you don't have a critical eye, but it really doesn't take much to break the illusion.

I really don't get this obsessive need to pretend your tools are alive. Y'all know when you watch YouTube that it's a trick and the tiny people on your screen don't live in your computer, right?

Post reply on HN