Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bag of words, have mercy on us

experimental-history.com

Post reply on HN
Print view View original

211–220 of 362 posts

Re: Bag of words, have mercy on us

#211

Post by ben_w »

littlestymaar wrote: post #201
ben_w wrote: post #162

Earlier quoted context omitted.

The general argument you make is correct, but you conclusion "And this one doesn't." is as yet uncertain. I will absolutely say that all ML methods known are literally too stupid to live, as in no living thing can get away with making so many mistakes before it's learned anything, but that's the rate of change of performance with respect to examples rather than what it learns by the time training is finished. What is…

> but that's the rate of change of performance with respect to examples rather than what it learns by the time training is finished. It's not just that. The problem of “deep learning” is that we use the word “learning” for something that really has no similarity with actual learning: it's not just that it converges way too slowly, it's also that it just seeks to minimize the predicted loss for every samples during tr…

> If you feed it enough flat-earther content, as well a physics books, an LLM will happily tells you that the earth is flat, and explain you with lots of physics why it cannot be flat.

This is a terrible example, because it's what humans do as well. See religious, or indeed military, indoctrination. All propaganda is as effective as it is, because the same message keeps getting hammered in.

And not just that, common misconceptions abound everywhere and not just conspiracy theories, religion, and politics. My dad absolutely insisted that the water draining in toilets or sinks are meaningfully influenced by the Coriolis effect, used an example of one time he went to the equator and saw a demonstration of this on both sides of the equator. University education and lifetime career in STEM, should have been able to figure out from first principles why the Coriolis effect is exactly zero on the equator itself, didn't.

> A human will learn one or the other first, and once the initial learning is made, it will disregards all the evidence of the contrary, until maybe at some point it doesn't and switches side entirely.

We don't have any way to know what a human would do if they could read the entire internet, because we don't live long enough to try.

The only bet I'd make is that we'd be more competent than any AI doing the same, because we learn faster from fewer examples, but that's about it.

> LLMs don't have an inner representation of the world and as such they don't have an opinion about the world.

There is evidence that they do have some inner representation of the world, e.g.:

• https://arxiv.org/abs/2506.02996

• https://arxiv.org/abs/2404.18202

Re: Bag of words, have mercy on us

#212

Post by Kim_Bruning »

Yhippa wrote: post #72
voidhorse wrote: post #47

Earlier quoted context omitted.

When you have a thought, are you "predicting the next thing"—can you confidently classify all mental activity that you experience as "predicting the next thing"? Language and society constrains the way we use words, but when you speak, are you "predicting"? Science allows human beings to predict various outcomes with varying degrees of success, but much of our experience of the world does not entail predicting things…

Boo LLM-generated comments!

But what [if the llms generate] constructive and helpful comments?

https://xkcd.com/810/

Re: Bag of words, have mercy on us

#213

Post by Timwi »

SequoiaHope wrote: post #159
FarmerPotato wrote: post #83

Earlier quoted context omitted.

One metaphor is to call the model a person, another metaphor is to call it a pile of words. These are quite opposite. I think that's the whole point. Person-metaphor does nothing to explain its behavior, either. "Bag of words" has a deep origin in English, the Anglo-Saxon kenning "word-hord", as when Beowulf addresses the Danish sea-scout (line 258) "He unlocked his word-hoard and delivered this answer." So, bag of w…

Word-hoard is a very good phrase.

It's an everyday word in German: Wortschatz, meaning (someone's) active vocabulary.

Re: Bag of words, have mercy on us

#214

Post by Kim_Bruning »

thesz wrote: post #192
akoboldfrying wrote: post #112

Earlier quoted context omitted.

LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others? If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.

> Human brains aren't magic, special or different. DNA inside neurons uses superconductive quantum computations [1]. [1] https://www.nature.com/articles/s41598-024-62539-5 As the result, all living cells with DNA emit coherent (as in lasers) light [2]. There is a theory that this light also facilitates intercellular communication. [2] https://www.sciencealert.com/we-emit-a-visible-light-that-va... Chemical structures…

All three appear to be technically correct, but are (normally) only incidental to the operation of neurons as neurons. We know this because we can test what aspects of neurons actually lead to practical real world effects. Neurophysiology is not a particularly obscure or occult field, so there are many many papers and textbooks on the topic.(And there's a large subset you can test on yourself, besides, though I wouldn't recommend patch-clamping!)

Re: Bag of words, have mercy on us

#216

Post by carb »

A4ET8a8uTh0_v2 wrote: post #199
balamatom wrote: post #190

Earlier quoted context omitted.

I'm definitely a stream of words. My "abstract thoughts" are a stream of words too, they just don't get sounded out. Tbf I'd rather they weren't there in the first place. But bodies which refuse to harbor an "interiority" are fast-tracked to destruction because they can't suf^W^W^W be productive. Funny movie scene from somewhere. The sergeant is drilling the troops: "You, private! What do you live for!", and expects…

Hmm, seems unlikely. They are not sounded out part is true, sure, but I question whether 'abstract thoughts' can be so easily dismissed as mere words. edit: come to think of it and I am asking this for a reason: do you hear your abstract thoughts?

Different people have different levels of internal monologuing or none at all. I don't generally think with words in sentences in my head, but many people I know do.

Re: Bag of words, have mercy on us

#217

Post by Kim_Bruning »

jrowen wrote: post #124
Kim_Bruning wrote: post #113

Earlier quoted context omitted.

Chinese room has been discussed to death of course. Here's one fun approach (out of 100s) : What if we answer the Chinese room with the Systems Reply [1]? Searle countered the systems reply by saying he would internalize the Chinese room. But at that point it's pretty much exactly the Cartesian theater[2] : with room, homunculus, implement. But the Cartesian theater is disproven, because we've cut open brains and the…

It just seemed like relevant background that the author might not have been aware of, adjacent and substantial enough to warrant a mention. I think there is some validity to the Cartesian theater, in that the whole of the experience that we perceive with our senses is at best an interpretation of a projection or subset of "reality."

Oh right, and, if you're interested, there were quite a number of interesting discussion on the chinese room on HN back when John Searle died!

https://news.ycombinator.com/item?id=45563627

Re: Bag of words, have mercy on us

#218

Post by d-lisp »

balamatom wrote: post #190
RayVR wrote: post #70

Earlier quoted context omitted.

Are you a stream of words or are your words the “simplistic” projection of your abstract thoughts? I don’t at all discount the importance of language in so many things, but the question that matters is whether statistical models of language can ever “learn” abstract thought, or become part of a system which uses them as a tool. My personal assessment is that LLMs can do neither.

I'm definitely a stream of words. My "abstract thoughts" are a stream of words too, they just don't get sounded out. Tbf I'd rather they weren't there in the first place. But bodies which refuse to harbor an "interiority" are fast-tracked to destruction because they can't suf^W^W^W be productive. Funny movie scene from somewhere. The sergeant is drilling the troops: "You, private! What do you live for!", and expects…

I doubt words are involved when we e.g. solve a mathematical problem.

To me, solving problems happens in a logico/aesthetical space which may be the same as when you are intellectually affected by a work of art. I don't remember myself being able to translate directly into words what I feel for a great movie or piece of music, even if in the late I can translate this "complex mental entity" into words, exactly like I can tell to someone how we need to change the architecture of a program in order to solve something after having looked up and right for a few seconds.

It seems to me that we have an inner system that is much faster than language, that creates entities that can then beslowly and sometimes painfully translated to language.

I do note that I'm not sure about any of the previous statements though'

Re: Bag of words, have mercy on us

#219

Post by littlestymaar »

ben_w wrote: post #211
littlestymaar wrote: post #201

Earlier quoted context omitted.

> but that's the rate of change of performance with respect to examples rather than what it learns by the time training is finished. It's not just that. The problem of “deep learning” is that we use the word “learning” for something that really has no similarity with actual learning: it's not just that it converges way too slowly, it's also that it just seeks to minimize the predicted loss for every samples during tr…

> If you feed it enough flat-earther content, as well a physics books, an LLM will happily tells you that the earth is flat, and explain you with lots of physics why it cannot be flat. This is a terrible example, because it's what humans do as well. See religious, or indeed military, indoctrination. All propaganda is as effective as it is, because the same message keeps getting hammered in. And not just that, common…

> This is a terrible example, because it's what humans do as well. See religious, or indeed military, indoctrination. All propaganda is as effective as it is, because the same message keeps getting hammered in.

You completely misread my point.

The key thing with humans isn't that they cannot believe in bullshit. They can definitely do. But we don't usually believe in both the bullshit and in the fact the BS is actually BS. We have opinions on the BS. And we, as a species, routinely die or kill for these opinions, by the way. LLM don't care about anything.

Re: Bag of words, have mercy on us

#220

Post by nottorp »

bloaf wrote: post #41

Everyone is out here acting like "predicting the next thing" is somehow fundamentally irrelevant to "human thinking" and it is simply not the case. What does it mean to say that we humans act with intent? It means that we have some expectation or prediction about how our actions will effect the next thing, and choose our actions based on how much we like that effect. The ability to predict is fundamental to our abili…

This is the "but LLMs will get better, trust me" thread?
Post reply on HN