Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bag of words, have mercy on us

experimental-history.com

Post reply on HN
Print view View original

81–90 of 362 posts

Re: Bag of words, have mercy on us

#81

Post by akersten »

viccis wrote: post #3

Every day I see people treat gen AI like a thinking human, Dijkstra's attitudes about anthropomorphizing computers is vindicated even more. That said, I think the author's use of "bag of words" here is a mistake. Not only does it have a real meaning in a similar area as LLMs, but I don't think the metaphor explains anything. Gen AI tricks laypeople into treating its token inferences as "thinking" because it is traine…

Bag of words is actually the perfect metaphor. The data structure is a bag. The output is a word. The selection strategy is opaquely undefined.

> Gen AI tricks laypeople into treating its token inferences as "thinking" because it is trained to replicate the semiotic appearance of doing so. A "bag of words" doesn't sufficiently explain this behavior.

Something about there being significant overlap between the smartest bears and the dumbest humans. Sorry you[0] were fooled by the magic bag.

[0] in the "not you, the layperson in question" sense

Re: Bag of words, have mercy on us

#82

Post by tibbar »

The problem with these metaphors is that they don't really explain anything. LLMs can solve countless problems today that we would have previously said were impossible because there are not enough examples in the training data. (EG, novel IMO/ICPC problems.) One way that we move the goal posts is to increase the level of abstraction: IMO/ICPC problems are just math problems, right? There are tons of those in the data set!

But the truth is there has been a major semantic shift. Previously LLMs could only solve puzzles whose answers were literally in the training data. It could answer a math puzzle it had seen before, but if you rephrased it only slightly it could no longer answer.

But now, LLMs can solve puzzles where, like, it has seen a certain strategy before. The newest IMO and ICPC problems were only "in the training data" for a very, very abstract definition of training data.

The goal posts will likely have to shift again, because the next target is training LLMs to independently perform longer chunks of economically useful work, interfacing with all the same tools that white-collar employees do. It's all LLM slop til it isn't, same as the IMO or Putnam exam.

And then we'll have people saying that "white collar employment was all in the training data anyway, if you think about it," at which point the metaphor will have become officially useless.

Re: Bag of words, have mercy on us

#83

Post by FarmerPotato »

viccis wrote: post #3

Every day I see people treat gen AI like a thinking human, Dijkstra's attitudes about anthropomorphizing computers is vindicated even more. That said, I think the author's use of "bag of words" here is a mistake. Not only does it have a real meaning in a similar area as LLMs, but I don't think the metaphor explains anything. Gen AI tricks laypeople into treating its token inferences as "thinking" because it is traine…

One metaphor is to call the model a person, another metaphor is to call it a pile of words. These are quite opposite. I think that's the whole point.

Person-metaphor does nothing to explain its behavior, either.

"Bag of words" has a deep origin in English, the Anglo-Saxon kenning "word-hord", as when Beowulf addresses the Danish sea-scout (line 258)

"He unlocked his word-hoard and delivered this answer."

So, bag of words, word-treasury, was already a metaphor for what makes a person a clever speaker.

Re: Bag of words, have mercy on us

#84

Post by MyOutfitIsVague »

coppsilgold wrote: post #74

Is a brain not a token prediction machine? Tokens in form of neural impulses go in, tokens in the form of neural impulses go out. We would like to believe that there is something profound happening inside and we call that consciousness. Unfortunately when reading about split-brain patient experiments or agenesis of the corpus callosum cases I feel like we are all deceived, every moment of every day. I came to realiza…

Could an LLM trained on nothing and looped upon itself eventually develop language, more complex concepts, and everything else, based on nothing? If you loop LLMs on each other, training them so they "learn" over time, will they eventually form and develop new concepts, cultures, and languages organically over time? I don't have an answer to that question, but I strongly doubt it.

There's clearly more going on in the human mind than just token prediction.

Re: Bag of words, have mercy on us

#85

Post by FarmerPotato »

akersten wrote: post #81
viccis wrote: post #3

Every day I see people treat gen AI like a thinking human, Dijkstra's attitudes about anthropomorphizing computers is vindicated even more. That said, I think the author's use of "bag of words" here is a mistake. Not only does it have a real meaning in a similar area as LLMs, but I don't think the metaphor explains anything. Gen AI tricks laypeople into treating its token inferences as "thinking" because it is traine…

Bag of words is actually the perfect metaphor. The data structure is a bag. The output is a word. The selection strategy is opaquely undefined. > Gen AI tricks laypeople into treating its token inferences as "thinking" because it is trained to replicate the semiotic appearance of doing so. A "bag of words" doesn't sufficiently explain this behavior. Something about there being significant overlap between the smartest…

lol magic bag.

Re: Bag of words, have mercy on us

#86

Post by MyOutfitIsVague »

djoldman wrote: post #58

As a consequence of my profession, I understand how LLMs work under the hood. I also know that we data and tech folks will probably never win the battle over anthropomorphization. The average user of AI, nevermind folks who should know better, is so easily convinced that AI "knows," "thinks," "lies," "wants," "understands," etc. Add to this that all AI hosts push this perspective (and why not, it's the easiest white…

These discussions often end up resembling religious arguments. "We don't know how any of this works, but we can fathom an intelligent god doing it, therefore an intelligent god did it."

"We don't really know how human consciousness works, but the LLM resembles things we associate with thought, therefore it is thought."

I think most people would agree that the functioning of an LLM resembles human thought, but I think most people, even the ones who think that LLMs can think, would agree that LLMs don't think in the exact same way that a human brain does. At best, you can argue that whatever they are doing could be classified as "thought" because we barely have a good definition for the word in the first place.

Re: Bag of words, have mercy on us

#87

Post by red75prime »

XorNot wrote: post #67
electroglyph wrote: post #22

Earlier quoted context omitted.

the anthropomorphization (say that 3 times quickly) is kinda weird, but also makes for a much more pleasant conversation imo. it's kinda tedious being pedantic all the time.

It also leads to fundamentally wrong conclusions: a related issue I have with this is the use of anthropomorphic shorthand when discussing international politics. You've heard a phrase like "the US thinks...", "China wants...", "Europe believes..." so much you don't even notice it. All useful shorthands, all which lead to people displaying fundamental misunderstandings of what they're talking about - i.e. expressing…

A country. A collective of people with a dedicated structure to represent interests and enforce strategies of the said collective as a whole.

Re: Bag of words, have mercy on us

#88

Post by coppsilgold »

MyOutfitIsVague wrote: post #84
coppsilgold wrote: post #74

Is a brain not a token prediction machine? Tokens in form of neural impulses go in, tokens in the form of neural impulses go out. We would like to believe that there is something profound happening inside and we call that consciousness. Unfortunately when reading about split-brain patient experiments or agenesis of the corpus callosum cases I feel like we are all deceived, every moment of every day. I came to realiza…

Could an LLM trained on nothing and looped upon itself eventually develop language, more complex concepts, and everything else, based on nothing? If you loop LLMs on each other, training them so they "learn" over time, will they eventually form and develop new concepts, cultures, and languages organically over time? I don't have an answer to that question, but I strongly doubt it. There's clearly more going on in the…

If you come up with a genetic algorithm scaffolding to affect both the architecture and the training algorithm, and then you instantiate it in an artificial selection environment, and you also give it trillions generations to evolve evolvability just right (as life had for billions of years) then the answer is yes, I'm certain it will and probably much sooner than we did.

Also, I think there is a very high chance that given an existing LLM architecture there exists a set of weights that would manifest a true intelligence immediately upon instantiation (with anterograde amnesia). Finding this set of weights is the problem.

Re: Bag of words, have mercy on us

#89

Post by Kim_Bruning »

ctoth wrote: post #66
voidhorse wrote: post #47

Earlier quoted context omitted.

When you have a thought, are you "predicting the next thing"—can you confidently classify all mental activity that you experience as "predicting the next thing"? Language and society constrains the way we use words, but when you speak, are you "predicting"? Science allows human beings to predict various outcomes with varying degrees of success, but much of our experience of the world does not entail predicting things…

> When you have a thought, are you "predicting the next thing" Yes. This is the core claim of the Free Energy Principle[0], from the most-cited neuroscientist alive. Predictive processing isn't AI hype - it's the dominant theoretical framework in computational neuroscience for ~15 years now. > much of our experience of the world does not entail predicting things Introspection isn't evidence about computational archit…

Oh, I was looking for something like that! Saved to zotero. Thank you!

Re: Bag of words, have mercy on us

#90

Post by FarmerPotato »

estearum wrote: post #60
djoldman wrote: post #58

As a consequence of my profession, I understand how LLMs work under the hood. I also know that we data and tech folks will probably never win the battle over anthropomorphization. The average user of AI, nevermind folks who should know better, is so easily convinced that AI "knows," "thinks," "lies," "wants," "understands," etc. Add to this that all AI hosts push this perspective (and why not, it's the easiest white…

I'm a neurologist, and as a consequence of my profession, I understand how humans work under the hood. The average human is so easily convinced that humans "know", "think", "lie", "want", "understand", etc. But really it's all just a probabilistic chain reaction of electrochemical and thermal interactions. There is literally nowhere in the brain's internals for anything like "knowing" or "thinking" or "lying" to happ…

>I'm a neurologist, and as a consequence of my profession, I understand how humans work under the hood.

There you go again, auto-morphizing the meat-bags. Vroom vroom.

Post reply on HN