Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bag of words, have mercy on us

experimental-history.com

Post reply on HN
Print view View original

31–40 of 362 posts

Re: Bag of words, have mercy on us

#31

Post by Kim_Bruning »

tkgally wrote: post #19

I am unsure myself whether we should regard LLMs as mere token-predicting automatons or as some new kind of incipient intelligence. Despite their origins as statistical parrots, the interpretability research from Anthropic [1] suggests that structures corresponding to meaning do exist inside those bundles of numbers and that there are signs of activity within those bundles of numbers that seem analogous to thought. T…

Amanda Askell studied under David Chalmers at NYU: the philosopher who coined "the hard problem of consciousness" and is famous for taking phenomenal experience seriously rather than explaining it away. That context makes her choice to speak this way more striking: this isn't naive anthropomorphizing from someone unfamiliar with the debates. It's someone trained by one of the most rigorous philosophers of consciousness, who knows all the arguments for dismissing mental states in non-biological systems, and is still choosing to speak carefully about models potentially having something like feelings or insecurities.

Re: Bag of words, have mercy on us

#32

Post by bloaf »

viccis wrote: post #3

Every day I see people treat gen AI like a thinking human, Dijkstra's attitudes about anthropomorphizing computers is vindicated even more. That said, I think the author's use of "bag of words" here is a mistake. Not only does it have a real meaning in a similar area as LLMs, but I don't think the metaphor explains anything. Gen AI tricks laypeople into treating its token inferences as "thinking" because it is traine…

I'll make the following observation:

The contra-positive of "All LLMs are not thinking like humans" is "No humans are thinking like LLMs"

And I do not believe we actually understand human thinking well enough to make that assertion.

Indeed, it is my deep suspicion that we will eventually achieve AGI not by totally abandoning today's LLMs for some other paradigm, but rather embedding them in a loop with the right persistence mechanisms.

Re: Bag of words, have mercy on us

#33

Post by grogenaut »

PrairieFire wrote: post #21
grogenaut wrote: post #14

Earlier quoted context omitted.

Your analogy makes no sense. VHS spawned the entire home market, which went through multiple quality upgrades well above beta. It would only make sense if in 2025 we were using vhs everywhere and that the current state of the art for LLMs is all there ever is.

I feel like their analogy could have worked if they had pushed a little further into it. The RNN and LSTM architectures (and Word2Vec, n-grams, etc) yielded language models that never got mass adoption. Like reel to reel. Then the transformer+attention hit the scene and several paths kicked off pretty close to each other. Google was working on Bert/encoder only transformer, maybe you could call that betamax. Doesn’t…

Vhs came out in 76, blockbuster started in 85 (we went to video stores well before that when I was a kid), dvd in 95. I remember the sopranos making a joke about how dvd was barely taking off, they started in 99. Lets call it VHS had a run from 80 to 99, that's 19 years. The iphone launched in 2007, when did mobile become huge or inseprable from doing life (by force by so many apps), probbably in the pandemic.

I wouldn't say VHS was a blip. It was the recorded half video of media for almost 20 years.

I agree with the rest of what you said.

I'll say that the differences in the AI you're talking about today might be like the differences between VAX, PC JR, and the Lisa. All things before computing went main stream. I do think things go mainstream from tech a lot faster these days, people don't want to miss out.

I don't know where I'm going with this, I'm reading and replying to HN while watching the late night NFL game in an airport lounge.

Re: Bag of words, have mercy on us

#34

Post by domador »

roxolotl wrote: post #18
viccis wrote: post #3

Every day I see people treat gen AI like a thinking human, Dijkstra's attitudes about anthropomorphizing computers is vindicated even more. That said, I think the author's use of "bag of words" here is a mistake. Not only does it have a real meaning in a similar area as LLMs, but I don't think the metaphor explains anything. Gen AI tricks laypeople into treating its token inferences as "thinking" because it is traine…

Yea bag of words isn’t helpful at all. I really do think that “superpowered sentence completion” is the best description. Not only is it reasonably accurate it is understandable, everyone has seen autocomplete function, and it’s useful. I don’t know how to “use” a bag of words. I do know how to use sentence completion. It also helps explains why context matters.

I've been recently using a similar description, referring to "AI" (LLMs) as "glorified autocomplete" or "luxury autocomplete".

Re: Bag of words, have mercy on us

#35

Post by bamboozled »

tkgally wrote: post #19

I am unsure myself whether we should regard LLMs as mere token-predicting automatons or as some new kind of incipient intelligence. Despite their origins as statistical parrots, the interpretability research from Anthropic [1] suggests that structures corresponding to meaning do exist inside those bundles of numbers and that there are signs of activity within those bundles of numbers that seem analogous to thought. T…

I use LLMs heavily for work, I have done so for about 6 months. I see almost zero "thought" going on and a LOT of pattern matching. You can use this knowledge to your advantage if you understand this. If you're relying on it to "think", disaster will ensue. At least that's been my experience.

I've completely given up on using LLMs for anything more than a typing assistant / translator and maybe an encyclopedia when I don't care about correctness.

Re: Bag of words, have mercy on us

#36

Post by CamperBob2 »

bitwize wrote: post #25

I was trying to explain the concept of "token prediction" to my wife, whose eyes glaze over when discussing such technical topics. (I think she has the brainpower to understand them, but a horrible math teacher gave her a taste aversion to even attempting to that hasn't gone away. So she just buys Apple stuff and hopes Tim Apple hasn't shuffled around the UI bits AGAIN.) I stumbled across a good-enough analogy based…

Did you explain how LLMs can achieve gold-medal performance at math competitions involving original problems, without any original knowledge or thought?

Did she ask if a "statistical soup of words," if large enough, might somehow encode or represent something a little more profound than just a bunch of words?

Re: Bag of words, have mercy on us

#37

Post by est »

> Who reassigned the species Brachiosaurus brancai to its own genus, and when?

To be fair, everage person couldn't answer this either, at least not without thorough research.

Re: Bag of words, have mercy on us

#38

Post by andai »

4bpp wrote: post #28

As usual with these, it helps to try to keep the metaphor used for downplaying AI, but flip the script. Let's grant the author's perception that AI is a "bag of words", which is already damn good at producing the "right words" for any given situation, and only keeps getting better at it. Sure, this is not the same as being a human. Does that really mean, as the author seems to believe without argument, that humans ne…

So a human is just a really expensive, unreliable bag of words. And we get more expensive and more unreliable by the day!

There's a quote I love but have misplaced, from the 19th century I think. "Our bodies are just contraptions for carrying our heads around." Or in this instance... bag of words transport system ;)

Re: Bag of words, have mercy on us

#39

Post by andai »

tkgally wrote: post #19

I am unsure myself whether we should regard LLMs as mere token-predicting automatons or as some new kind of incipient intelligence. Despite their origins as statistical parrots, the interpretability research from Anthropic [1] suggests that structures corresponding to meaning do exist inside those bundles of numbers and that there are signs of activity within those bundles of numbers that seem analogous to thought. T…

Well, she's describing the system's behavior.

My fridge happily reads inputs without consciousness, has goals and takes decisions without "thinking", and consistently takes action to achieve those goals. (And it's not even a smart fridge! It's the one with a copper coil or whatever.)

I guess the cybernetic language might be less triggering here (talking about systems and measurements and control) but it's basically the same underlying principles. One is just "human flavored" and I therefore more prone to invite unhelpful lines of thinking?

Except that the "fridge" in this case is specifically and explicitly designed to emulate human behavior so... you would indeed expect to find structures corresponding to the patterns it's been designed to simulate.

Wondering if it's internalized any other human-like tendencies — having been explicitly trained to simulate the mechanisms that produced all human text — doesn't seem too unreasonable to me.

Re: Bag of words, have mercy on us

#40

Post by drivebyhooting »

paulryanrogers wrote: post #27
drivebyhooting wrote: post #17

Earlier quoted context omitted.

That old saw is patently false.

Why? It suggests to me, having encountered it for the first time, that programs must be readable to remain useful. Otherwise they'll be increasingly difficult to execute.

Maybe difficult to change but they can still serve their purpose.

It’s patently false in that code gets executed much more than it is read by humans.

Post reply on HN