Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bag of words, have mercy on us

experimental-history.com

Post reply on HN
Print view View original

171–180 of 362 posts

Re: Bag of words, have mercy on us

#171

Post by meheleventyone »

panarky wrote: post #152
meheleventyone wrote: post #147

Earlier quoted context omitted.

Human brains aren’t magic in the literal sense but do have a lot of mechanisms we don’t understand. They’re certainly special both within the individual but also as a species on this planet. There are many similar to human brains but none we know of with similar capabilities. They’re also most obviously certainly different to LLMs both in how they work foundationally and in capability. I definitely agree with the mat…

When someone says "AIs aren't really thinking" because AIs don't think like people do, what I hear is "Airplanes aren't really flying" because airplanes don't fly like birds do.

This really shows how imprecise a term 'thinking' is here. In this sense any predictive probabilistic blackbox model could be termed 'thinking'. Particularly when juxtaposed against something as concrete as flight that we have modelled extremely accurately.

Re: Bag of words, have mercy on us

#172

Post by zkmon »

bloaf wrote: post #41

Everyone is out here acting like "predicting the next thing" is somehow fundamentally irrelevant to "human thinking" and it is simply not the case. What does it mean to say that we humans act with intent? It means that we have some expectation or prediction about how our actions will effect the next thing, and choose our actions based on how much we like that effect. The ability to predict is fundamental to our abili…

It may be doing the "thinking" and could reach AGI. But we don't want it. We don't want to take a fork lift to the gym. We don't want plastic aliens showing off their AGI and asking humanity to outsource human thinking and decision-making to them.

Re: Bag of words, have mercy on us

#173

Post by Terr_ »

voidhorse wrote: post #48
bloaf wrote: post #46

Earlier quoted context omitted.

And the big players have built a bunch of workflows which embed many other elements besides just "predictions" into their AI product. Things like web search, to incorporating feedback from code testing, to feeding outputs back into future iterations. Who is to say that one or more of these additions has pushed the ensemble across the threshold and into "real actual thinking." The near-religious fervor which people in…

I completely agree that we don't know enough, but I suggest that that entails that the critics and those who want to be cautious are correct. The harms engendered by underestimating LLM capabilities are largely that people won't use the LLMs. The harms engendered by overestimating their capabilities can be as severe as psychological delusion, of which we have an increasing number of cases. Given we don't actually hav…

> can be as severe as psychological delusion

Much worse, when insufficiently skeptical humans link the LLM to real-world decisions to make their own lives easier.

Consider the Brazil-movie-esque bureaucratic violence of someone using it to recommend fines or sentencing.

https://www.nature.com/articles/s41586-024-07856-5

Re: Bag of words, have mercy on us

#174

Post by yfontana »

bamboozled wrote: post #54
bloaf wrote: post #46

Earlier quoted context omitted.

And the big players have built a bunch of workflows which embed many other elements besides just "predictions" into their AI product. Things like web search, to incorporating feedback from code testing, to feeding outputs back into future iterations. Who is to say that one or more of these additions has pushed the ensemble across the threshold and into "real actual thinking." The near-religious fervor which people in…

I take a offence in the idea I’m “religiously downplaying LLMs”. I pay top dollar for access to the best models because I want the capabilities to be good / better. Just because I’m documenting my experience it doesn’t mean I have an Anti-ai agenda ? I pay because I find LLMs to be useful. Just not in the way suggested by the marketing teams. I’m downplaying because I have honestly been burned by these tools when I’v…

> I’m downplaying because I have honestly been burned by these tools when I’ve put trust in their ability to understand anything, provide a novel suggestion or even solve some basic bugs without causing other issues.?

I've had that experience plenty of times with actual people... LLMs don't "think" like people do, that much is pretty obvious. But I'm not at all sure whether what they do can be called "thinking" or not.

Re: Bag of words, have mercy on us

#175

Post by red75prime »

dapperdrake wrote: post #132
raincole wrote: post #122

> “Bag of words” is a also a useful heuristic for predicting where an AI will do well and where it will fail. “Give me a list of the ten worst transportation disasters in North America” is an easy task for a bag of words, because disasters are well-documented. On the other hand, “Who reassigned the species Brachiosaurus brancai to its own genus, and when?” is a hard task for a bag of words, because the bag just doesn…

When sensitivity analysis of ordinary least-squares regression became a thing it was also a "retrospective narrative". That seems reasonable for detecting fundamental issues with statistical models of the world. This point generalizes even if the concrete example falls down.

Does it generalize though? What a bag-of-words metaphor can say about a question "How many reinforcement learning training examples an LLM need to significantly improve performance on mathematical questions?"

Re: Bag of words, have mercy on us

#177

Post by habinero »

akersten wrote: post #81
viccis wrote: post #3

Every day I see people treat gen AI like a thinking human, Dijkstra's attitudes about anthropomorphizing computers is vindicated even more. That said, I think the author's use of "bag of words" here is a mistake. Not only does it have a real meaning in a similar area as LLMs, but I don't think the metaphor explains anything. Gen AI tricks laypeople into treating its token inferences as "thinking" because it is traine…

Bag of words is actually the perfect metaphor. The data structure is a bag. The output is a word. The selection strategy is opaquely undefined. > Gen AI tricks laypeople into treating its token inferences as "thinking" because it is trained to replicate the semiotic appearance of doing so. A "bag of words" doesn't sufficiently explain this behavior. Something about there being significant overlap between the smartest…

Yeah. I have a half-cynical/half-serious pet theory that a decent fraction of humanity has a broken theory of mind and thinks everyone has the same thought patterns they do. If it talks like me, it thinks like me.

Whenever the comment section takes a long hit and goes "but what is thinking, really" I get slightly more cynical about it lol

Re: Bag of words, have mercy on us

#178

Post by Davidzheng »

meheleventyone wrote: post #147
akoboldfrying wrote: post #112

Earlier quoted context omitted.

LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others? If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.

Human brains aren’t magic in the literal sense but do have a lot of mechanisms we don’t understand. They’re certainly special both within the individual but also as a species on this planet. There are many similar to human brains but none we know of with similar capabilities. They’re also most obviously certainly different to LLMs both in how they work foundationally and in capability. I definitely agree with the mat…

ofc, and probably will never understand because of sheer complexity. It doesn't mean we can't replicate the output distribution through data. Probably when we do in efficient manners, the mechanisms (if they are efficient) will be learned too.

Re: Bag of words, have mercy on us

#179

Post by Davidzheng »

RayVR wrote: post #70
bloaf wrote: post #41

Everyone is out here acting like "predicting the next thing" is somehow fundamentally irrelevant to "human thinking" and it is simply not the case. What does it mean to say that we humans act with intent? It means that we have some expectation or prediction about how our actions will effect the next thing, and choose our actions based on how much we like that effect. The ability to predict is fundamental to our abili…

Are you a stream of words or are your words the “simplistic” projection of your abstract thoughts? I don’t at all discount the importance of language in so many things, but the question that matters is whether statistical models of language can ever “learn” abstract thought, or become part of a system which uses them as a tool. My personal assessment is that LLMs can do neither.

Even if they are "simplistic projections", which I don't think is the correct way to think about it, there's no reason that more LLM thoughts in middle layers can't also exist and project down at the end. Though there might be efficency issues because the latent thoughts have to be recomputed a lot.

Though I do think in human brains it's also an interplay where what we write/say also loops back into the thinking as well. Which is something which is efficient for LLMs.

Re: Bag of words, have mercy on us

#180

Post by Davidzheng »

viccis wrote: post #3

Every day I see people treat gen AI like a thinking human, Dijkstra's attitudes about anthropomorphizing computers is vindicated even more. That said, I think the author's use of "bag of words" here is a mistake. Not only does it have a real meaning in a similar area as LLMs, but I don't think the metaphor explains anything. Gen AI tricks laypeople into treating its token inferences as "thinking" because it is traine…

well they are trained to be almost in distribution as a thinking human. So...
Post reply on HN