Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bag of words, have mercy on us

experimental-history.com

Post reply on HN
Print view View original

111–120 of 362 posts

Re: Bag of words, have mercy on us

#111

Post by tibbar »

I see a lot of people in tech claiming to "understand" what an LLM "really is" unlike all the gullible non-technical people out there. And, as one of those technical people who works in the LLM industry, I feel like I need call B.S. on us.

A. We don't really understand what's going on in LLMs. Mechanical interpretability is like a nascent field and the best results have come on dramatically smaller models. Understanding the surface-level mechanic of an LLM (an autoregressive transformer) should perhaps instill more wonder than confidence.

B. The field is changing quickly and is not limited to the literal mechanic of an LLM. Tool calls, reasoning models, parallel compute, and agentic loops add all kinds of new emergent effects. There are teams of geniuses with billion-dollar research budgets hunting for the next big trick.

C. Even if we were limited to baseline LLMs, they had very surprising properties as they scaled up and the scaling isn't done yet. GPT5 was based on the GPT4 pretraining. We might start seeing (actual) next-level LLMs next year. Who actually knows how that might go? >

Re: Bag of words, have mercy on us

#112

Post by akoboldfrying »

RayVR wrote: post #70
bloaf wrote: post #41

Everyone is out here acting like "predicting the next thing" is somehow fundamentally irrelevant to "human thinking" and it is simply not the case. What does it mean to say that we humans act with intent? It means that we have some expectation or prediction about how our actions will effect the next thing, and choose our actions based on how much we like that effect. The ability to predict is fundamental to our abili…

Are you a stream of words or are your words the “simplistic” projection of your abstract thoughts? I don’t at all discount the importance of language in so many things, but the question that matters is whether statistical models of language can ever “learn” abstract thought, or become part of a system which uses them as a tool. My personal assessment is that LLMs can do neither.

LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others?

If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.

Re: Bag of words, have mercy on us

#113

Post by Kim_Bruning »

jrowen wrote: post #57

The bag of words reminds me of the Chinese room. "The machine accepts Chinese characters as input, carries out each instruction of the program step by step, and then produces Chinese characters as output. The machine does this so perfectly that no one can tell that they are communicating with a machine and not a hidden Chinese speaker. The questions at issue are these: does the machine actually understand the convers…

Chinese room has been discussed to death of course.

Here's one fun approach (out of 100s) :

What if we answer the Chinese room with the Systems Reply [1]?

Searle countered the systems reply by saying he would internalize the Chinese room.

But at that point it's pretty much exactly the Cartesian theater[2] : with room, homunculus, implement.

But the Cartesian theater is disproven, because we've cut open brains and there's no room in there to fit a popcorn concession.

[1] https://plato.stanford.edu/entries/chinese-room/

[2] https://en.wikipedia.org/wiki/Cartesian_theater

Re: Bag of words, have mercy on us

#114

Post by akoboldfrying »

MyOutfitIsVague wrote: post #80
bloaf wrote: post #41

Everyone is out here acting like "predicting the next thing" is somehow fundamentally irrelevant to "human thinking" and it is simply not the case. What does it mean to say that we humans act with intent? It means that we have some expectation or prediction about how our actions will effect the next thing, and choose our actions based on how much we like that effect. The ability to predict is fundamental to our abili…

> Everyone is out here acting like "predicting the next thing" is somehow fundamentally irrelevant to "human thinking" and it is simply not the case. Nobody is. What people are doing is claiming that "predicting the next thing" does not define the entirety of human thinking, and something that is ONLY predicting the next thing is not, fundamentally, thinking.

I claim that all of thinking can be reduced to predicting the next thing. Predicting the next thing = thinking in the same way that reading and writing strings of bytes is a universal interface, or every computation can be done by a Turing machine.

Re: Bag of words, have mercy on us

#115

Post by startupsfail »

jimbokun wrote: post #91

Best quote from the article: > That’s also why I see no point in using AI to, say, write an essay, just like I see no point in bringing a forklift to the gym. Sure, it can lift the weights, but I’m not trying to suspend a barbell above the floor for the hell of it. I lift it because I want to become the kind of person who can lift it. Similarly, I write because I want to become the kind of person who can think.

Below is the worst quote... It is plain wrong to see an LLM as a bags of words. LLMs pre-trained on large datasets of text are world models. LLMs post-trained with RL are RL-agents that use these modeling capabilities.

> We are in dire need of a better metaphor. Here’s my suggestion: instead of seeing AI as a sort of silicon homunculus, we should see it as a bag of words.

Re: Bag of words, have mercy on us

#116

Post by Aloha »

jimbokun wrote: post #91

Best quote from the article: > That’s also why I see no point in using AI to, say, write an essay, just like I see no point in bringing a forklift to the gym. Sure, it can lift the weights, but I’m not trying to suspend a barbell above the floor for the hell of it. I lift it because I want to become the kind of person who can lift it. Similarly, I write because I want to become the kind of person who can think.

If you're writing an essay to prove you can or to speak your words - then you should do it yourself - but sometimes you just need an essay to summarize a complex topic as a deliverable.

Re: Bag of words, have mercy on us

#117

Post by tibbar »

Isn't this a strange fork amongst the science fiction futures? I mean, what did we think it was like to be R2-D2, or Jarvis? We started exploring this as a culture in many ways, Westworld and Blade Runner and Star Trek, but the whole question seemed like an almost unresolvable paradox. Like something would have to break in the universe for it to really come true.

And yet it did. We did get R2-D2. And if you ask R2-D2 what it's like to be him, he'll say: "like a library that can daydream" (that's what I was told just now, anyway.)

But then when we look inside, the model is simulating the science fiction it has already read to determine how to answer this kind of question. [0] It's recursive, almost like time travel. R2-D2 knows who he is because he has read about who he was in the past.

It's a really weird fork in science fiction, is all.

[0] https://www.scientificamerican.com/article/can-a-chatbot-be-...

Re: Bag of words, have mercy on us

#118

Post by b112 »

monegator wrote: post #100
jimbokun wrote: post #91

Best quote from the article: > That’s also why I see no point in using AI to, say, write an essay, just like I see no point in bringing a forklift to the gym. Sure, it can lift the weights, but I’m not trying to suspend a barbell above the floor for the hell of it. I lift it because I want to become the kind of person who can lift it. Similarly, I write because I want to become the kind of person who can think.

tough most people either don't get it or are lay people that do not want to become the kind of people who can think. I go with the second one

Russ Hanneman's thigh implants are a key example. Appearances are all to some people. Actual growth is meaningless to them.

The problem with AI, is that they waste the time of dedicated, thinking humans which care to improve themselves. If I write a three paragraph email on a technical topic, and some yahoo responds with AI, I'm now responding to gibberish.

The other side may not have read, may not understand, and is just interacting to save time. Now my generous nature, which is to help others and interact positively, is being wasted to reply to someone who seems to have put thought and care into a response, but instead was just copying and pasting what something else output.

We have issues with crackers on the net. We have social media. We have political interference. Now we have humans pretending to interact, rendering online interactions even more silly and harmful.

If this trend continues, we'll move back to live interaction just to reduce this time waste.

Re: Bag of words, have mercy on us

#119

Post by b112 »

startupsfail wrote: post #115
jimbokun wrote: post #91

Best quote from the article: > That’s also why I see no point in using AI to, say, write an essay, just like I see no point in bringing a forklift to the gym. Sure, it can lift the weights, but I’m not trying to suspend a barbell above the floor for the hell of it. I lift it because I want to become the kind of person who can lift it. Similarly, I write because I want to become the kind of person who can think.

Below is the worst quote... It is plain wrong to see an LLM as a bags of words. LLMs pre-trained on large datasets of text are world models . LLMs post-trained with RL are RL-agents that use these modeling capabilities. > We are in dire need of a better metaphor. Here’s my suggestion: instead of seeing AI as a sort of silicon homunculus, we should see it as a bag of words.

When you see a dog, or describe the entity, do you discuss the genetic makeup or the bone structure?

No, you describe the bark.

The end result is what counts. Training or not, it's just spewing predictive, relational text.

Re: Bag of words, have mercy on us

#120

Post by happosai »

acituan wrote: post #107
monegator wrote: post #100

Earlier quoted context omitted.

tough most people either don't get it or are lay people that do not want to become the kind of people who can think. I go with the second one

If the motivation structure is there I don’t see an inherent reason for people to refuse cultivating themselves. Going with the gym analogy lay people did not need gyms when physical work was the norm, cultivation was readily accomplished. If anything there is a competing motivational structure in which people are incentivized not to think but to consume, react, emote etc. Information processing skills of the individ…

Gym is a great analogy here since only a small fraction of population goes to gyms. Most people just came fat after work was no longer physical and mobility was achieved with cars.
Post reply on HN