Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bag of words, have mercy on us

experimental-history.com

Post reply on HN
Print view View original

261–270 of 362 posts

Re: Bag of words, have mercy on us

#261

Post by tjr »

efitz wrote: post #252
bloaf wrote: post #41

Everyone is out here acting like "predicting the next thing" is somehow fundamentally irrelevant to "human thinking" and it is simply not the case. What does it mean to say that we humans act with intent? It means that we have some expectation or prediction about how our actions will effect the next thing, and choose our actions based on how much we like that effect. The ability to predict is fundamental to our abili…

AI has made me question what it is to be a human. I am not having some existential crisis, but if we get to a point where X% of humans cannot outperform “AI” on any task that humans deem “useful”, for some nontrivial value of X, then many assumptions that culture has inculcated into me about humanity are no longer valid. What is the role of humans then? Can it be said that humans “think” if they can’t think a thought…

If all humans were suddenly wiped off the face of the earth, AI would go silent, and the hardware it runs on would eventually shut down.

If all AI was suddenly wiped off the face of the earth, humans would rebuild it, and would carry on fine in the meantime.

One AI researcher decades ago said something to the effect of: researchers in biology look at living organisms and wonder how they live; researchers in physics look at the cosmos and wonder what all is out there; researchers in artificial intelligence look at computer systems and wonder how they can be made to wonder such things.

Re: Bag of words, have mercy on us

#262

Post by quantummagic »

littlestymaar wrote: post #257
quantummagic wrote: post #227

Earlier quoted context omitted.

I'm not sure. If you see what they're doing with feedback already in code generation. The LLM makes a "hallucination", generates the wrong idea, then tests its code only to find out it doesn't compile. And goes on to change its idea, and try again.

A few minutes worth of “personal experience” doesn't really deserve the “personal experience” qualifier.

Why not? It's just a minor example of what's possible, to show the general concept has already started.

Re: Bag of words, have mercy on us

#263

Post by xtracto »

eric-p7 wrote: post #249
red75prime wrote: post #78

Earlier quoted context omitted.

We need a difference to discover what it is. How can we know that all LLMs don't?

If you tediously work out the LLM math by hand, is the pen and paper conscious too? Consciousness is not computation. You need something else.

This comment here is pure gold. I love it.

On the flip side: If you do that, YOU are conscious and intelligent.

Would it mean that the machine that did the computation became conscious when it did it?

What is consciousness?

Re: Bag of words, have mercy on us

#264

Post by 4bpp »

jimbokun wrote: post #259
4bpp wrote: post #128

Earlier quoted context omitted.

"Her"->"the"? (Or, who is "she" here?) Either way, in what way is this relevant? If the human's labor is not useful at any price point to any entity with money, food or housing, then they presumably will not get paid/given food/housing for it.

Why are you repeating what I said with slightly different words?

Maybe because I didn't understand what you said. Who does "her" refer to?

Re: Bag of words, have mercy on us

#265

Post by Ukv »

ACCount37 wrote: post #195
Ukv wrote: post #30

Earlier quoted context omitted.

> Spoken Query Language? Just like SQL, but for unstructured blobs of text as a database and unstructured language as a query? I feel that's more a description of a search engine. Doesn't really give an intuition of why LLMs can do the things they do (beyond retrieval), or where/why they'll fail.

If you want actionable intuition, try "a human with almost zero self-awareness". "Self-awareness" used in a purely mechanical sense here: having actionable information about itself and its own capabilities. If you ask an old LLM whether it's able to count the Rs in "strawberry" successfully, it'll say "yes". And then you ask it to do so, and it'll say "2 Rs". It doesn't have the self-awareness to know the practical l…

Agree that's a better intuition, with pretraining pushing the model towards saying "I don't know" in the kinds of situations where people write that as opposed to by introspection of its own confidence.

Re: Bag of words, have mercy on us

#266

Post by visarga »

happosai wrote: post #125
visarga wrote: post #110

Earlier quoted context omitted.

> the interpretability research from Anthropic [1] suggests that structures corresponding to meaning do exist inside those bundles of numbers and that there are signs of activity within those bundles of numbers that seem analogous to thought I did a simple experiment - took a photo of my kid in the park, showed it to Gemini and asked for a "detailed description". Then I took that description and put it into a generat…

> How did these 2 models do it if not actually using language like a thinking agent? By having a gazillion of other, almost identical pictures of kids in parks in their training data.

Not pictures with this composition, same jacket, etc - yes, there are images but they are different, this fits like a key in the lock to the original

Re: Bag of words, have mercy on us

#267

Post by voidhorse »

ctoth wrote: post #66
voidhorse wrote: post #47

Earlier quoted context omitted.

When you have a thought, are you "predicting the next thing"—can you confidently classify all mental activity that you experience as "predicting the next thing"? Language and society constrains the way we use words, but when you speak, are you "predicting"? Science allows human beings to predict various outcomes with varying degrees of success, but much of our experience of the world does not entail predicting things…

> When you have a thought, are you "predicting the next thing" Yes. This is the core claim of the Free Energy Principle[0], from the most-cited neuroscientist alive. Predictive processing isn't AI hype - it's the dominant theoretical framework in computational neuroscience for ~15 years now. > much of our experience of the world does not entail predicting things Introspection isn't evidence about computational archit…

Thanks for the links! I'll have to dig into this more for sure. Looking at the bulleted summary, I'm not sure your argument is sufficiently nuanced or being made in good faith.

The article argues that the brain "predicts" acts of perception in order to minimize surprise. First of all, very few people mean to talk about these unconscious operations of the brain when they claim they are "thinking". Most people have not read enough neuroscience literature to have such a definition. Instead, they tend to mean "self-conscious activity" when they say "thinking". Thinking, the way the term is used in the vernacular, usually implies some amount of self-reflexivity. This is why we have the term "intuition" as opposed to thinking after all. From a neuronal perspective, intuition is still thinking, but most people don't think (ha) of the word thinking to encompass this, and companies know that.

It is clear to me, as it is to everyone one the planet, that when OpenAI for example claims that ChatGPT "thinks" they want consumers to make the leap to cognitive equivalence at the level of self-conscious thought, abstract logical reasoning, long-term learning, and autonomy. These machines are designed such that they do not even learn and retain/embed new information past their training date. That already disqualifies them from strong equivalence to human beings, who are able to rework their own tendencies toward prediction in a meta cognitive fashion by incorporating new information.

Re: Bag of words, have mercy on us

#268

Post by jnd-cz »

mapontosevenths wrote: post #240
bloaf wrote: post #41

Everyone is out here acting like "predicting the next thing" is somehow fundamentally irrelevant to "human thinking" and it is simply not the case. What does it mean to say that we humans act with intent? It means that we have some expectation or prediction about how our actions will effect the next thing, and choose our actions based on how much we like that effect. The ability to predict is fundamental to our abili…

I suspect that people instinctively believe they have free will, both because it feels like we do, and because society requires us to behave that way even when we don't. The truth is that the evidence says we don't. See the Libet experiment and its many replications. Your decisions can be predicted from brain scans up to 10 seconds before you make them, which means they are as deterministic as an LLM's. Sorry, I gues…

I looked up the Libet experiment:

"Implications

The experiment raised significant questions about free will and determinism. While it suggested that unconscious brain activity precedes conscious decision-making, Libet argued that this does not negate free will, as individuals can still choose to suppress actions initiated by unconscious processes."

Re: Bag of words, have mercy on us

#269

Post by traverseda »

Hendrikto wrote: post #251
ACCount37 wrote: post #181

Earlier quoted context omitted.

Words are the "simplistic" projection of an LLM's abstract thoughts. An LLM has: words in its input plane, words in its output plane, and A LOT of cross-linked internals between the two. Those internals aren't "words" at all - and it's where most of the "action" happens. It's how LLMs can do things like translate from language to language, or recall knowledge they only encountered in English in the training data whil…

> It's how LLMs can do things like translate from language to language The heavy lifting here is done by embeddings. This does not require a world model or “thought”.

LLMs are compression and prediction. The most efficient way to (lossfully) compress most things is by actually understanding them. Not saying LLMs are doing a good job of that, but that is the fundamental mechanism here.

Re: Bag of words, have mercy on us

#270

Post by voidhorse »

ctoth wrote: post #131
belZaah wrote: post #123

Earlier quoted context omitted.

How does the free energy principle align with system dynamics and the concept of emergence? Yes, our brain might want to optimize for lack of surprise, but that does not mean it can fully avoid emergent or chaotic behavior stemming from the incredibly complex dynamics of the linked neurons?

FEP doesn't conflict with complex dynamics, it's a mathematical framework for explaining how self-organizing behavior arises from simpler variational principles. That's what makes it a theory rather than a label. The thing you're doing here has a name: using "emergence" as a semantic stopsign. "The system is complex, therefore emergence, therefore we can't really say" feels like it's adding something, but try removin…

This essay completely misunderstands how the notion of emergence gained prominence and how people tend to actually use it. It's a straw man that itself devolves into a circular argument "embrace a reductionist epistemology because you should embrace a reductionist epistemology".

It doesn't even meaningfully engage with the historical literature that established the term, etc. If you want to actually understand why the term gained prominence, check out the work of Edgar Morin.

Post reply on HN