Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

ranpara.net

Post reply on HN
Print view View original

101–110 of 204 posts

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#101

Post by epihelix »

DevarshRanpara wrote: post #2

Stop for one second and ask yourself a simple question. Where do your words come from? When you speak, what comes first, the idea or the word? Do you first feel a thought inside you, and only after that go searching for the right word to wrap around it? I think we all do. The word is never the start. The word is just the skin. The idea, the consciousness, is the thing sitting under it.

I don't know about you, but I think with words all the time when reasoning through complex ideas. Where do you think the phrase "thinking out loud" comes from?

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#102

Post by palmotea »

fasteddie31003 wrote: post #59

Maybe consciousness just does not exist. Could it be that humans invented it to feel superior to animals, but it does not really exist? I wonder if we will think of the word "consciousness" in the same way we think of the word "aether" in the future.

> Maybe consciousness just does not exist. Could it be that humans invented it to feel superior to animals, but it does not really exist? I wonder if we will think of the word "consciousness" in the same way we think of the word "aether" in the future.

That's just lame, motivated denialism.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#103

Post by cathalharte »

I share your positivity, however I don’t think figuring out how to market through the noise is the important part.

LLMs don’t need to think, they just need to be a more efficient data retrieval system than what already exists (and what has already been sold to the highest bidder). With LLMs, I’ve find I’ve started reaching the corners of the internet, where the true lovers of knowledge are doing the work they have always been doing. Quality does eventually speak for itself, at least once we’re able to strip away the marketing (you mentioned Apple, do you believe they are truly a cut above, or is it just hype?)

If LLMs can filter the noise, build it and they will come might return as a valid way to be.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#105

Post by lightedman »

Fraterkes wrote: post #87
estearum wrote: post #55

Earlier quoted context omitted.

IMO it's far more likely that people don't agree on what it means to "have an internal monologue" than that people actually bifurcate that dramatically on cognition. Surely there are some people with an actual difference from baseline, but I don't think it's the near-equal split that seems to appear when you just ask people about internal monologue This is in contrast with aphantasia (complete lack of mental imagery)…

Do you have a source for the pupil dilation thing? Sounds fascinating!

I scored a 68 on the VVIQ and I can watch my pupils dilate in response to me 'seeing' something in my mind. It doesn't have to be a bright light in my mind's eye, I can imagine something that would trigger a fight or flight response and make the same thing happen.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#106

Post by jvanderbot »

adamzwasserman wrote: post #47
jacobr1 wrote: post #20

LLMs as clearly doing more than just modeling words. In order to predict word placement, they need to build some kind of model, their latent space, of the types of things they are able to predict. Not really full world models yet - but they have decent "blog space" or "github project" models. And you can see this with multimodality, or non-text modality modals such as images and audio. They map from their latent spac…

Or.. the LLMs are just "telescope" sufficiently powerful to detect the aligned internal representation that already exists. A more powerful telescope does not "create" new galaxies.

Love the analogy. I a lot of people had their mind blown by vector space representation of words. The idea that Female + King averaged out around "Queen". Words might foundational to concepts or just really-well-designed ways to transmit them.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#107

Post by sometimelurker »

jacobr1 wrote: post #20

LLMs as clearly doing more than just modeling words. In order to predict word placement, they need to build some kind of model, their latent space, of the types of things they are able to predict. Not really full world models yet - but they have decent "blog space" or "github project" models. And you can see this with multimodality, or non-text modality modals such as images and audio. They map from their latent spac…

And they do have internal latent vectors for their own states, so there's some kind of recursion/introspectiveness dynamic.

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#108

Post by Fraterkes »

jacobr1 wrote: post #20

LLMs as clearly doing more than just modeling words. In order to predict word placement, they need to build some kind of model, their latent space, of the types of things they are able to predict. Not really full world models yet - but they have decent "blog space" or "github project" models. And you can see this with multimodality, or non-text modality modals such as images and audio. They map from their latent spac…

The assumption of this sort of argument, if I understand it correctly, is that consciousness is just an ordinary byproduct that appears (or grows gradually) somewhere on the spectrum of complexity.

If that’s true, mastering language is basically orthogonal to being conscious (as you’d maybe expect, GPT-2 was pretty good at language, but had a relatively tiny amount of “neurons”).

The same is arguably true for world modeling: a math textbook has a very complex and coherent model of a world, and is, probably, unconscious.

Then the question is: what reason do we have to assume that these models have some form of consciousness?

Re: Words Are a Byproduct of Consciousness. For LLMs, It's Backwards

#110

Post by sometimelurker »

jvanderbot wrote: post #106
adamzwasserman wrote: post #47

Earlier quoted context omitted.

Or.. the LLMs are just "telescope" sufficiently powerful to detect the aligned internal representation that already exists. A more powerful telescope does not "create" new galaxies.

Love the analogy. I a lot of people had their mind blown by vector space representation of words. The idea that Female + King averaged out around "Queen". Words might foundational to concepts or just really-well-designed ways to transmit them.

You have the word 'bot' in your name and judging by your comment history love using em-dashes. Are you a machine or not?
Post reply on HN