Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They’re made out of weights

maxleiter.com

Post reply on HN
Print view View original

181–190 of 739 posts

Re: They’re made out of weights

#181

Post by Planktonne »

coldtea wrote: post #169
Planktonne wrote: post #165

Earlier quoted context omitted.

> Not knowing the original story, this reads fine on its own. Yes. Because it's heavily based on the original story. The existence of the original story is kind of a critical piece here.

Might be. But after having read the original, it could just as well be the weights version and still be about us to begin with. I don't see how "you could do that with a toaster" still. The whole point between the original and this, is that you can't do that with a toaster or a sofa, but you can do it with meat and weights, because both share all the other analogies in the story, as well as the basic premise: the imp…

The key point here, I think, and why it's necessary to have read the original story, is that being able to express an idea is not the same as that idea being correct.

You could go through and change all the points in the original story to be about a toaster instead. It would require you to edit text, but there is no barrier to doing that.

What I think a lot of people are getting stuck on is that editing the text to say 'toaster' would not mean that toasters were conscious, and that editing the text to say 'weights' doesn't mean that weights are conscious either. Stories aren't factual just because they are written.

The original story was written by the thing claiming to be conscious; the LLM and toaster ones would not be, which undermines the claim to consciousness a lot.

Re: They’re made out of weights

#182

Post by Nevermark »

ludwik wrote: post #136
Nevermark wrote: post #89

Earlier quoted context omitted.

For obvious survival reasons we evolved to have sensory/cognitive access to our own activity, self-monitoring, and self-modeling ourselves. The self-modeling, is in such a tight loop, it melds "ourselves" and our model of ourselves, our thinking and choices, and experience of our thinking and choices, into one component. Like you can't analyze half a wheel of a bicycle and be talking about the same thing. This awaren…

I think this is exactly it, but let me ask another question (which is not rhetorical, I really don't know). Does the fact that one can describe what consciousness is and where it came from in humans help them to detect it in non-human and/or non-biological entities?

That is a really good point. Yes, I think function is diagnosis on this.

Constant self-awareness, self-experience, self-focus, self-management, and self-improvement of one's own self (mind), is going to be an adaptive behavior for anything intelligent with resources to leverage. Whether truly independent, or highly motivated to serve others. The mind is the greatest tool.

I think that is more than simply a good functional definition of consciousness. How could all that integration and self-integration not be conscious.

Re: They’re made out of weights

#183

Post by dyauspitr »

gobdovan wrote: post #166
dyauspitr wrote: post #129

Earlier quoted context omitted.

Probably pretty similar. Weights are how many synapses there are between neurons. Temperature is whatever hormonal chemical mix is going on at the moment. Inputs tokens are electrical signals from our senses. Output tokens are thoughts, muscle movements. How you’re raised and your interactions with society are the RLHF. Some people are born with a GB300 while others have an L40S and lower token/sec output rates…

Although I get that it's a metaphor, I really dislike the "some people are born with a GB300 while others have an L40S" part. Even as a joke, it ranks people like hardware tiers, which is dehumanising and uncomfortably close to eugenic language. On top of that, the analogy also breaks down if you try to implement it literally. For an LLM, you have clear stages of mostly feedforward computation over finite numbers and…

Yes reality is non deterministic but we don’t know if you absolutely need to consider anything outside the immediate closed system for a reasonably close approximation given the same inputs. They can already repeatably (inexact of course) simulate muscular actions with the same stimuli like from Neuralink. Exact enough to let a paralyzed person draw with their mind.

Yeah the last line is a cheap shot, possibly at myself.

Re: They’re made out of weights

#184

Post by maxbond »

Planktonne wrote: post #181
coldtea wrote: post #169

Earlier quoted context omitted.

Might be. But after having read the original, it could just as well be the weights version and still be about us to begin with. I don't see how "you could do that with a toaster" still. The whole point between the original and this, is that you can't do that with a toaster or a sofa, but you can do it with meat and weights, because both share all the other analogies in the story, as well as the basic premise: the imp…

The key point here, I think, and why it's necessary to have read the original story, is that being able to express an idea is not the same as that idea being correct. You could go through and change all the points in the original story to be about a toaster instead. It would require you to edit text, but there is no barrier to doing that. What I think a lot of people are getting stuck on is that editing the text to s…

I don't think you understood the point of the story. It's not that LLMs or agents are conscious, it's that our dismissal of the possibility is reflexive and uninformed. Personally I think anyone who has made their mind up about whether or not LLMs/agents are or can be conscious has done so before the evidence is in.

The story does not assert that search and replacing "meat" with "weight" makes them conscious through some magical mechanism. It's a thought experiment.

Re: They’re made out of weights

#186

Post by JauntyHatAngle »

why_at wrote: post #157
Planktonne wrote: post #141

The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness. That rather undercuts the point; if this was generated by an LLM unprompted, it would be differe…

I was having a hard time pinning down what bothered me about this but I think you put it pretty well. It draws an analogy between us and the skeptical aliens in the original story which feel silly to us, so the obvious implication is that we're being as silly as they were. But it doesn't really give a reason to accept the analogy, it just asserts it. There's a big difference between a whole civilization and a piece o…

The bit that lost me quite early in the piece was

>"A side effect. You're asking me to believe in sentient weights."

Huh? Did I miss that logical jump? Genuine question, maybe I'm not clueing into something here.

Re: They’re made out of weights

#187

Post by eszed »

doctoboggan wrote: post #51
slopinthebag wrote: post #30

Earlier quoted context omitted.

This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I assume you don't believe consciousness is a physical property in the brain, so what entity is actually experiencing that consciousness? Or, what does it even mean to experience consciousness? Or are these not even the right questions?

> This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I was about to post the exact opposite question? How could it not be an emergent property? Unlike consciousness, the concept of emergence is pretty well defined: An emergent property is a characteristic or behavior that a complex system has, but which its individual components do not have on their own. C…

I'd expand a little bit to say inevitably emergent property. That is to say, if you create a sufficiently complex information-processing network, some level of consciousness will result. With regards to current AI, we're a fair way away from building something with enough connections, but we'll get there.

One thing that gives me pause about the inevitability hypothesis is that the type of connection, or manner of information processing, may matter: there might be something about neurons that isn't (currently) reproducible in silicon. I don't know, and there's not (yet) any evidence for or against, but it at least seems like plausible speculation. We just don't know enough about any of this right now.

Re: They’re made out of weights

#188

Post by coldtea »

bayindirh wrote: post #180
coldtea wrote: post #160

Earlier quoted context omitted.

Not having yet read the original story, this reads fine on its own. And I didn't see it as much as a literary attempt for art's sake, but more of a dialogue-based technical parable trying to convey a real-world insight. Kind of like the ones in Godel Escher Bach. > You could perform exactly the same rhetorical trick with a toaster or anything else. Not sure which rhetorical trick is that. The point of the story, as I…

> Not sure which rhetorical trick is that. The point of the story, as I read it, is the technical insight (and some social implications of it). Take a simple mechanism which has exceedingly low number of inputs and states and create a narrative around it to convey it as intelligent. For a toaster, I can rewrite the think as "They're made of metal strips!", pointing out that their thermostat is a bimetal strip, and ex…

>Take a simple mechanism which has exceedingly low number of inputs and states and create a narrative around it to convey it as intelligent. (...) For a toaster, I can rewrite the think as "They're made of metal strips!", pointing out that their thermostat is a bimetal strip, and extrapolate from there.

Doesn't that miss the whole point?

You could write "They're made of metal strips!". You wouldn't be able to write much else, as toasters don't have showcase in the way of human-level intelligent behavior. Which is the whole point in the meat and weights versions.

At best you could write "They're made of metal strips!" for toasters AND other metallic devices, and use some analogies of features BOTH have in common. But they wouldn't be intelligence related behaviors.

Re: They’re made out of weights

#189

Post by ffwd »

eszed wrote: post #176
ffwd wrote: post #128

Earlier quoted context omitted.

Personally I'm not a fan of the emergence story, for a number of reasons. First is, it doesn't really make sense for consciousness to have emerged gradually through natural selection. When we and animals are conscious, the whole brain is coordinated and works to for example turn off consciousness when we sleep. And as someone else mentioned, animals with much fewer numbers seem to have a similar consciousness to us.…

I proposed in another comment that consciousness and self-awareness are at least close cousins, and perhaps the same phenomenon. If that's true, then that's an axis upon which you might create comparative measures. Yes, hamsters are conscious, but they don't have a sense of self to the same degree that gorillas do. If you posit capacity for language as another emergent property of sufficiently-complex networks, then…

I'm not sure consciousness and self-awareness are the same thing. First is we can be conscious when we sleep/during REM sleep, where it's arguable we are not self-aware. And if not that, we can even do it when awake, for example when we think about a movie, or a philosophical problem, we can have conscious thoughts that are not related to the self. This leads me to believe consciousness is separate from self-awareness. Self-awareness is _one thing_, among many, that the brain can think about and be conscious of.

Re: They’re made out of weights

#190

Post by coldtea »

Planktonne wrote: post #181
coldtea wrote: post #169

Earlier quoted context omitted.

Might be. But after having read the original, it could just as well be the weights version and still be about us to begin with. I don't see how "you could do that with a toaster" still. The whole point between the original and this, is that you can't do that with a toaster or a sofa, but you can do it with meat and weights, because both share all the other analogies in the story, as well as the basic premise: the imp…

The key point here, I think, and why it's necessary to have read the original story, is that being able to express an idea is not the same as that idea being correct. You could go through and change all the points in the original story to be about a toaster instead. It would require you to edit text, but there is no barrier to doing that. What I think a lot of people are getting stuck on is that editing the text to s…

>You could go through and change all the points in the original story to be about a toaster instead. It would require you to edit text, but there is no barrier to doing that.

Yes, there very much is a huge barrier. The copy and the original both keep the same subject matter: intelligence/human-like behavior.

The toaster doesn't. We could do edits, and the story (original or copy) would lose all its potency.

The surprise is "but how is the richness of intelligent behavior produced from something as basic as meat/weights". Why is kind of surprising reductionism is both cases.

Whereas nobody is surprised that "a metal strip with electricity flowing through it from a power source" heats pieces of bread to a specific temperature. Even if a toaster was "metal strips" all the way down, it's nowhere near as impressive jump from substrate to behavior, nor is the behavior as important to us and touches the core of our existance.

Post reply on HN