Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They’re made out of weights

maxleiter.com

Post reply on HN
Print view View original

161–170 of 739 posts

Re: They’re made out of weights

#161

Post by f_klem »

After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind. What all these theories have in common is the underlying belief that our brain/mind works as the machines we build. Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things. The idea of 'internal models' and 'control loops' inside us is a projection of the aforementioned assumption.

There is also an epistemological assumption that prevails, and that is that we understand (or we think we understand) how our brain/mind works. But the truth is that we don't know. And there's even not a single clue that we actually know too much, and not a clue that our brain/mind and cells work 'as the machines we build'. Only by bypassing this epistemological problem, we can build 'theories of computational mind'.

These assumptions are there for already long time, to the point that when Turing asked himself 'can machines think?', he already assumed our thinking could be modeled as a machine.

I highly recommend people in the AI research space should read philosophy and modern linguistics. But not stopping at Descartes/Leibniz. Heidegger made contributions that cannot be avoided.

Re: They’re made out of weights

#162

Post by noduerme »

sumitkumar wrote: post #148

The weights start with a random manifold. The training takes data and shapes the manifold, weight by weight, in many cycles. Once the training is the done manifold is fixed. When a new inference has to be done the query(q) is projected in the manifold space. This projection is dropped on the manifold and the gravity of the manifold gives an answer of q+1 length. Which(qw+i) is dropped qw+n times to output a final res…

In what way is that different from any other model of reality that you'd use to winnow a dataset into an answer to a question? The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with. It's almost like people desperately want to give up their agency and creativity to black boxes, whether those weights produce answers that are right or wrong. Factor in that psychology and it looks a lot less like we have invented something useful, and a lot more like we as a species are choosing to quit life en masse.

Re: They’re made out of weights

#163

Post by coldtea »

why_at wrote: post #157
Planktonne wrote: post #141

The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness. That rather undercuts the point; if this was generated by an LLM unprompted, it would be differe…

I was having a hard time pinning down what bothered me about this but I think you put it pretty well. It draws an analogy between us and the skeptical aliens in the original story which feel silly to us, so the obvious implication is that we're being as silly as they were. But it doesn't really give a reason to accept the analogy, it just asserts it. There's a big difference between a whole civilization and a piece o…

>But it doesn't really give a reason to accept the analogy, it just asserts it.

It's not a paper or a proof. It's a story. Doesn't want to prove the analogy, it wants to convey it.

Re: They’re made out of weights

#164

Post by sigmoid10 »

sumitkumar wrote: post #155
Planktonne wrote: post #141

The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness. That rather undercuts the point; if this was generated by an LLM unprompted, it would be differe…

The original did not come out of a vacuum. It was done on multiple generations of meat. Even though this one uses a little bit of silicon, it is still standing on the same shoulders.

I genuinely thought this one was a satirical take on the narrow-mindedness of the aliens in the original, even though the story tries to paint humans as narrow minded. I guess this fundamental human trait to believe that their cognition is the ultimate way to think in the universe ironically leaked into all these stories as well. Real spacefaring civilisations would probably have seen all kinds of intelligence rise from sufficiently complex systems.

Re: They’re made out of weights

#165

Post by Planktonne »

coldtea wrote: post #160
Planktonne wrote: post #141

The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness. That rather undercuts the point; if this was generated by an LLM unprompted, it would be differe…

Not having yet read the original story, this reads fine on its own. And I didn't see it as much as a literary attempt for art's sake, but more of a dialogue-based technical parable trying to convey a real-world insight. Kind of like the ones in Godel Escher Bach. > You could perform exactly the same rhetorical trick with a toaster or anything else. Not sure which rhetorical trick is that. The point of the story, as I…

> Not knowing the original story, this reads fine on its own.

Yes. Because it's heavily based on the original story. The existence of the original story is kind of a critical piece here.

Re: They’re made out of weights

#166

Post by gobdovan »

dyauspitr wrote: post #129
gobdovan wrote: post #83

You can take the weights and model description, write them down on a notebook, then, by hand, compute the next token. Try to do the same with meat.

Probably pretty similar. Weights are how many synapses there are between neurons. Temperature is whatever hormonal chemical mix is going on at the moment. Inputs tokens are electrical signals from our senses. Output tokens are thoughts, muscle movements. How you’re raised and your interactions with society are the RLHF. Some people are born with a GB300 while others have an L40S and lower token/sec output rates…

Although I get that it's a metaphor, I really dislike the "some people are born with a GB300 while others have an L40S" part. Even as a joke, it ranks people like hardware tiers, which is dehumanising and uncomfortably close to eugenic language. On top of that, the analogy also breaks down if you try to implement it literally.

For an LLM, you have clear stages of mostly feedforward computation over finite numbers and a perfect way to reconstruct the computation.

For meat, even if you model it under a purely Newtonian approximation, you need to simulate at least the immediate closed system around it which is continuous, thermodynamic, chemical and so on. You'd need to choose an arbitrary time step and update enormous amounts of coupled physical state to get an inexact simulation of a minimal slice of reality.

You would have a much harder time obtaining even a substrate-independent dead organism, comapred to LLMs that are already substrate-independent, which is basically what my notebook example shows.

Re: They’re made out of weights

#167

Post by psychoslave »

trick-or-treat wrote: post #50
kevin_thibedeau wrote: post #43

Earlier quoted context omitted.

Our machines won't have biological systems driving their needs which in turn fuel behaviors like desire and planning for the future. They may imitate them but it won't be innate.

For an LLM, "innate" means "in their training data". So yeah, those things are pretty much innate.

When we say "it rains", do we consider that "it" has any intention of agentivity?

Some questions are just ill formed.

Plus even if "LLMs are alive and conscious", this still would scratch the surface of the morale/ethical/societal considerations that people really care about.

Because even with other humans, we can argue if they exist or if they are mere npc in a solipsist world view.

Re: They’re made out of weights

#168

Post by akie »

sumitkumar wrote: post #148

The weights start with a random manifold. The training takes data and shapes the manifold, weight by weight, in many cycles. Once the training is the done manifold is fixed. When a new inference has to be done the query(q) is projected in the manifold space. This projection is dropped on the manifold and the gravity of the manifold gives an answer of q+1 length. Which(qw+i) is dropped qw+n times to output a final res…

That's a very concise and illuminating way to think about what's happening, IF (and only if) you already know how these models work. Thanks for that.

Re: They’re made out of weights

#169

Post by coldtea »

Planktonne wrote: post #165
coldtea wrote: post #160

Earlier quoted context omitted.

Not having yet read the original story, this reads fine on its own. And I didn't see it as much as a literary attempt for art's sake, but more of a dialogue-based technical parable trying to convey a real-world insight. Kind of like the ones in Godel Escher Bach. > You could perform exactly the same rhetorical trick with a toaster or anything else. Not sure which rhetorical trick is that. The point of the story, as I…

> Not knowing the original story, this reads fine on its own. Yes. Because it's heavily based on the original story. The existence of the original story is kind of a critical piece here.

Might be. But after having read the original, it could just as well be the weights version and still be about us to begin with.

I don't see how "you could do that with a toaster" still. The whole point between the original and this, is that you can't do that with a toaster or a sofa, but you can do it with meat and weights, because both share all the other analogies in the story, as well as the basic premise: the improbability of something like thinking, feeling, etc arising from a lowly substrate.

And having read both now, I see how the existence of the original is a plus for this story, not a minus. Instead of making look like mere copy (as would be the case for a typical story modelled after another), in this case, it adds a meta layer, and enriches it.

Re: They’re made out of weights

#170

Post by plastic-enjoyer »

IsTom wrote: post #146
trick-or-treat wrote: post #140

Earlier quoted context omitted.

You don't think it's sillier than "rocks don't have consciousness"?

Potential for consciousness. To have a working mind you need to have a way to gather, process and store information. In case of rocks it would be very empty, static, timeless experience.

If I believe that machines or neural networks can develop consciousness, then I also believe in some kind of substrate independence which presupposes some kind of panpyschism imho because as you say things would have a potential for consciousness, some more and some less.
Post reply on HN