Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They’re made out of weights

maxleiter.com

Post reply on HN
Print view View original

71–80 of 739 posts

Re: They’re made out of weights

#71

Post by photochemsyn »

No mention of ‘static’ vs. ‘dynamic’ is a bit disappointing in reference to the weights. Because you could argue that every neuron in your nervous system can be modeled as a collection of weights, firing likelihoods, receptor sensitivities, current dynamic state of that neuron - but LLMs are static collections of weights at inference time, with the dynamic adjustment of weights takes place at training time. So, just a ROM construct, like something out of Neuromancer, just trained on all written knowledge, not just one person’s total lived experience.

The above take fails in the real world because neuronal cells don’t exist in a vacuum; they are products of cellular development from a zygotic union of haploid contributors of sequential genetic information optimized for survival in an oxygen-rich biosphere powered largely by our local star that supports mammalian life (and microbial, plant, avian, etc.). Real AI would thus be AL - artificial life - as much as artificial intelligence. I don’t think you can have the one without the other, which upsets the simulationists who think an agent in the Matrix would be intelligent.

What either interpretation implies is that any real ‘artificial’ intelligence would be no more artificial than you or I, but it would have to dynamically update its weights at the same speed a human nervous system could (think how quickly we learn not to poke a cactus). For it to be at all trustworthy, then like a human, it would have to undergo a socialization process, one of the results of which is the development of a sense of embarrassment when it breaks acceptable social norms.

Hmm, this reminds me of the recent statement of the Pope about AI, of which I immediately thought, “Wait a second, aren’t there a fair number of people like this? The narcissistic sociopath profile, I think it’s called, a bit unfair to assume any real AI would turn out this way, isn’t it?”

Pope: “ Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate or even simulate, but they do not understand what they produce, for they lack the affective, relational, and spiritual perspective through which human beings grow in wisdom.”

Re: They’re made out of weights

#72

Post by ProllyInfamous »

Imagine writing something so incredibly brilliant (rather: adapting from the original) that it's entirely unlikely that you'll ever write something so incredible ever again.

But congrats: this is absolutely & incredibly brilliant.

Can't wait for the Jon Benjamin voiceover.

Re: They’re made out of weights

#73

Post by anon84873628 »

phito wrote: post #61
noosphr wrote: post #4

It's not often I see something that's fractally wrong but here we are. There is a dictionary, it's called the tokenizer. There are grammar rules, they are just very weak because the structure of human language is generally quite weak. When presented with languages which have strong consistent grammars the weights are very easily interpretable as a grammar: https://arxiv.org/abs/2201.02177 The point of the original sh…

Also there's a brain, the GPU

Not at all. A brain is interesting because it is the computer, memory, and weights all in one. A GPU is just the calculator.

You can't move your mind to and any other brain, but weights can run on any GPU.

Re: They’re made out of weights

#74

Post by throwaway173738 »

doctoboggan wrote: post #51
slopinthebag wrote: post #30

Earlier quoted context omitted.

This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I assume you don't believe consciousness is a physical property in the brain, so what entity is actually experiencing that consciousness? Or, what does it even mean to experience consciousness? Or are these not even the right questions?

> This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I was about to post the exact opposite question? How could it not be an emergent property? Unlike consciousness, the concept of emergence is pretty well defined: An emergent property is a characteristic or behavior that a complex system has, but which its individual components do not have on their own. C…

I think consciousness is going to turn out to be very challenging to define rigorously enough that we can test for its presence or absence. Emergent or not, the question is how do you determine when it has emerged? Is it a quantity or an attribute? Discrete or continuous? Does it have a finite or infinite range?

We can all agree on what color something is, but we can’t describe the color a priori, only by example. I think consciousness may be a similar phenomenon and the only test is by shared experience. If so then we are in deep trouble because we will not be able to anticipate when a system becomes conscious.

Re: They’re made out of weights

#76

Post by wisty »

kami23 wrote: post #12

This read like poetry to me. Thank you for sharing it. I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness. For a little bit I was working on having linguistics based evals for a kaggle competition. My challenge was whether or not I could mask things well enough to not trigge…

AFAIK every argument against conciousness being emergent is just a weak "God of the gaps" argument (since we don't fully understand it all) or a nonsense analogy like the Chinsese room where if you seperate the hardware and software it's not concious anymore (like, duh, remove a brain from a body and it is no longer concious either).

Yeah, the weights not updating online makes them less like a living organism that can update and learn and evolve ... ok ....

Re: They’re made out of weights

#77

Post by anon84873628 »

benlivengood wrote: post #6
noosphr wrote: post #4

It's not often I see something that's fractally wrong but here we are. There is a dictionary, it's called the tokenizer. There are grammar rules, they are just very weak because the structure of human language is generally quite weak. When presented with languages which have strong consistent grammars the weights are very easily interpretable as a grammar: https://arxiv.org/abs/2201.02177 The point of the original sh…

I don't think the grokking paper is a great argument for the difference between weights and meat. E.g. https://en.wikipedia.org/wiki/Cortical_Labs learning to play Pong. The tokenizer is, at best, a sensory mechanism as evidenced by 1) the random generation of the tokenization scheme, and 2) vastly different tokenization schemes produce virtually identical behavior. It'd be like if Noah Webster threw a bunch of movab…

Comparing the tokenizer to sensory processing is a great analogy. That's exactly what your visual cortex and initial layers of the language center are doing: decoding visual representation of text into the internal neural representation.

It's a learned mapping from one representation to another, not some semantic lookup against an exogenous source.

Re: They’re made out of weights

#78

Post by lloeki »

ProllyInfamous wrote: post #72

Imagine writing something so incredibly brilliant (rather: adapting from the original) that it's entirely unlikely that you'll ever write something so incredible ever again . But congrats: this is absolutely & incredibly brilliant. Can't wait for the Jon Benjamin voiceover.

They're Made out of Meat

- Terry Bisson, 1991

https://web.mit.edu/people/dpolicar/writing/prose/text/think...

Radio play by Miriam Tolan and Russ Armstrong:

https://www.wnycstudios.org/podcasts/studio/segments/168264-...

(EDIT: the original parent was missing "rather adapting from the original")

Re: They’re made out of weights

#79

Post by rendall »

lloeki wrote: post #78
ProllyInfamous wrote: post #72

Imagine writing something so incredibly brilliant (rather: adapting from the original) that it's entirely unlikely that you'll ever write something so incredible ever again . But congrats: this is absolutely & incredibly brilliant. Can't wait for the Jon Benjamin voiceover.

They're Made out of Meat - Terry Bisson, 1991 https://web.mit.edu/people/dpolicar/writing/prose/text/think... Radio play by Miriam Tolan and Russ Armstrong: https://www.wnycstudios.org/podcasts/studio/segments/168264-... (EDIT: the original parent was missing "rather adapting from the original")

Video adaptation.

https://youtu.be/T6JFTmQCFHg

Re: They’re made out of weights

#80

Post by throwaway173738 »

noosphr wrote: post #23
benlivengood wrote: post #16

Earlier quoted context omitted.

I might have misunderstood the point you are making. I read the original article as "weights are like meat", and so I'm confused by what you consider fractally wrong.

The point that when the rules the model learns are simple enough they stop being spread out over all the layers and become as easily interpretable as any expert system. It's just that the rules we feed in the model are extremely poorly defined and we end up with the soup of disjoint rules smeared all across the weights. This isn't a feature of the models. It's a feature of the training set. Being shocked that you can…

So basically there are rules, we just can’t articulate them and so we can’t decode them from the weights. The Goedel Numbering metaphor is pretty appealing to me. You can represent any finite series of real numbers with a series of computations performed on some other finite series of real numbers. We just happen to be using matrices because the math is easy to parallelize. The trick is to realize that when you know the sequence you have and the sequence you want then you can compute the calculations. If you constrain the calculations to only matrix multiplication then you arrive at the scheme we have.
Post reply on HN