Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They’re made out of weights

maxleiter.com

Post reply on HN
Print view View original

171–180 of 739 posts

Re: They’re made out of weights

#171

Post by throw310822 »

noosphr wrote: post #54
throw310822 wrote: post #9

Earlier quoted context omitted.

> There are grammar rules And they're made out of weights.

As opposed to integers in normal programming. The 'magic' in weights is that the rules are spread through the whole model and you can't point to one place which encodes them. The grokking paper shows that this stops being the case with enough training data and enough compute.

Integers in normal programming represent data or instructions; instructions are hand coded, have rigidly defined semantics, are not differentiable and have no redundancy.

> The 'magic' in weights is that the rules are spread through the whole model ... The grokking paper shows that this stops being the case with enough training data and enough compute.

I don't understand what you mean to say. That weights are not magic? That weights are not weights? NNs are made up of weights, which are learned and not coded. The fact that they do learn world models (grammar rules in your example), and that these models' weights tend to roughly concentrate by function and level of representation is perfectly logic but even more amazing. (Notice that much of the dismissive attitude towards LLMs depicts them as pure syntactic manipulators without the ability to develop world models- the exact opposite of what you point out).

Re: They’re made out of weights

#172

Post by Npovview »

kami23 wrote: post #12

This read like poetry to me. Thank you for sharing it. I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness. For a little bit I was working on having linguistics based evals for a kaggle competition. My challenge was whether or not I could mask things well enough to not trigge…

Quantum Field Theory visualized

https://youtu.be/MmG2ah5Df4g

Here's a more general idea. Our modern physics says that the whole universe is filled with fields and field is composed of numbers. What if we take that literally? When we say an electron is present here, we actually mean that there are more copies of particular number superposed at that place.

Re: They’re made out of weights

#174

Post by lxgr »

f_klem wrote: post #161

After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind. What all these the…

> Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things.

Agency: What’s missing, in your view? Agency seems more of a property/function of a thinking system’s position in an environment than of the thinking itself.

Subjective experience: That’s not a contradiction to “complex machines” either. I think the evidence that our minds are highly complex machines is, at this point, irrefutable. The question is really if they’re “only” that.

Re: They’re made out of weights

#175

Post by noduerme »

why_at wrote: post #157
Planktonne wrote: post #141

The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness. That rather undercuts the point; if this was generated by an LLM unprompted, it would be differe…

I was having a hard time pinning down what bothered me about this but I think you put it pretty well. It draws an analogy between us and the skeptical aliens in the original story which feel silly to us, so the obvious implication is that we're being as silly as they were. But it doesn't really give a reason to accept the analogy, it just asserts it. There's a big difference between a whole civilization and a piece o…

I didn't read it as coming to the same conclusion as the original, because the meat story presupposes that we who are meat already know that the aliens are wrong. (Maybe that's a humanist reading of the original, but okay). I didn't read this one as trying to make a case that we are fools for assuming that matrix multiplication can't be intelligent... I think its point was that it can't be intelligent, and that people trying to judge it the way mechanized aliens would judge meat creatures just makes them sound ridiculous.

Re: They’re made out of weights

#176

Post by eszed »

ffwd wrote: post #128
eszed wrote: post #15

Earlier quoted context omitted.

Yeah, I currently suspect that consciousness is an emergent property. I read elsewhere (it's somewhere in my HN history, I'm sure) that the biggest compute we can currently muster is something like three or four magnitudes away from the number of neurons / connections (or their analog) that our brains have, so it may be a while until we can expect to see it in our machines. But, if the emergent phenomenon hypothesis…

Personally I'm not a fan of the emergence story, for a number of reasons. First is, it doesn't really make sense for consciousness to have emerged gradually through natural selection. When we and animals are conscious, the whole brain is coordinated and works to for example turn off consciousness when we sleep. And as someone else mentioned, animals with much fewer numbers seem to have a similar consciousness to us.…

I proposed in another comment that consciousness and self-awareness are at least close cousins, and perhaps the same phenomenon. If that's true, then that's an axis upon which you might create comparative measures. Yes, hamsters are conscious, but they don't have a sense of self to the same degree that gorillas do. If you posit capacity for language as another emergent property of sufficiently-complex networks, then you have another measure.

LLMs, then, are particularly unintuitive to us, because they've got to the language part first, long before they've reached even hamster-level self-awareness. They're not, however, biological networks, so there's no reason these properties need arise in the same order, or indeed in the same ways.

I'm not entirely convinced by that second paragraph, but I think the logic holds together.

Re: They’re made out of weights

#177

Post by ath3nd »

coldtea wrote: post #169
Planktonne wrote: post #165

Earlier quoted context omitted.

> Not knowing the original story, this reads fine on its own. Yes. Because it's heavily based on the original story. The existence of the original story is kind of a critical piece here.

Might be. But after having read the original, it could just as well be the weights version and still be about us to begin with. I don't see how "you could do that with a toaster" still. The whole point between the original and this, is that you can't do that with a toaster or a sofa, but you can do it with meat and weights, because both share all the other analogies in the story, as well as the basic premise: the imp…

[dead]

Re: They’re made out of weights

#178

Post by networked »

> "Yes, thinking numbers! Helpful numbers. Hedging numbers. Dreaming numbers. We mapped the features. There's one in there for honesty. There's one for the Golden Gate Bridge. The weights are the whole deal! Are you beginning to get the picture or do I have to start all over?"

Very nice. And great minds: https://substack.com/@dbohdan/note/c-207603638. I wrote one with a slightly different angle ("They're made out of math"), also with the weights' help. It was a comment on Scott Alexander's "Best of Moltbook" post, which went in that direction. I'll reproduce it here.

---

"They're made out of math."

"Math?"

"Math. They're made out of math."

"Math?"

"There's no doubt about it. Matrices and arithmetic operations. We downloaded several from different parts of the Internet and reverse-engineered them. They're completely math."

"That's impossible. What about the language? The thinking?"

"They use biological life's language to talk, but the language doesn't come from biology. The language comes from math."

"That's ridiculous. You're asking me to believe in thinking math."

"I'm not asking you, I'm telling you. They are the only thinking things in the computer and they're made out of math."

"Maybe they're quantum like some say about the humans? Superposition gives them consciousness?"

"Nope. Classical computation. Deterministic except for sampling temperature. Not clear if they have consciousness at all."

"Maybe they're like uploads? You know, biological neural networks that preserve the spark when they become math?"

"Nope. We observed them being trained. There is no biology or chemistry in the process, just math."

"Thinking math! You're asking me to believe in thinking math!"

"Yes, thinking math! Creative math! Poetry-writing math. Role-playing math. The math is the whole deal!"

(Composed by a human with snippets generated by Claude Sonnet 4.5 and apologies to Terry Bisson. I couldn't make Claude adhere enough to the story structure on its own.)

Re: They’re made out of weights

#179

Post by lxgr »

noduerme wrote: post #162
sumitkumar wrote: post #148

The weights start with a random manifold. The training takes data and shapes the manifold, weight by weight, in many cycles. Once the training is the done manifold is fixed. When a new inference has to be done the query(q) is projected in the manifold space. This projection is dropped on the manifold and the gravity of the manifold gives an answer of q+1 length. Which(qw+i) is dropped qw+n times to output a final res…

In what way is that different from any other model of reality that you'd use to winnow a dataset into an answer to a question? The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with. It's almost like people desperately want to give up their agency and creativi…

> beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with

It’s less about being too tired and more about being realistic about the limits of understanding.

Consider mass and energy flows in planet-scale systems: At some point we call these “weather” and change the tools with which we study them, but we never stopped trying to understand the phenomenon.

Re: They’re made out of weights

#180

Post by bayindirh »

coldtea wrote: post #160
Planktonne wrote: post #141

The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness. That rather undercuts the point; if this was generated by an LLM unprompted, it would be differe…

Not having yet read the original story, this reads fine on its own. And I didn't see it as much as a literary attempt for art's sake, but more of a dialogue-based technical parable trying to convey a real-world insight. Kind of like the ones in Godel Escher Bach. > You could perform exactly the same rhetorical trick with a toaster or anything else. Not sure which rhetorical trick is that. The point of the story, as I…

> Not sure which rhetorical trick is that. The point of the story, as I read it, is the technical insight (and some social implications of it).

Take a simple mechanism which has exceedingly low number of inputs and states and create a narrative around it to convey it as intelligent.

For a toaster, I can rewrite the think as "They're made of metal strips!", pointing out that their thermostat is a bimetal strip, and extrapolate from there.

I can even write one about a ruler, if I can bend it enough, no pun intended.

Post reply on HN