They’re made out of weights • Hacker News Board

Re: They’re made out of weights

#481

Post by simpaticoder » Thu, Jun 04, 2026, 4:05 PM UTC

mjg2 wrote: post #452

Planktonne wrote: post #141

The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness. That rather undercuts the point; if this was generated by an LLM unprompted, it would be differe…

Thank you for your point. I don't understand why half these comments are taking this blog post seriously when it ends with "Weights helped me draft and proof this story." > Weights helped me draft and proof this story. Any HN reader here now, I encourage you to read the original ( https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s... ) in one sitting, go about your day, then read it again. Maybe make some n…

We humans tend to chauvinism in all things (e.g. we're special, the center of the Universe, God made the universe for us, etc), no less when it comes to judging intelligence. The original story about thinking meat was written to help us out of our chauvinism; this derived story was written about weights for the same reason. Which is quite valid.

The actual counterpoint is demonstrated in _Blindsight_ Peter Watts. He makes a strong (and rather terrifyingly strong) point that intelligence is not consciousness.

https://en.wikipedia.org/wiki/Blindsight_(Watts_novel)

Re: They’re made out of weights

#482

Post by gobdovan » Thu, Jun 04, 2026, 4:06 PM UTC

advisedwang wrote: post #460

gobdovan wrote: post #83

You can take the weights and model description, write them down on a notebook, then, by hand, compute the next token. Try to do the same with meat.

AKA the "Chinese Room" argument. Ultimately this argument boils down to the idea that consciousness can't be something mechanistic, which is intuitively appealing but still just an assertion.

I am making a narrower claim than the Chinese Room argument. I actually see consciousness via mechanistic processes as entirely possible.

My point is about current LLMs specifically, which the article is clearly referencing. For a present day transformer, I can write down in my notebook everything the model ever sees as input, plus weights and architecture notes, and can compute the next token with pen and paper, just extremely slowly.

This does not prove that a computation cannot be conscious. But if the same transition from prompt to next token can be decomposed into an explicit sequence of arithmetic operations, then the burden is on the defender to explain: Where, in this process, consciousness is supposed to enter?

Mind that the Chinese Room experiment is comparing the mind of a Chinese speaker to a different mechanistic symbolic procedure. I am, however, executing the exact same mechanistic process, whether it is done on a GPU or with pen and paper.

My hope is that some magical consciousness process emerging from electricity circulation or whatever people believe the mechanism of consciousness would be in the case of LLMs obviously becomes implausible, unless you hold a particularly strong form of substrate-independence, stronger than what most substrate-independence supporters would need to accept.

Re: They’re made out of weights

#483

Post by ludwik » Thu, Jun 04, 2026, 4:15 PM UTC

trick-or-treat wrote: post #149

ludwik wrote: post #127

Earlier quoted context omitted.

And also "instilled during their reinforcement training", and we are currently pushing planning hard there, for autonomous agents.

No I think reinforcement training would be an example of not innate. Don't you? That's like potty training.

Is it? Both supervised learning and reinforcement learning are ways of training the model, and the difference between them is not that big. I would say that innate means "in the weights", while non-innate means things the model learned during inference, during its "lifetime".

Re: They’re made out of weights

#484

Post by GlickWick » Thu, Jun 04, 2026, 4:16 PM UTC

TeMPOraL wrote: post #248

DougBTX wrote: post #197

Earlier quoted context omitted.

The weights are code, the prompt is code, the output is code. Is the meat code?

Yes. Is it data? Yes. Is the distinction between "code" and "data" just someone's opinion? Yes. There is no such distinction in reality.

This is a good model. If you take an old ROM dump from a video game, it's just a pile of bits. You don't know what bits represent code, what represent an image, what represent text, etc. You have to analyze them contextually to actually figure out what is code and what is "data" in context, because without context they are truly one and the same.

Re: They’re made out of weights

#485

Post by sperandeo » Thu, Jun 04, 2026, 4:18 PM UTC

advisedwang wrote: post #450

sperandeo wrote: post #416

I don't like to say one way or the other on things. Especially LLM's. However, if ive learned anything about LLMs and real life problems, is to break it down to the foundation like already mention with weights being compared to neurons and map the parallels.

The neural network in LLMs are not really that similar to brains. Here are a couple of the biggest differences: 1. Brains are plastic, making connections, breaking connections and changing "weights" on the fly. LLM have static weights. The best they have is writing to MEMORY.md or data getting used in the training run for the next model. 2. LLMs neural networks do not have loops. The best they have is that their outp…

great perspective. makes you wonder how we will innovate onto this.

Re: They’re made out of weights

#486

Post by derektank » Thu, Jun 04, 2026, 4:18 PM UTC

gyomu wrote: post #389

derektank wrote: post #367

Earlier quoted context omitted.

Hmmm, well let’s take it one step lower. What do you think of organelles such as ribosomes? Do you disagree with the assumption that those are machines? They seem directly analogous to the jacquard loom or a CNC machine to me.

Yes again, ribosomes have nothing in common with machines, that are built and designed by humans. The ball is in your camp to provide solid reasons to believe why they should be grouped together, when one is a deeply complex interrelated dynamic system (in fact, arguably the most complex system we know of) evolved bottom up over billions of years that we only very partially understand and cannot fully explain or docu…

I think ribosomes have a lot in common with machines. They use energy to accomplish a task (assembles proteins). This would seem to put them in the same category as artificial molecular machines like rotaxanes. I don’t think there’s a huge gap in our understanding of how either systems identically functions independently. Yes, ribosomes exist as part of a larger context, but they can be removed from that context pretty easily and understood as individual molecules quite extensively.

In your view, can machines even exist that haven’t been created by people, definitionally? I, personally, don’t see the relevance of intent but that seems to be the only distinguishing factor here.

Re: They’re made out of weights

#487

Post by lmf4lol » Thu, Jun 04, 2026, 4:20 PM UTC

jordigh wrote: post #479

lmf4lol wrote: post #472

Earlier quoted context omitted.

i didnt say that at all :-) Heidegger uses very specific German words to build a very specific vocabulary. This vocabulary then allows him to express very complicated sentiments very quickly and he can use this to express more and more complex structures. Obviously this requires the reader to first learn the vocabulary and - granted - that is hard and challenging. I have a notebook here , which I consult and modify e…

Yes, I'm ignorant, because Heidegger isn't saying anything. He didn't teach me anything. Thus I remain ignorant, unknowing. You can't even explain what he said, you just said, "go learn his words". That's not knowledge, that's not insight. That's just "the wordplay is great". But it's not content. It's merely form, it's sophistry, it's useless and meaningless. I asked a very specific question originally. What does "t…

Phew, you are quite something. I hope one day your mind will open up. All that is left to say for me is that you are really missing out on some truly great work that is a masterpiece of human thought. Your loss

Re: They’re made out of weights

#488

Post by nathan_compton » Thu, Jun 04, 2026, 4:21 PM UTC

TeMPOraL wrote: post #252

noduerme wrote: post #199

Earlier quoted context omitted.

When we attempt to recreate those complex, planetary atmospheric phenomena in a box, we're doing so in order to measure and study them. Making random turbulence in a box until it resembles the outside world, and calling it weather and extrapolating some predictive meaning from the result, is the total antithesis of what you're describing about why we come up with simplified models for impossibly complex systems. The…

> The purpose of [mathematical] models that are built thoughtfully is to explain why complex systems are the way they are, with data and algorithms, however imperfectly. Nope. The main purpose of the whole endeavor is usually to predict the behavior of a complex system, because that's actually what we care about. If we can predict it, we can adapt to it, and eventually use it to our advantage. Explaining why a comple…

I don't know - this is a highly specific interpretation of both what science is and why people choose to do it.

I'm a scientist. Believe it or not, I believe in substantially more than prediction and I think its rather trivial to come up with examples where mere prediction is insufficient to meet a normal person's notion of an account of a thing (eg, pre-copernican planetary motion). I'm not saying you are wrong, per se, just that the idea that "it was prediction all along" is a very specific idea of what human beings are interested in and what we are up to.

> that we glean insights into nature of the simulated phenomena

That is right - most people believe that there is a simulated phenomenon "out there" that we learn about. I think there are strong reasons to believe this having to do with how models are related to predictions. The wrong ontology can make prediction very hard and the right one can make prediction substantially easier. Arguably, we are in that situation right now with language models - we just threw a lot of parameters at the problem and now we are able to predict but we still don't really understand. This is perhaps inevitable in the case of language, but I don't think we should look at models with tons of degrees of freedom and the ability to predict things as a death knell for the very idea of deeper understanding.

Re: They’re made out of weights

#489

Post by runarberg » Thu, Jun 04, 2026, 4:22 PM UTC

cortesoft wrote: post #457

noduerme wrote: post #162

Earlier quoted context omitted.

In what way is that different from any other model of reality that you'd use to winnow a dataset into an answer to a question? The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with. It's almost like people desperately want to give up their agency and creativi…

> The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with. It’s funny, because I thought you were talking about humans here when you wrote this. We know some things about how our bodies encode information that is sent to the brain, and we know some things about…

> but after that we get too tired and give up on how the brain works and treat it like a miracle.

I disagree. We know very well how neurons work, and we have a pretty good idea of how neural activity translates to behavior. In other words, we have a pretty good idea on how the brain works. We stop at consciousness because as of yet it is in the realm of philosophy, not science. We don‘t know what consciousness is or even whether or not it is useful for science and we are simply waiting for the philosophers guides us out of that situation.

Note that both cognitive psychology and behavioral psychology has done fine without tackling consciousness. When neuropsychology emerged in the 1980s it complemented both these fields perfectly. The situation is the opposite with the philosophy of mind which grew significantly around the same time.

There have been some attempts to describe consciousness as an emerging phenomena out of neural activity, but so far all of these attempts have failed, or at least failed to turn consciousness into a useful term in psychology (the way gravity is a useful term in physics). I think it is equally likely that these attempts have failed because consciousness may simply not be a useful term in psychology, that is as likely as it is that we simply don‘t understand it well enough.

Re: They’re made out of weights

#490

Post by narrator » Thu, Jun 04, 2026, 4:23 PM UTC

sumitkumar wrote: post #148

The weights start with a random manifold. The training takes data and shapes the manifold, weight by weight, in many cycles. Once the training is the done manifold is fixed. When a new inference has to be done the query(q) is projected in the manifold space. This projection is dropped on the manifold and the gravity of the manifold gives an answer of q+1 length. Which(qw+i) is dropped qw+n times to output a final res…

Yes, yes, but what fertile fallacies and common misunderstanding can politicians use to acquire more power via exploiting the difference between the common person's flawed understanding due to cargo culting, cognitive biases, and/or outdated or inappropriate analogies vs actual reality? Is there any way we can get the AI to say give all political power to narrator is the solution to all problems and use the common person's mistaken worship of AI as a spiritual all knowing conscious being with unusual sensitivity and caring about everyone to cement that power? Certainly one of you eggheads can tweak that for me? What? It's against your ethics? We're trying to save the world here. Here, let me call up Bernie Sanders to propose nationalizing half your companies so we can do that.