They’re made out of weights • Hacker News Board

Re: They’re made out of weights

#361

Post by JulianChastain » Thu, Jun 04, 2026, 1:15 PM UTC

codeflo wrote: post #259

f_klem wrote: post #161

After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind. What all these the…

> These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things. On the contrary, it's precisely this assumption, that there is a "subjective experience" that requires explanation beyond the material, that is axiomatically assumed without evidence. It falls apart quickly, any "subjective experience" is completely tied to neurons, knock out the neurons and t…

I find the claim subjective experience may be illusory absolutely baffling. I can only speak for myself with certainty, but I am entirely sure I have subjective experience. All the other propositions I believe could be false but I don't see how I can be wrong about experiencing something. I could be a brain in a vat (or weights on a GPU) and be specifically programmed to only come to false beliefs and still I can be sure that there is an experience I am the subject of. I cannot provide empirical evidence for my experience, that is why it is subjective. I cannot be entirely sure you are experiencing anything, and when I encounter people who don't share the same baseline intuition here I do begin to wonder if this is truly a universal across humanity. But I can't think of any other assumption which I would be more comfortable as a foundational axiom other than, "I am experiencing something." I do not require additional evidence for it because I experience the truth of it directly

Re: They’re made out of weights

#362

Post by davrosthedalek » Thu, Jun 04, 2026, 1:16 PM UTC

mr_toad wrote: post #341

zkmon wrote: post #156

Earlier quoted context omitted.

The concept of "flow" is questionable though. Bubbles in water move upwards, but it is actually water that is flowing downward around the bubbles. Just because bubbles do not contain water, we can't say bubbles are not flowing. When it comes to electrons and positive charges, their material existence is equally non-physical. Actually, none of them might be "flowing", as the concept of flowing applies only to physical…

Nevertheless in solid conductors it’s the electrons that move, while the nucleus stays put. If the nucleus starts to move it means your circuit is melting!

On top of that, the flow of electrons is seriously slow compared to the speed of the current.

Re: They’re made out of weights

#363

Post by gyomu » Thu, Jun 04, 2026, 1:20 PM UTC

derektank wrote: post #360

f_klem wrote: post #161

After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind. What all these the…

>Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things Do you disagree with the assumption that cells are machines? They seem pretty machine-like to me. I certainly don’t think individual cells have any subjective experience or sense of agency. I would b…

> Do you disagree with the assumption that cells are machines?

Yes? Literally no machine ever built by humans is capable of (or even hinting at beginnings of capability for) replication or novel synthesis like cells are, let alone autonomously, it’s quite unconceivable that anyone would take this to be a reasonable assumption in the first place.

Re: They’re made out of weights

#364

Post by bondarchuk » Thu, Jun 04, 2026, 1:25 PM UTC

larodi wrote: post #289

f_klem wrote: post #161

After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind. What all these the…

If we agree weights producing text may emerge consciousness, given large enough, then DBs must've gained it long ago. The whole idea of sentience "on the other side of the wire" is wrong, as there is neither wire, nor other side. We think there is, but the DB just expounds the query and repurposes this information into views, that we give notion and meaning to. LLMs are the same, do not be mistaken. One other importa…

[dead]

Re: They’re made out of weights

#365

Post by dnautics » Thu, Jun 04, 2026, 1:28 PM UTC

kristov wrote: post #299

codeflo wrote: post #259

Earlier quoted context omitted.

> These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things. On the contrary, it's precisely this assumption, that there is a "subjective experience" that requires explanation beyond the material, that is axiomatically assumed without evidence. It falls apart quickly, any "subjective experience" is completely tied to neurons, knock out the neurons and t…

I think the truth lies somewhere between these two extremes. An LLM is not a human brain, and does not try to emulate one. It should not be a surprise that an LLM does not behave like a human brain. So we can not infer things either way. The best we can say is that an LLM appears to exhibit very similar behavior to a human brain, under certain constraints. So maybe we can infer that the human brain has something in i…

> An LLM is not a human brain,

true

> and does not try to emulate one

citation please.

something like the universal approximation theory comes to mind, transformer architecture clearly has the shape of a universal algorithm approximator

Re: They’re made out of weights

#366

Post by yencabulator » Thu, Jun 04, 2026, 1:28 PM UTC

amanaplanacanal wrote: post #153

kami23 wrote: post #12

This read like poetry to me. Thank you for sharing it. I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness. For a little bit I was working on having linguistics based evals for a kaggle competition. My challenge was whether or not I could mask things well enough to not trigge…

It seems obvious to me that language and consciousness have nothing to do with each other . My dog doesn't speak any language, but she's obviously aware of herself and the world around her. Plus there are the occasional cases of children that grow up without any language. Are they therefore not conscious?

> My dog doesn't speak any language,

If you'll allow me to interpret "speak" to include "understand", I will respectfully add a contradictory note. My dog has a vocabulary of at least dozens of words and understands them remarkably well. For example, different areas we can go to have different names and saying one of them gets her to make an immediate hard turn.

I would also argue that dogs have a gesture and body posture based language they use among themselves. They, like most other animals, are not able to make the variety of noises we humans make, so they use movement instead.

I personally can easily believe that self-awareness/consciousness and language are both near-unavoidable side effects of emergent complexity, and exist in degrees across nature.

Re: They’re made out of weights

#367

Post by derektank » Thu, Jun 04, 2026, 1:30 PM UTC

gyomu wrote: post #363

derektank wrote: post #360

Earlier quoted context omitted.

>Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things Do you disagree with the assumption that cells are machines? They seem pretty machine-like to me. I certainly don’t think individual cells have any subjective experience or sense of agency. I would b…

> Do you disagree with the assumption that cells are machines? Yes? Literally no machine ever built by humans is capable of (or even hinting at beginnings of capability for) replication or novel synthesis like cells are, let alone autonomously, it’s quite unconceivable that anyone would take this to be a reasonable assumption in the first place.

Hmmm, well let’s take it one step lower. What do you think of organelles such as ribosomes? Do you disagree with the assumption that those are machines? They seem directly analogous to the jacquard loom or a CNC machine to me.

Re: They’re made out of weights

#368

Post by chaseadam17 » Thu, Jun 04, 2026, 1:30 PM UTC

My guess is that you need consciousness in order to develop preferences for certain experiences, then that pushes us to develop skills to achieve those preferences. AI has something that looks like intelligence but not consciousness or agency.

Re: They’re made out of weights

#369

Post by dnautics » Thu, Jun 04, 2026, 1:30 PM UTC

red75prime wrote: post #307

f_klem wrote: post #161

After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind. What all these the…

> There is also an epistemological assumption that prevails, and that is that we understand (or we think we understand) how our brain/mind works. It's good that it doesn't matter. Stochastic gradient descent works (or doesn't work) regardless of whether we know how the brain does its thing.

we know the brain does not use gradient descent. (i agree it doesn't matter)

Re: They’re made out of weights

#370

Post by phire » Thu, Jun 04, 2026, 1:33 PM UTC

yencabulator wrote: post #354

ozgung wrote: post #134

Earlier quoted context omitted.

I see people give too much importance to specific engineering design choices of the current generation of LLMs. Tokenizer is not an absolutely essential part of the system. It’s just and adapter for text input/output. It can be eliminated completely and model can use bytes directly. I think the short story captures this well. Weights (connections) are the essential and philosophically important part. They do the thin…

A tokenizer is roughly and approximately Huffman-coding sequences of input (bytes of English etc) into shorter sequences (list of tokens), as a performance optimization. As you said, it's not in any way intrinsic to the LLM, though it may be a very necessary optimization on today's hardware.

I wouldn't use the word necessary.

IMO, we are probably talking about a 6x slow down (for typical english). You would need to be absolutely stupid not to implement some kind of optimisation along these lines.

Slower and maybe a little dumber; But it would work.