They’re made out of weights • Hacker News Board

Re: They’re made out of weights

#261

Post by coldtea » Thu, Jun 04, 2026, 9:41 AM UTC

bayindirh wrote: post #191

coldtea wrote: post #188

Earlier quoted context omitted.

> Take a simple mechanism which has exceedingly low number of inputs and states and create a narrative around it to convey it as intelligent. (...) For a toaster, I can rewrite the think as "They're made of metal strips!", pointing out that their thermostat is a bimetal strip, and extrapolate from there. Doesn't that miss the whole point? You could write "They're made of metal strips!". You wouldn't be able to write…

Oh, no. When you add thermal sensing and offsetting ambient temperature, you can add all kind of "seemingly" intelligent features like feeling the emotions of the bread, and creating the perfect toast without hurting the cute little bread slices, making them perfectly blonde while not showing cruelty to them. They can even adapt to their environment and the characteristics of the bread even with simplest of mechanism…

>Oh, no. When you add thermal sensing and offsetting ambient temperature, you can add all kind of "seemingly" intelligent features like feeling the emotions of the bread, and creating the perfect toast without hurting the cute little bread slices, making them perfectly blonde while not showing cruelty to them.

Yeah, that's called "stretching it beyond any recognition".

You could do that. It will have none of the effectiveness or resonance of the two stories.

Re: They’re made out of weights

#262

Post by layer8 » Thu, Jun 04, 2026, 9:41 AM UTC

JauntyHatAngle wrote: post #186

why_at wrote: post #157

Earlier quoted context omitted.

I was having a hard time pinning down what bothered me about this but I think you put it pretty well. It draws an analogy between us and the skeptical aliens in the original story which feel silly to us, so the obvious implication is that we're being as silly as they were. But it doesn't really give a reason to accept the analogy, it just asserts it. There's a big difference between a whole civilization and a piece o…

The bit that lost me quite early in the piece was >"A side effect. You're asking me to believe in sentient weights." Huh? Did I miss that logical jump? Genuine question, maybe I'm not clueing into something here.

I noticed that jump too, but I also don’t think the interlocutor is supposed to be 100% logical and rational. The aliens in the original weren’t either.

Re: They’re made out of weights

#263

Post by toilet » Thu, Jun 04, 2026, 9:47 AM UTC

kami23 wrote: post #12

This read like poetry to me. Thank you for sharing it. I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness. For a little bit I was working on having linguistics based evals for a kaggle competition. My challenge was whether or not I could mask things well enough to not trigge…

>I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness.

Then you shouldn't have dropped out of your linguistics programme.

Re: They’re made out of weights

#264

Post by coldtea » Thu, Jun 04, 2026, 9:48 AM UTC

Planktonne wrote: post #211

coldtea wrote: post #190

Earlier quoted context omitted.

> You could go through and change all the points in the original story to be about a toaster instead. It would require you to edit text, but there is no barrier to doing that. Yes, there very much is a huge barrier. The copy and the original both keep the same subject matter: intelligence/human-like behavior. The toaster doesn't. We could do edits, and the story (original or copy) would lose all its potency. The surp…

I think you're misunderstanding me. There's no barrier to writing this story but about a toaster: you literally just type different words. Writing a story in which a toaster in conscious is not even new [1]. That is entirely separate to whether or not it's meaningful to write a story about a conscious toaster. Again, expressing an idea in fiction is not related to the accuracy of that idea. What makes the original st…

>I think you're misunderstanding me. There's no barrier to writing this story but about a toaster: you literally just type different words.

And you're misunderstanding me. There is no practical barrier to writing this story about a toaster. But there is a conceptual one. It would be bullshit, that has little resonance, and little connection, as the whole point of the meat and weights stories is the "lowly substrate -> surprisingly intelligent human (or human-like in the case of the weights version) behavior".

Nobody thinks a toaster's behavior as surprising or intelligent or human-like to begin with.

So you'll just be stretching the analogy beyond all recognition, with little to no payoff.

>Writing a story in which a toaster in conscious is not even new

It's not, but it has nothing to do with the meats or weights stories' point, as it's just an arbitrary choice, like making the candles and teapot sentient in Beauty and the Beast.

Whereas in the meat and weights stories the whole point is the surprise from us already seeing the human or human-like sentience of the thing, and comparing it to the "dead" substrate.

Re: They’re made out of weights

#265

Post by coldtea » Thu, Jun 04, 2026, 9:49 AM UTC

Planktonne wrote: post #213

maxbond wrote: post #184

Earlier quoted context omitted.

I don't think you understood the point of the story. It's not that LLMs or agents are conscious, it's that our dismissal of the possibility is reflexive and uninformed. Personally I think anyone who has made their mind up about whether or not LLMs/agents are or can be conscious has done so before the evidence is in. The story does not assert that search and replacing "meat" with "weight" makes them conscious through…

No, it's a story. Stories can be about real things (the original story is about a real consciousness, as shown by it being produced by that consciousness), but they aren't automatically just because they're good stories. You're falling into the trap of assuming that a good presentation--being convincing--is the same as being truthful.

I don't think you understand how stories work. Or what makes these two stories tick.

Re: They’re made out of weights

#266

Post by layer8 » Thu, Jun 04, 2026, 9:49 AM UTC

tasuki wrote: post #260

Planktonne wrote: post #141

The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness. That rather undercuts the point; if this was generated by an LLM unprompted, it would be differe…

I feel the original story contained in it the whole point of this one, and then some. This did not add anything, just rephrased it so rather than humans viewed through the pov of aliens it's LLMs viewed through the pov of humans. Well, we are the humans, so surely we do not need to learn about this point of view?

It lets us consider the relativity of viewpoints. The point of the original isn’t just how humans might be viewed from the perspective of aliens. It’s that the apparent strangeness of something depends on the viewpoint and on expectations. The present story is not supposed to stand on its own like the original, it’s supposed to be considered in light of the original. In the original, thinking meat appears strange to the aliens because of their viewpoint and expectations. The present story thus points out by analogy that thinking LLMs might seem strange to us merely due to our viewpoint and expectations.

Re: They’re made out of weights

#267

Post by Joeri » Thu, Jun 04, 2026, 9:55 AM UTC

IsTom wrote: post #139

petra wrote: post #123

Earlier quoted context omitted.

Regarding consciousness , I like the explanation by neuroscientist Ramachandran: https://www.edge.org/3rd_culture/ramachandran07/ramachandran... In short, as far as I can remember: evolutionary, it makes sense to understand other humans, to feel what they feel(empathy - the mirror neurons system), and simulate their thinking and feelings. And once we have those systems, we can also use those on ourselves. And that's…

This way of thinking can only explain externally-visible parts of consciousness. It does nothing to address internal experience of being conscious and qualia. I don't think the internal experience has any bearing on physical reality (P-zombies would act the same externally) which makes this something outside of the realm of currently understood physics.

Internal to what? The brain is not a monolithic thing, it is different parts communicating and interconnecting. When the connection between the halves is cut, the person objectively becomes two people, but still experiences and presents themselves as one. Observing ourselves is just one part of the brain responding to another, or theorizing on past behavior. There is probably no actual introspection going on inside of the human brain, only the perception of one.

Re: They’re made out of weights

#268

Post by leothetechguy » Thu, Jun 04, 2026, 9:55 AM UTC

>Weights helped me draft and proof this story.

I'm suprised no one talks about this. AI Art isn't Art. AI Poetry isn't Art. And I'm tired of it. I know hacker news isn't the best place to complain about that but still... I'm not gonna read something somebody didn't put in the effort to write on their own. Especially not Poetry.

Re: They’re made out of weights

#269

Post by adaml_623 » Thu, Jun 04, 2026, 9:56 AM UTC

darkwater wrote: post #111

throwaway173738 wrote: post #74

Earlier quoted context omitted.

I think consciousness is going to turn out to be very challenging to define rigorously enough that we can test for its presence or absence. Emergent or not, the question is how do you determine when it has emerged? Is it a quantity or an attribute? Discrete or continuous? Does it have a finite or infinite range? We can all agree on what color something is, but we can’t describe the color a priori, only by example. I…

I like the color example, but colors can be reproduced at will by trial and error and by observing the results (and collectively compare them to our shared experience). Why cannot this be applied to consciousness as well? I mean, it's surely much more difficult to do compared to colors but... impossible?

You and previous comment seem to agree that trial and error/shared experience can determine if consciousness has emerged. And this might/will be challenging.

Previous comment used the word "anticipate" and I think they mean that we won't know in advance before we run the trial and error process.

When they say "deep trouble" I assume they mean because creating a non-friendly conscious AI might pose an existential risk for humanity.

However there is also the ethical issue of creating a consciousness and then destroying/murdering it.

Re: They’re made out of weights

#270

Post by ACCount37 » Thu, Jun 04, 2026, 9:57 AM UTC

f_klem wrote: post #161

After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind. What all these the…

I don't think cognitive linguistics has that much to say about AI nowadays. Let alone philosophy.

The biggest contributions from linguistics are probably "human languages mostly have statistical regularities rather than hard rules" and "the sum of data humans get from birth to language acquisition is insufficient to learn a language from scratch". Which LLMs already work with, and work around, respectively. From there, nothing.

And philosophy just exists to be a distractor. "Subjective experience" is too subjective to matter in practice. "Task performance" is measurable, "consciousness" isn't. "Agency" is something an LLM in a tool calling loop, a rat in a maze and a human in an office tower may or may not have, depending on your favorite definition. Agentic LLMs are years in the making, and that's a product of engineering, not philosophy: "agentic" is whatever gets the job done.

We are yet to discover any physical process whatsoever that can't be represented as mathematical operations and implemented by a Turing machine. So all of that "treating human mind as a machine is wrong" amounts to "human mind must be powered by magic fairy dust" paired with "a functionally similar magic-free replacement is impossible". I'm not about to give much weight to any hypothesis that requires undiscovered magic fairy dust. At least find the hyper-computational magic fairy dust first - not just assume it absolutely must be there because you want the human mind to be unique and special.

Want to know why Turing did what he did? It's because he didn't want to engage with any of that "what is mind" bullshit either. So he proposed actual metrics - measurables that are harder to argue in circles about. Not that it stopped anyone. But at least he tried.