Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They’re made out of weights

maxleiter.com

Post reply on HN
Print view View original

451–460 of 739 posts

Re: They’re made out of weights

#451

Post by cortesoft »

skydhash wrote: post #315
Lplololopo wrote: post #230

Earlier quoted context omitted.

Compression is the reason why these Models are able to learn and understand. My brain is doing the exact same thing. I learned enough to compress concepts like a bike and what a bike does and for what i can use a bike. Ask a LLM and it will answer you similiar to humans. Blind people learn concepts of bikes too and in a smiliar way: by description. LLMs just have so much data in form of text available and are able to…

> Blind people learn concepts of bikes too and in a smiliar way: by description. And by touch and sound. And maybe some were daring enough to drive one, or unlucky enough to get hit by one. But have way more input than just texts.

LLMs also have other inputs, like audio and images. They get encoded (just like a human eye encodes an image) and passed to the weights.

Re: They’re made out of weights

#452

Post by mjg2 »

Planktonne wrote: post #141

The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness. That rather undercuts the point; if this was generated by an LLM unprompted, it would be differe…

Thank you for your point. I don't understand why half these comments are taking this blog post seriously when it ends with "Weights helped me draft and proof this story."

> Weights helped me draft and proof this story.

Any HN reader here now, I encourage you to read the original ( https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s... ) in one sitting, go about your day, then read it again. Maybe make some notes on personal critical questions.

Now read the post's topic again ( https://maxleiter.com/blog/weights ) and reflect on the prior fact that weights helped [the author] draft and proof this story.

My reaction (and I'm sorry that it is harsh according to some) is that there is no intelligence found in either the author nor their tool. This is extreme navel gazing, based in science fiction, wanting (wishing) to believe those stories to be true.

I'm skeptical of AI sentience because we must do our due diligence, not because it's impossible. Skepticism is the only respectful approach because to grant sentience is a step away from granting rights.

Re: They’re made out of weights

#453

Post by Lplololopo »

cootsnuck wrote: post #422
Lplololopo wrote: post #232

Earlier quoted context omitted.

Agency? What are you talking about? I want freedom. I want freedom to do what i want and not sitting in front of a computer and coding for some company. Please AI lets burn down knowledge work and labor work. Lets create so much stress to our society that we start rethinking what works mean. Lets redefine work into discovering the world again. Let people do old handcraft jobs, let them do more sports, let them read m…

> Lets redefine work into discovering the world again. Let people do old handcraft jobs, let them do more sports, let them read more, let them write and make more. Let them enjoy nature. Why leave something so important up to what AI does or doesn't do?

Because capitalism doesn't allow for that.

Only a fundamental change to our society will allow this for the masses when pressure to the rich skyerockets

Re: They’re made out of weights

#454

Post by ozgung »

Avicebron wrote: post #441
ozgung wrote: post #431

Earlier quoted context omitted.

> I highly recommend people in the AI research space should read philosophy and modern linguistics. On the contrary, I highly recommend people in Philosophy of mind and linguistics should start reading AI research papers because their theories and ideas are highly outdated, even ancient. Your books are from 1927 and 1972 respectively and Turing's article is from 1950s. And they are relatively new with respect to othe…

Yikes dude. You may need to expand your horizons a bit.

That's my point. Everything from 1927 is already in plain sight and a part of the current public knowledge. Horizon can only be expanded at the cutting edges.

Re: They’re made out of weights

#455

Post by LEDThereBeLight »

teiferer wrote: post #92
zkmon wrote: post #88

They are made out of data bits (memory) and switching bits (transistors/compute). Bits are made out of electric voltage and no voltage. Voltage is made out of flow of positive electric charges. Charges are made out of quarks ...

I have never thought of such a distinction between "bits" into "data bits" and "switching bits". From a circuit perspective that makes kinda sense, but from the abstract "bit" perspective, the "switching bit" is a mechanism that operates on bits which in the end are also data. In other words there is only one type of bit: the data bit, and the switching comes on top of it.

It’s just a matter of perspective. Procedures and memory are the same, and they’re also different, depending on how you want to look at it.

Re: They’re made out of weights

#456

Post by layer8 »

IsTom wrote: post #308
layer8 wrote: post #273

Earlier quoted context omitted.

The internal experience has bearing on physical reality right here, because without it you wouldn’t have written about it and caused these words to appear on my screen in physical reality. It affects your thinking, and hence your actions and utterances in the physical word. For reasons like that, I don’t think that P-zombies are possible.

Internal experience affects movement of physical molecules in what way?

Internal experience is an input to your mind (that’s how you know about it), and what your mind perceives affects what it does. The better question is: how can what you experience possibly not affect your actions?

Re: They’re made out of weights

#457

Post by cortesoft »

noduerme wrote: post #162
sumitkumar wrote: post #148

The weights start with a random manifold. The training takes data and shapes the manifold, weight by weight, in many cycles. Once the training is the done manifold is fixed. When a new inference has to be done the query(q) is projected in the manifold space. This projection is dropped on the manifold and the gravity of the manifold gives an answer of q+1 length. Which(qw+i) is dropped qw+n times to output a final res…

In what way is that different from any other model of reality that you'd use to winnow a dataset into an answer to a question? The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with. It's almost like people desperately want to give up their agency and creativi…

> The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with.

It’s funny, because I thought you were talking about humans here when you wrote this. We know some things about how our bodies encode information that is sent to the brain, and we know some things about how neurons receive information and act on it, but after that we get too tired and give up on how the brain works and treat it like a miracle.

It’s like we desperately want to believe our consciousness is not just electrical impulses in our brain, and we want to ascribe agency and uniqueness to the physical processes going on in our head.

Re: They’re made out of weights

#458

Post by jenniferhooley »

Lplololopo wrote: post #232
noduerme wrote: post #162

Earlier quoted context omitted.

In what way is that different from any other model of reality that you'd use to winnow a dataset into an answer to a question? The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with. It's almost like people desperately want to give up their agency and creativi…

Agency? What are you talking about? I want freedom. I want freedom to do what i want and not sitting in front of a computer and coding for some company. Please AI lets burn down knowledge work and labor work. Lets create so much stress to our society that we start rethinking what works mean. Lets redefine work into discovering the world again. Let people do old handcraft jobs, let them do more sports, let them read m…

"I want freedom to do what i want and not sitting in front of a computer and coding for some company."

"Please AI lets burn down knowledge work and labor work"

"Let people do old handcraft jobs."

So many presuppositions about what people want to do.

As a child I spent a lot of time programming and doing "knowledge work" because it's fun - I don't enjoy "old hand-crafted jobs". Sure, let's definitely destroy capitalism in it's current state I suppose. But I find people like you who hate knowledge-work/coding and think everyone else must feel the same and only do it for the money a bit out-of-touch.

Re: They’re made out of weights

#459

Post by wredcoll »

Planktonne wrote: post #303
iainmerrick wrote: post #301

Earlier quoted context omitted.

No, the original "they're made out of meat" works because we're confident that we are in fact intelligent and conscious, despite how ridiculous and unlikely the author manages to make it sound. "They're made out of weights" works precisely because LLMs really do have this mysterious property that they seem somehow intelligent even though nobody can explain exactly why, and there's active debate over whether they coul…

I disagree; it works in the original because it's the unlikely consciousness that produces the text itself; in the LLM case, it's produced by the likely consciousness. "Imagine how other intelligences would view us", written by us, hits a lot less hard when it's "imagine how our intelligences view a thing we are claiming is intelligent", not written by it.

> Imagine how other intelligences would view us", written by us, hits a lot less hard when it's "imagine how our intelligences view a thing we are claiming is intelligent", not written by it

This is well put. We don't need to imagine how a human views a llm because we can ... just do that. Everyone capable of reading the story is also capable of thinking about how they feel abouy llms that exist right now and you've probably used.

The trick of the original story is inverting your perspective, moving your view point fron yourself to an "other" (which I think is a primary qualifier for most good fiction).

Re: They’re made out of weights

#460

Post by advisedwang »

gobdovan wrote: post #83

You can take the weights and model description, write them down on a notebook, then, by hand, compute the next token. Try to do the same with meat.

AKA the "Chinese Room" argument. Ultimately this argument boils down to the idea that consciousness can't be something mechanistic, which is intuitively appealing but still just an assertion.
Post reply on HN