Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They’re made out of weights

maxleiter.com

Post reply on HN
Print view View original

41–50 of 739 posts

Re: They’re made out of weights

#41

Post by [deleted] »

hippich wrote: post #24
kridsdale3 wrote: post #22

Earlier quoted context omitted.

Time is entropy unfolding as things with nonzero temperature do what they do. Psychological time is your own weights being updated in response to stimuli and internal processing. When there isn't anything interesting happening, no updates are needed, and you don't perceive much time. That's why there's a logarithmic effect on the "density" of time as you age.

This is actually something I was always confused about. If nothing interesting happens as we get old, it should be boring and as result, slow slog. Yet it feels like time accelerates as I get older.

[deleted]

Re: They’re made out of weights

#42

Post by pixl97 »

hippich wrote: post #24
kridsdale3 wrote: post #22

Earlier quoted context omitted.

Time is entropy unfolding as things with nonzero temperature do what they do. Psychological time is your own weights being updated in response to stimuli and internal processing. When there isn't anything interesting happening, no updates are needed, and you don't perceive much time. That's why there's a logarithmic effect on the "density" of time as you age.

This is actually something I was always confused about. If nothing interesting happens as we get old, it should be boring and as result, slow slog. Yet it feels like time accelerates as I get older.

Myself I believe the opposite. The brain itself is one of the most powerful filters that exists, and it attempts to be lazy and fill things in and compresses away the common. All that time we're not doing anything novel just gets compressed away to almost nothing. When you're a kid and seeing new things, feeling new things, learning new things you can't compress that away.

Re: They’re made out of weights

#43

Post by kevin_thibedeau »

eszed wrote: post #15
kami23 wrote: post #12

This read like poetry to me. Thank you for sharing it. I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness. For a little bit I was working on having linguistics based evals for a kaggle competition. My challenge was whether or not I could mask things well enough to not trigge…

Yeah, I currently suspect that consciousness is an emergent property. I read elsewhere (it's somewhere in my HN history, I'm sure) that the biggest compute we can currently muster is something like three or four magnitudes away from the number of neurons / connections (or their analog) that our brains have, so it may be a while until we can expect to see it in our machines. But, if the emergent phenomenon hypothesis…

Our machines won't have biological systems driving their needs which in turn fuel behaviors like desire and planning for the future. They may imitate them but it won't be innate.

Re: They’re made out of weights

#44

Post by modzu »

samrus wrote: post #39

I have to agree. It is messed up that transformers can just talk, and it been pretty normalized. We are only talking about the impact they will have and whether they can do what people say they can, but we arent talking about how crazy it is that they can talk

if youve ever seen a pile of wrinkly mush and wondered.. pretty damn crazy too

https://web.mit.edu/people/dpolicar/writing/prose/text/think...

Re: They’re made out of weights

#45

Post by lotyrin »

kevin_thibedeau wrote: post #43
eszed wrote: post #15

Earlier quoted context omitted.

Yeah, I currently suspect that consciousness is an emergent property. I read elsewhere (it's somewhere in my HN history, I'm sure) that the biggest compute we can currently muster is something like three or four magnitudes away from the number of neurons / connections (or their analog) that our brains have, so it may be a while until we can expect to see it in our machines. But, if the emergent phenomenon hypothesis…

Our machines won't have biological systems driving their needs which in turn fuel behaviors like desire and planning for the future. They may imitate them but it won't be innate.

I think those are things human consciousness has, not is.

Re: They’re made out of weights

#46

Post by agumonkey »

hippich wrote: post #24
kridsdale3 wrote: post #22

Earlier quoted context omitted.

Time is entropy unfolding as things with nonzero temperature do what they do. Psychological time is your own weights being updated in response to stimuli and internal processing. When there isn't anything interesting happening, no updates are needed, and you don't perceive much time. That's why there's a logarithmic effect on the "density" of time as you age.

This is actually something I was always confused about. If nothing interesting happens as we get old, it should be boring and as result, slow slog. Yet it feels like time accelerates as I get older.

It's coherent. More newness => more memories per period ~ slower to go through. Less newness => less memories ~ nothing to go through (faster sense of time)

Re: They’re made out of weights

#47

Post by slopinthebag »

pixl97 wrote: post #40
slopinthebag wrote: post #30

Earlier quoted context omitted.

This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I assume you don't believe consciousness is a physical property in the brain, so what entity is actually experiencing that consciousness? Or, what does it even mean to experience consciousness? Or are these not even the right questions?

Is a video game a physical property of a computer?

Yes

Re: They’re made out of weights

#49

Post by agumonkey »

BobbyTables2 wrote: post #28
kami23 wrote: post #12

This read like poetry to me. Thank you for sharing it. I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness. For a little bit I was working on having linguistics based evals for a kaggle competition. My challenge was whether or not I could mask things well enough to not trigge…

I’ve wondered the same myself, without being a cunning linguist. I understand the math pretty well but still find it crazy that a bunch of matrices can converse in human languages without ever being “taught”. Imagine decoding an encyclopedia written in a foreign language where the characters, punctuation, and grammar are unknown — supplemented by a million other texts the same way. Feels like it should be utterly imp…

My best guess as a noob is that the vector spaces allow for unbounded contextualization. As long as the training set is large enough, it can 'infer' anything.

Proofread has a spot in that space, and layers allow patterns like terminology consistency to be expressed so your query will now tap into a subspace that will infer tokens based on whatever consistency patterns were ingested with proofreading texts.

Re: They’re made out of weights

#50

Post by trick-or-treat »

kevin_thibedeau wrote: post #43
eszed wrote: post #15

Earlier quoted context omitted.

Yeah, I currently suspect that consciousness is an emergent property. I read elsewhere (it's somewhere in my HN history, I'm sure) that the biggest compute we can currently muster is something like three or four magnitudes away from the number of neurons / connections (or their analog) that our brains have, so it may be a while until we can expect to see it in our machines. But, if the emergent phenomenon hypothesis…

Our machines won't have biological systems driving their needs which in turn fuel behaviors like desire and planning for the future. They may imitate them but it won't be innate.

For an LLM, "innate" means "in their training data". So yeah, those things are pretty much innate.
Post reply on HN