Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They’re made out of weights

maxleiter.com

Post reply on HN
Print view View original

641–650 of 739 posts

Re: They’re made out of weights

#641

Post by wnmurphy »

darepublic wrote: post #491
sumitkumar wrote: post #148

The weights start with a random manifold. The training takes data and shapes the manifold, weight by weight, in many cycles. Once the training is the done manifold is fixed. When a new inference has to be done the query(q) is projected in the manifold space. This projection is dropped on the manifold and the gravity of the manifold gives an answer of q+1 length. Which(qw+i) is dropped qw+n times to output a final res…

It's like a giant plinko board where the shape of the original disc guides how it falls through the apparatus, and the apparatus has been tuned so that different discs end up in the exits we want them to

This is the exact analogy I use to explain how models work to laypersons as well.

Re: They’re made out of weights

#642

Post by fc417fc802 »

kelseyfrog wrote: post #594
fc417fc802 wrote: post #591

Earlier quoted context omitted.

I agree that a monist can have a map. But you said "consciousness exists on the map" which (AFAICT) would imply the map to be "real" else how would something happen on it as opposed to being an emergent property of the contents of the territory? For example temperature is an emergent property of matter, it does not happen on the map, do you agree? Assuming you do, what would it mean for temperature to happen "on the…

Temperature is a model. It exists on the map.

Temperature is a classic example of an emergent phenomenon. It can be physically measured using (for example) a mercury thermometer which is an exceedingly simple construction. It is a real thing. You can touch a hot or cold surface and feel it for yourself, no map required.

Anyway even if you don't like the example I chose can you see the point I was trying to make? What would it mean for a quantifiable phenomenon to happen on the map as opposed to happening in the territory? How would it interact with the world (ie the territory)? Would it not necessarily imply dualism?

Re: They’re made out of weights

#644

Post by bfung »

jrahmy wrote: post #97
bfung wrote: post #82

Earlier quoted context omitted.

And you know what the tokenizer is made of? Weights.

A tokenizer is a deterministic string-matching program, it's not made out of weights in the same sense as a neural network itself.

How does one choose what sequence of bytes constitutes a token?

Re: They’re made out of weights

#645

Post by bogdanoff_2 »

aspenmartin wrote: post #414
jimbokun wrote: post #396

Earlier quoted context omitted.

It’s the opposite. Descartes made clear that subjective experience is the ONLY thing we know. Everything else is theories to explain the phenomena we subjectively experience. We theorize that there is a physical world and other beings like us having similar subjective experiences, because that seems the best explanation for our subjective experiences. But we might be living in the Matrix, with all the people we think…

And this has some bearing on the debate about whether these systems do or could in the future exhibit something similar?

If we go along with the though that our own subjective experience is the only thing we truly know, and that we cannot really know if any other humans are having the same experience (and any belief of that sorts is purely extrapolation), then there isn't a fundamental difference between LLMs and "other humans" in terms of whether or not they're "conscious". Sure, it appears more likely that "other humans" are real conscious beings, but there's no fundamental difference.

>whether these systems do or could in the future exhibit something similar?

I think the whole discussion is based on the idea that consciousness isn't something you can "exhibit". (Tell me, how can you "exhibit consciousness"?)

Re: They’re made out of weights

#646

Post by mahogany »

f_klem wrote: post #161

After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind. What all these the…

> What all these theories have in common is the underlying belief that our brain/mind works as the machines we build

It’s interesting to note that this is not a new phenomenon. If you read writers from the past, they were always comparing the body or the brain to whatever the modern machine, or idea of a machine, was at the time, whether a steam engine or automata.

Re: They’re made out of weights

#647

Post by cortesoft »

runarberg wrote: post #639
cortesoft wrote: post #630

Earlier quoted context omitted.

Quantum Mechanics and Chaos Theory don't preclude reductionism, and I wouldn't say it has fallen out of favor as a whole. Certain types certainly have, but not the overall idea. Also, nothing about the idea of the mind only being made up of physical processes means things have to be deterministic.

It doesn’t preclude it correct, however it provides a pretty strong examples of where reductionism is lacking, which what I believe has turned a lot of philosophers of science against pure reductionism (I am probably oversimplifying here. An expert science historian [which I am not] could probably write a whole book about why reductionism is not as popular today as it was in the 1970s). There are a whole lot more phy…

I feel like these are separate things... neural activity being necessary but not sufficient for consciousness does not mean reductionism is wrong, it just means the fundamental building block is not a neuron.

It might not even be possible to fully understand the physical mechanisms that underlie consciousness, but that doesn't mean there has to be something more than physical mechanisms.

Re: They’re made out of weights

#648

Post by mandymoorefan »

Lplololopo wrote: post #604
mandymoorefan wrote: post #536

Earlier quoted context omitted.

This seems to be a little naive about how humans consume the benefits we create in society. "Let people do old handcraft jobs, let them do more sports, let them read more, let them write and make more. Let them enjoy nature." Very nice thoughts. You know we all could do this today without "burning it down"? Get in your pod, eat your slop, and watch your screen is where this is headed. "I want freedom to do what i wan…

Have you heard of property tax? We have that here. I also do not earn currently a farm, im planing to buy one, i'm still 100k short though. Its not me creating the misery, its our capitalistic system.

Good luck in life.

Re: They’re made out of weights

#649

Post by sarchertech »

raincole wrote: post #493
sarchertech wrote: post #388

Earlier quoted context omitted.

I don’t think the OP is discussing mind-body dualism. There are many materialists who believe that consciousness maybe a non-computational process. But it is worth pointing out that something like 80% of the world (it fluctuates depending on the survey but its around that) believes in some non-physical spirit, life force, or soul. It’s a very HN bubble thing to start a discussion with the assumption that everyone mus…

I acknowledge and understand that many people believe in the existence of souls. But I hope they just say it rather than try to wrap the concepts in layers of jargons.

Again I don’t think the person you were talking to was doing that.

But one of the reasons that people might do that is because on a forum like hacker news they’ll be attacked as a lunatic. Just look at some of the comments on this article. I saw one where someone said that dualism is so ridiculous that it can’t even be considered philosophy.

Re: They’re made out of weights

#650

Post by runarberg »

cortesoft wrote: post #647
runarberg wrote: post #639

Earlier quoted context omitted.

It doesn’t preclude it correct, however it provides a pretty strong examples of where reductionism is lacking, which what I believe has turned a lot of philosophers of science against pure reductionism (I am probably oversimplifying here. An expert science historian [which I am not] could probably write a whole book about why reductionism is not as popular today as it was in the 1970s). There are a whole lot more phy…

I feel like these are separate things... neural activity being necessary but not sufficient for consciousness does not mean reductionism is wrong, it just means the fundamental building block is not a neuron. It might not even be possible to fully understand the physical mechanisms that underlie consciousness, but that doesn't mean there has to be something more than physical mechanisms.

I didn’t say it was wrong, I said it was lacking and unpopular among modern philosophers of science. If you want to explain consciousness as arising from interaction with the environment (like Ted Chiang did in yesterday’s article) holism is a much better approach, same if you want to use evolutionary explanations, like Daniel Dennett did at the turn of the century.

I think reductionism is simply to limited of a philosophical framework for modern science and philosophy.

Post reply on HN