Live data from Hacker News

They’re made out of weights

maxleiter.com

681–690 of 739 posts

Re: They’re made out of weights

#681
post #677
post #675

Earlier quoted context omitted.

I think the conversation derailed a bit. I see also a common pattern of jumping through different topics at different levels (from theoretical to concrete and back), and that is confusing. My original comment was that it seems (and it is actually documented in the books I referenced) that the AI research space builds its claims on assumptions, not on facts, and that those assumptions are flawed. So a nice discussion,…

> I see also a common pattern of jumping through different topics at different levels (from theoretical to concrete and back), and that is confusing. I can probably to clarify it. 1. I don't want to end in a situation when AI deserves human rights but I deny it. There are moral reasons for that, and they are important. 2. I employ a systemic view. Not just arguments for consciousness or against it. I look at the peop…

> So why we call it subjective experience then? Probably it is irrelevant, and the reasons are purely historic... or maybe not. How about the idea of computers experiencing things, just not "subjectively" but rather "digitally"? Or choose any other adjective you like. You are arguing against assumptions, but why you just accept the idea of experience with the assumption that human way to experience things is the only possible way?

We call it subjective because is 'we, ourselves' and not the objects we perceive there in the world, where the 'experience' is manifested, as perception. We do not always codify dichotomies in language.

> No sane AI researcher would stop researching AI to get PhD in linguistics to build a bridge between AI research and linguistics. Probably in an ideal world this shouldn't happen, maybe it is short-sighted behavior of a system, but it is just how things work in our real world.

In the real world, anyone doing a serious PhD thesis will read whatever is necessary to build a proper, sound theory or body of work. This dismissal just makes me think that you don't know how a PhD thesis is done.

> BTW it is a good example of what happens with all the philosophy when shit hits the fan. When possibility of empirical studies arrives, no one bothers themselves with philosophy of things.

From what I see, you never took philosophy seriously. I don´t know how you can then seriously engage in a conversation about philosophy.

> I studied psychology, I've read some linguists (cognitive ones, because they are in an adjacent field), and you see, I don't have trust in either. They do their research, they find some interesting facts and devise interesting theories, but it is all looks more like a chemistry in the first half of a XIX century, than a chemistry after periodic table was created. They can't find their building blocks to create a sound theory.

This is true, cognitive linguistics do not represent a unified theory. Not yet, at least, and maybe it will not come to that. The same happens in psychology, but nobody is dismissing psychology all at once just because of that.

> No, I'm not implying "modeling a thinking process". We don't know what thinking process is. What we observe in our minds is not thinking by itself, it is some kind of a mirror process in our consciousness. The real thinking is hidden from us, but it creates echoes in our consciousness we can observe. If we don't know how thinking works, we can't model it. BTW the reverse is also true: if we can't model thinking, we don't know how it works.

This is exactly what the assumptions I challenge are about. The AI space already declared that they 'know' how such processes work. Or at least, they pretend they do.

> I'm defining thinking more in terms of a problem solving ability. Like psychologists do. Science still doesn't have a good enough definition for thinking, but it has some definitions that a) operational; b) good enough for some limited tasks. "Operational" means that they are defined in terms how to measure what you define, not in terms of modeling some process.

You would agree that 'problem solving' is just a small portion of what 'thinking' constitutes.

> You see, until philosophy methods successfully proved that something is conscious despite it was deemed unconscious before, we can't really know that their methods really work. Maybe they work, or maybe they just mirror our biases and heuristics.

You talk about philosophy as if argumentative biases were completely strange to philosophers. Not the case, and a great deal of XX century philosophy is exactly about that. But if you read the sources on AI research through its own history, you will see how the AI research space is full of such biases and assumptions. Again, my original argument is about that.

> I hadn't read books you mentioned, but in your words I see nothing that can hint that those books have something I don't considered already. So maybe I'll read them in a future, but I wouldn't postpone my engagement in the conversation till I read them.

You don't have to trust me. Just pick them up and think for yourself, do some research. Regarding postponing the conversation, I really appreciate that, but it is really difficult to argue about books you haven't read, especially if they are complex ones. Unfortunately I don't have that much time to explain the books in detail.

Re: They’re made out of weights

#682
post #676

Earlier quoted context omitted.

I am not an expert in panpsychism, but for what I know: 1) the idea that everything has a degree of consciousness proportional to its complexity, introduces the problem of compound consciousness. How do they compose, how is each consciousness contributing to the overall, upper-level one? how is experience explained at the different complexity levels? 2) it is impossible to test whether something is conscious or not 3…

1 is an open question, but it's an easier question than the ones related to emergence, since it reduces from "how do we detect emergence of consciousness" (to which people have settled for behavioral tests, hence the whole AI consciousness kerfuffle) to "how do we correlate physical coupling with assumed (human) consciousness." This framing admits study via sleep/brainwave/dreaming/sedation experiments, though there…

1) I think that is precisely the flaw: it is reductionist.

2) how do you actually measure if a rock has consciousness? if you redefine consciousness as simply some kind of physical manifestation, like radiance or quantum fluctuation (measured as compound at the macro-level), then you will have to redefine what we understand as 'human consciousness' is, otherwise it will be a characteristic so generic that won't be useful at all. Then this new characteristic will suffer from the same original interpretation problem... So at least the panspychism approach is just evading the main problem.

3) I would argue that the only way of thinking about a complex consciousness emerging from a diverse and vast set of things, is considering complexity. If it is not complexity (like, structurally complex things can have simple consciousness and the other way around) then there must be something, material or not, that provides for the emergence of such complex consciousness. This is rather tricky: you will need to postulate some kind of non-physical (or not discovered yet) characteristic that generates consciousness, or you will have to come up with some causal relation between non-related-to-complexity but still-related-to-physicality and complex consciousness, which from our current physics framework might not be possible.

Re: They’re made out of weights

#683

The weights start with a random manifold. The training takes data and shapes the manifold, weight by weight, in many cycles. Once the training is the done manifold is fixed. When a new inference has to be done the query(q) is projected in the manifold space. This projection is dropped on the manifold and the gravity of the manifold gives an answer of q+1 length. Which(qw+i) is dropped qw+n times to output a final res…

In what way is that different from any other model of reality that you'd use to winnow a dataset into an answer to a question? The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with. It's almost like people desperately want to give up their agency and creativi…

> The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle

Like with every invention ever? Cause that's the literal goal and idea?

You take things and then combine them until the ensemble performs a desired abstract function the individual parts alone could not. The end result then is a new thing of its own, arguably indeed a miracle (not in a religious sense of the word).

> and too tired to figure out why it came up with the answer it came up with

There are people whose entire career is this. They work at these companies.

> Factor in that psychology and it looks a lot less like we have invented something useful, and a lot more like we as a species are choosing to quit life en masse.

You're saying this as if people haven't been historically the masters of optimizing out the enjoyment from things. It's what we do. Provide an ultimate solution, and of course you'll extinguish a whole lot of motivation across the board.

Re: They’re made out of weights

#684

Earlier quoted context omitted.

You have in several places repeated that "no one is working on the field to get food", but that's not actually true. While most people in modern society don't work on fields to feed themselves (and others), a small fraction actually do! They feed not only themselves, but all of us. In this future utopia where everyone gets to be a mediocre handmade-furniture maker, who exactly will be the 1-2% needed to work the fiel…

Machines. Or the 1% of people will be allowed to live in the city centers or on the beach or we share the load across peple. Everyone has to work hard and good for 5 years.

who's choosing this lucky 1%? Who's choosing what this 5 years of "hard and good" work looks like? Is 5 years of work a person really enough? Are you aware that farming is predominantly done by machine, and that's why we're down to so few people working in it?

Sure, the idea of a life of leisure and choice sounds great, sign me up. But I think resources are not distributed evenly, the folks with the most power distribute resources have little inclination to distribute them evenly, and even if we did distribute resources as evenly as possible we would still have scarcity, as with your city and beach examples. We will still need people to deal with toilets, to deal with food and so on.

If we've invented the magical cybersyn dream, and we can have central planning done for us, so everything is efficiently allocated and automated, how can you be so sure your personal allocation will be leisure and not ditch digging or bum wiping? I will bet a jelly donut that what you have described will not come to pass in my (or your) lifetime.

Re: They’re made out of weights

#685
post #277
post #202

Earlier quoted context omitted.

> We literally spend almost all day, every day, thinking about ourselves, in terms of our inner self. > That is consciousness. So thinking is consciousness? Can there be consciousness without content? E.g. can I just be conscious of being conscious? If so, consciousness cannot be defined as the thing(s) we're conscious of.

> Can there be consciousness without content? E.g. can I just be conscious of being conscious? Being conscious of being conscious means that there is content. You are conscious of something. It’s a bit like a Gödel statement that quotes itself, that is a statement about itself. It doesn’t mean that it has no content. Thinking isn’t consciousness. Consciousness doesn’t require thinking, it only requires perception. Th…

I don’t think thinking is consciousness, but I don’t see how consciousness operates without thinking.

Consciousness might feel like it’s an isolatable experience, but everything we experience is a result of processing it. Pause a brain and it isn’t experiencing anything,

We can be conscious without overt reasoning, but not without thinking. Thinking about ourselves and the experience is a basic component of the self-awareness in consciousness.

Re: They’re made out of weights

#686
post #661

Earlier quoted context omitted.

There are two things here: one is how an LLM is fundamentally structured and designed, the other is how an LLM distributes and 'lays out' the relationship between inputs and outputs through layers and weights. We might not know how the actual distribution works, but we do know how it i s fundamentally structured and designed -- because we did it. We also know that there is something like a representation system insid…

And knowing that structure is about as meaningful as knowing "a PC consists of a keyboard, on which you type, a screen, at which you look, and a processor, which does things with binary logic". None of that helps you understand how exactly LLMs do what they do. Because it describes an interface, not a mechanism. The inner mechanisms of an LLM are more learned than designed. We know what an LLM does on a low level, bu…

We know how LLMs learn at the fundamental level. What we do not know is the actual dynamic process of encoding embeddings and their distributions.

Your analogies about the PC and web browser are not correctly formulated, because in the case of the PC you talk about 'external components' (you should be talking about cpu arch, structure, digital components, interfaces, etc); in the case of the web browser, you should be talking about modules, code, etc.

We do know how LLMs are laid out: layers, att heads, etc. So what we need to look at are the fundamental possibilities of the structure of LLMs, not how the weights are distributed.

> > And we also know that human beings do not hold 'internal representations' like any AI system needs to.

> Bold fucking claim. Got a source on that?

Part of the sources are in the books I mentioned. Nonetheless, you can still fact-check and refute in an adult and serious manner, not in an disrespectful and arrogant way. If my claim sounded arrogant I apologize, but then as I already mentioned, my references back that claim.

Regarding internal representations in the brain: I guess you are referring to areas of the brain being activated when a subject receives a stimuli, and this is tested through MRI. I would be cautious to causally relate stimuli to neuron activations, since you first need to know if the exact configuration of cell involved and their connections allow for such representation (which I think it is still not known -- again, AFAIK, the contrary seems to be the case).

Re: They’re made out of weights

#687

Earlier quoted context omitted.

Indeed, the leap takes place in the second half. > Cruel. But you said it yourself, who wants to apologize to weights? How did we leap to this? There's nothing to apologize to. They're weights.

"There's nothing to apologize to. They're weights." - The Meat

This type of metaphysics is alien to me as well as the vast majority of humans to have ever existed.

Re: They’re made out of weights

#688
post #681
post #677

Earlier quoted context omitted.

> I see also a common pattern of jumping through different topics at different levels (from theoretical to concrete and back), and that is confusing. I can probably to clarify it. 1. I don't want to end in a situation when AI deserves human rights but I deny it. There are moral reasons for that, and they are important. 2. I employ a systemic view. Not just arguments for consciousness or against it. I look at the peop…

> So why we call it subjective experience then? Probably it is irrelevant, and the reasons are purely historic... or maybe not. How about the idea of computers experiencing things, just not "subjectively" but rather "digitally"? Or choose any other adjective you like. You are arguing against assumptions, but why you just accept the idea of experience with the assumption that human way to experience things is the only…

Let me start with: > if you read the sources on AI research through its own history, you will see how the AI research space is full of such biases and assumptions.

You are repeating it in different forms all the time, so it seems really important to you. I should state, that I don't believe that AI research has some special insights into human nature or into what agency is. I'm sure that they have some biases and assumptions, and maybe it is full of them. You don't need to prove it to me, but if you want to, you could list some of them. It would be interesting to read.

OTOH I really like what AI research does right now. I believe it is beneficial for science and philosophy. "Throw away all that garbage from XX century and build something new." It will be something new, because they are creating things, not just daydreaming philosophically. It may be that the result of their research would be laughable, but it doesn't matter in the long term. It would be another way to look at things, which is the very important for pushing human knowledge forward. I expect AI research would create a heap of knowledge that would be mined for decades then to be incorporated into existing knowledge frameworks. The existing frameworks would break and people would create new ones. And it is an unhappy path, assuming that AI research will not stumble onto a good enough framework to consume all (or some of) others.

Ah... There is one more assumption in there. The assumption that AI wouldn't change the world enough for my intuition of how things go becomes wrong, because it was trained on data from the old world where there was no AI.

> We call it subjective because is 'we, ourselves' and not the objects we perceive there in the world, where the 'experience' is manifested, as perception. We do not always codify dichotomies in language.

So you assume that this naming normalized due to stupid historic reasons, and reject the possibility that it reflects something deeper? Well, I don't. I'm consciously keep myself in an uncertain state, in other words I keep my mind wide open. Psychology teaches us to keep an eye on the exact phrasing, it reflects the real thinking process. It may be hard or even impossible to decipher these cues, but we should try at least.

> In the real world, anyone doing a serious PhD thesis will read whatever is necessary to build a proper, sound theory or body of work. This dismissal just makes me think that you don't know how a PhD thesis is done.

You are talking about about an established science, AI research regarding LLMs is not an established science. Things happens too fast for science to be established. If they slow down, then would be the great time to learn linguistics and to build bridges.

> From what I see, you never took philosophy seriously. I don´t know how you can then seriously engage in a conversation about philosophy.

I took it seriously enough to go from standard philosophic meta level to meta-meta level. I'm seeing not just what philosophy says it is doing, I see what it is really doing de facto. It is like POSIWID principle: the system purpose is what it does.

> The AI space already declared that they 'know' how such processes work. Or at least, they pretend they do.

I'd ask "did they really declared that", but you phrasing suggests that they didn't. I do not exactly on the topic, I do not read all they say, so I may be wrong, but I suppose what they really say, is more like "our machines can think". It is not the same "we know how thinking processes work" in general. They claim that they created some thinking processes. Am I right?

> You would agree that 'problem solving' is just a small portion of what 'thinking' constitutes.

I'm not sure really. You see, I don't know how thinking works. Especially I don't know how human thinking works. It may be that it involves more than just problem solving, or may be not. I can agree that when I reflect my own thinking it looks like something more than just problem solving. Or maybe something less: I have a strong suspicion that human thinking is tuned evolutionary to solve a specific kind of problems. Cultural training repurpose it for some other kinds, but it is like to drive in screws with a hammer, or to hammer nails with a screwdriver. And yes, it is the reasons I do not trust philosophers and scientists to judge if AI has agency already. The space of possible answers includes very scary ones, and in such cases human thinking strongly prefers safe options and creates very strong arguments in favor of them.

Wow. I really glad I spent time arguing with you. I don't know if it was beneficial for you in some way, but it was for me, so thank you for your time. I've just found that I doubt that human thinking is up to the problem. Yeah... Probably the problem will be solved in a chaotic way, when society would just picks some "random" opinion after the dust settles. Yes! It is very likely outcome, we'll end with "they're made out of weights", so the text is prophetic.

> You don't have to trust me. Just pick them up and think for yourself

I'd like to, but you failed to spark a real interest in those books in me. I still do not see what fundamental shifts in my beliefs they may bring. They look like reiteration of thoughts and ideas I know already. Or maybe I know these ideas from reiterations of these books, which are the original sources for the ideas.

I took notice of them, and I'll try to read them. But not right now.

Re: They’re made out of weights

#689
post #231

Earlier quoted context omitted.

You could absolutely write a compelling story about a sentient toaster; it's been done before [1]. That is entirely separate to whether or not it would be a meaningful way to understand the world; a convincing story is not the same thing as one that is true. [1] https://en.wikipedia.org/wiki/The_Brave_Little_Toaster

> a compelling story about a sentient toaster "Howdy-doodly-doo! Anybody like any toast?" https://youtu.be/LRq_SAuQDec?si=YbQfnZbrCe01Bicy

your link has a si= tracking parameter

Re: They’re made out of weights

#690

Earlier quoted context omitted.

You can't do the same with a toaster. Physically you could write that story. But it would fall flat because the toaster is not a compelling subject in a discussion of consciousness. You don't have to believe that LLMs or AI agents are conscious to acknowledge that the argument for their consciousness is far more compelling than any other technological artifact.

Only because you haven't met toasty yet. https://youtu.be/LRq_SAuQDec?si=LFJMXZ2yGu4fxXzL

your link has a si= tracking parameter
Post reply on HN