Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Myth of AI Omniscience: AI's Epistemological Limits

cpwalker.substack.com

Post reply on HN
Print view View original

101–107 of 107 posts

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#101

Post by seanhunter »

K0balt wrote: post #100
seanhunter wrote: post #99

Earlier quoted context omitted.

AI research went off the idea of modelling human reasoning a long time ago. To the extent that current models reproduce reasoning in any sense comparable to a human it is purely for the purpose of user interface. For example if you ask a transformer model like chatgpt to solve some problem and explain its reasoning step by step it will give you a reasonable facsimile of a human thought process, but if you’ve read the…

I don’t think it’s a foregone conclusion that human reasoning doesn’t operate on the basis of statistical prediction of the next most probable “token” at its most granular level. Humans are certainly capable of hallucination in the LLM sense, and without training we often struggle to produce (or even outright fabricate) the rationale behind our “conclusions”. We stopped intentionally modelling human reasoning because…

    > Biological neural networks are fundamentally linear algebra processing device that integrate data into functions through training, so the fact that we understand the process as linear algebra is actually an argument that the process is similar.
Do you have a reference for this? My understanding of biological neural networks is very different from this. Specifically I find it really hard to believe that a biological brain is a linear algebra processing device.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#102

Post by visarga »

K0balt wrote: post #98
visarga wrote: post #61

Earlier quoted context omitted.

Not necessarily. For example Anthropic's ConstitutionalAI (CAI) leverages the model to substitute human judgments in RLHF, effectuating essentially RLAIF. CAI information is used to fine-tune the Claude model. Broadly speaking, you require statistics at echelon N+1 when you are at rung N. We can amplify models by providing them additional time, self-reflexion, demand step by step planning, allow external tools, tune…

Yeah, it makes some sense that you could use a more intense introspection to train weaker ones… I wonder what the human analogue for that looks like. Maybe working up a proof and then quizzing yourself on it? As long as we get >N supervision and the difference is more than the model retrograde, it seems that could work. But it seems like there is a definite limit to that. The N-n1 difference will only stay above the…

The model would learn from feedback, not just regurgitate the training set, as long as the model is part of a system that can generate this feedback. AlphaGo Zero had self play for feedback. Robots can check task execution success. Even chatting with us generates feedback to the model.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#103

Post by K0balt »

seanhunter wrote: post #101
K0balt wrote: post #100

Earlier quoted context omitted.

I don’t think it’s a foregone conclusion that human reasoning doesn’t operate on the basis of statistical prediction of the next most probable “token” at its most granular level. Humans are certainly capable of hallucination in the LLM sense, and without training we often struggle to produce (or even outright fabricate) the rationale behind our “conclusions”. We stopped intentionally modelling human reasoning because…

> Biological neural networks are fundamentally linear algebra processing device that integrate data into functions through training, so the fact that we understand the process as linear algebra is actually an argument that the process is similar. Do you have a reference for this? My understanding of biological neural networks is very different from this. Specifically I find it really hard to believe that a biological…

Matrix multiplication with a different function and weights for each input/output. That’s the primary operation.

There are many other operational modes overlaid that we typically do not model with mainstream models, but they primarily act as broad modifiers (suppression, gain, etc) we understand very well how it works, for example, when we get a few brain cells together to play pong or doom. We can transcribe the functionality of these simple biological networks, once trained, onto models and they function as expected.

The simplified computational model we use for neural networks closely models the primary mechanism, but lacks self reinforcement systems, biochemical moderators, self directed connectivity modifications, etc. Basically we do the training with a separate process for most of the systems in common use, in contrast with biological systems which self organise and train in-situ.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#104

Post by seanhunter »

K0balt wrote: post #103
seanhunter wrote: post #101

Earlier quoted context omitted.

> Biological neural networks are fundamentally linear algebra processing device that integrate data into functions through training, so the fact that we understand the process as linear algebra is actually an argument that the process is similar. Do you have a reference for this? My understanding of biological neural networks is very different from this. Specifically I find it really hard to believe that a biological…

Matrix multiplication with a different function and weights for each input/output. That’s the primary operation. There are many other operational modes overlaid that we typically do not model with mainstream models, but they primarily act as broad modifiers (suppression, gain, etc) we understand very well how it works, for example, when we get a few brain cells together to play pong or doom. We can transcribe the fun…

I notice that you’re consistently just claiming that the biological system does linear algebra without any source for this.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#105

Post by K0balt »

seanhunter wrote: post #104
K0balt wrote: post #103

Earlier quoted context omitted.

Matrix multiplication with a different function and weights for each input/output. That’s the primary operation. There are many other operational modes overlaid that we typically do not model with mainstream models, but they primarily act as broad modifiers (suppression, gain, etc) we understand very well how it works, for example, when we get a few brain cells together to play pong or doom. We can transcribe the fun…

I notice that you’re consistently just claiming that the biological system does linear algebra without any source for this.

I’m not claiming that biological systems “do” linear algebra, I’m claiming that their behaviour can be partially modelled using linear algebra. If I gave the impression that I thought they were actually computing linear algebra I apologise for my unclear writing.

But AFAIK modeling neural network nodes with linear algebra is a foundational principle and has been for quite some time, since I first started messing with neural networks 30 years ago.

But hey, maybe you’re right and computational neural networks aren’t roughly modelled after their biological counterparts and I’ve just misunderstood the field all this time. I guess the good thing about neural networks is you don’t really have to understand them completely to use them lol.

That said, if you’ve got information that refutes the notion that they are roughly modelled after their biological counterparts, or that somehow linear algebra is not the right tool for modelling them, I’d sure be interested to see it.

Just because I’ve ben doing something for decades doesn’t mean I haven’t been doing it -wrong- for decades lol. Wouldn’t be the first thing.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#106

Post by lsy »

vidarh wrote: post #86
lsy wrote: post #83

Earlier quoted context omitted.

What does it take to "hook up" an LLM to a camera and get it to output something cogent? We have a set of human-selected discrete photographs and a set of human-written text labels for those photographs, then a model that is trained to assign those text labels to other photographs. Then a new photo is put into the model, a set of labels is emitted, and those labels are processed by the LLM as text (or more accurately…

> What does it take to "hook up" an LLM to a camera and get it to output something cogent? The same as it takes to teach a human to recognise objects and output something cogent, except for machine learning models the effort does not need to be replicated from scratch each time. But you're entirely missing the point of the comment you replied to, which is that giving them feedback loops and ability to carry out actio…

An LLM can't generate new training data for itself. Mathematically speaking, the output of the LLM is governed by the same statistics embedded in its parameters, and a large enough training set of LLM output should have zero impact on those parameters. The difference is that training on human output contains the implicit assumption that the human output is "correct" in some way. So if the LLM gets more human-generated training data, it can come closer to "correctness". An LLM training on its own output is merely reinforcing its weights.

Maybe to put it more simply, there is no LLM in the world that can determine for itself whether a proposition or piece of training data is sound without a person saying so at some point, either implicitly or explicitly. Even efforts to give LLMs some non-human-mediated reward function are merely people attempting to encode our own principles. While people are no doubt physical, we have a biological imperative to survival and well-being that governs our sense-making faculties, and to which nobody has yet proposed an analogue for LLMs. Our "transformations" are not only governed by other people's "training data" but by our urge to be able to skillfully cope with a world in which we are biologically invested.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#107

Post by Capricorn2481 »

bloaf wrote: post #92
melicerte wrote: post #91

Earlier quoted context omitted.

Confetti cannon is a thing. So there is an obvious link between confetti and cannon. Furthermore, These ("cannon" and "conffetis) are two very common concepts used in a single sentence. Excuse me for being genuine but I totally fail to understand how original the phrase is.

Why don't you come up with an example of a new phrase with the same meaning that is actually original so we can compare your "real" originality to this "fake" originality?

The dazzling ribbons, stripped from the corpses of trees, shot out of the pressurized tube
Post reply on HN