Live data from Hacker News

The Myth of AI Omniscience: AI's Epistemological Limits

cpwalker.substack.com

91–100 of 107 posts

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#91
post #49

Earlier quoted context omitted.

That's not an LLM, though. We haven't seen evidence yet that an LLM can combine existing language in novel ways. In fact, we've seen over and over that LLMs are quite generic. Compared to text-to-image, which is seemingly impossible to use without getting something weird

There was a post here on hacker news not that long ago where GPT4 came up with "the confetti has left the cannon" when asked for an original phrase similar to "the cat is out of the bag". Other users confirmed Google could not identify any other use of that phrase. People saying GPT4 is unoriginal have an uphill battle, it is not the default assumption of anyone who has worked with it.

Confetti cannon is a thing. So there is an obvious link between confetti and cannon. Furthermore, These ("cannon" and "conffetis) are two very common concepts used in a single sentence.

Excuse me for being genuine but I totally fail to understand how original the phrase is.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#92
post #49

Earlier quoted context omitted.

There was a post here on hacker news not that long ago where GPT4 came up with "the confetti has left the cannon" when asked for an original phrase similar to "the cat is out of the bag". Other users confirmed Google could not identify any other use of that phrase. People saying GPT4 is unoriginal have an uphill battle, it is not the default assumption of anyone who has worked with it.

Confetti cannon is a thing. So there is an obvious link between confetti and cannon. Furthermore, These ("cannon" and "conffetis) are two very common concepts used in a single sentence. Excuse me for being genuine but I totally fail to understand how original the phrase is.

Why don't you come up with an example of a new phrase with the same meaning that is actually original so we can compare your "real" originality to this "fake" originality?

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#93
post #75

he lost me at "a language model’s vocabulary is limited to the words that exist within the model’s training texts, which means a LLM can only refer to objects and relations that we humans have already discerned, named, and written about." This is trivially demonstrated to be a false statement, as GPT is capable of synthesizing entirely novel words based on very little input guidance.

I find it exhausting when articles are written about ChatGPT which can be trivially demonstrated false by simply testing their assumptions. Noam Chomsky pulled a similar one a while ago.

Whats up with the number of mind-numbingly stupid takes in this thread? Are you actually this dumb or is there some elaborate in-joke i missed? Just because LLMs can (poorly) come up with new words when you instruct them to doesn't mean they're actually "thinking" in novel concepts, its just another indirectly grounded label for existing concepts. https://dai.fmph.uniba.sk/~retova/CSCTR/materials/CSCTR_07se...

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#94
Why do we need anything close to superintelligence or "omniscience" (wtf) before we get profound changes to how humanity operates. It's really an obsession!

Instead compare to the average human, at most. Compare to refrigeration? Compare to fire? Compare to computers? And that's even if the bar is placed at the singularity. Human progress as we have known it so far is not founded on superhumans but on a lot of work by a mix of merely visionary, hard working, and just plain average people. Just speeding that an order of magnitude changes everything.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#95
post #30
post #16

The article is claiming that advanced AIs cannot become more intelligent than humans, essentially because LLMs cannot become more intelligent than humans. LLMs aren’t the be all and end all of AI though. They’re an impressive but inherently limited stepping stone, with a very constrained scope of applicability and capabilities. There is no reason to suppose that future, much more advanced architectures can’t surpass…

More specifically, the article assumes that the only way "AI"s will ever be produced is the way that LLMs are -- by being trained off pre-existing human generated media, and existing as static objects after that. In particular, this assumes that the system doesn't subsequently modify itself based on further interactions with the real world (e.g., doing experiments, and taking note of the results). That's assuming an…

It's assuming something we know to be false. AlphaZero wasn't even trained against humans at all, but still beats them at Go and Chess handily.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#96
post #72
post #67

Earlier quoted context omitted.

How much of the "novel ways which have never been considered before" is just the novelty effect of having your very own artist? A human being could certainly produce any of the works of Dall-E 2, given the same prompt. The change here is the cost, and not the capability. Of course, this is still significant, but it doesn't suggest to me that Dall-E 2 "thinks" differently or would be able to seriously alter the nature…

What would be an example of a work that a human being could not produce?

A vast, intricate design with microscopic details at hundreds of levels of emergent forms, coloured in semi-transparency generating holographic effects, using spectra of light far beyond the human visual range, executed in three dimensions and occupying hundreds of cubic miles. Oh, and why not animate it? It would have to be virtual I suppose, or vastly too expensive to ever be created physically. In fact, why design it to be physically possible? Go nuts, make it 7 dimensional.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#97
post #7

Rather than viewing generative AI as a form of artificial intelligence, I posit that it should be seen as an automated tool for tapping into human cultural, linguistic, and empirical knowledge. Data and computation are two sides of the same coin. The 'intelligence' in AI is embedded within the data, with the computational model serving as a tool to access and express this inherent intelligence. I would argue for a ch…

How can you be confident that by scaling up the models their reasoning skills will not surpass humans

Because, ultimately, the models are modelling human reasoning.

Not only that we do not have any evidence to support the idea that reasoning has a higher quality than we can apply to it using logical processes. It can be done faster, it can be done many times concurrently but ultimately 1 || 1 !& 1 is going to be have the same answer.

The whole idea of superintelligence except as a measure of speed or quantity seems flawed on its face. And if we are to call speed or concurrency superintelligence, then we have already hit that mark some time ago.

AI wins at games because of time compression, not because of an inherently superior logic. They just have time to consider more potential outcomes.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#98
post #61
post #25

Earlier quoted context omitted.

Recursive insight is possible with a model that self trains, but right now that would result in a detour into unreality. Perhaps with the right systems of vetting prior to incorporating new data into the retraining set. Right now they just get stupider if you train them on their own output, which suggests that the quality of the data available in the training set is higher than the quality of output produced by the m…

Not necessarily. For example Anthropic's ConstitutionalAI (CAI) leverages the model to substitute human judgments in RLHF, effectuating essentially RLAIF. CAI information is used to fine-tune the Claude model. Broadly speaking, you require statistics at echelon N+1 when you are at rung N. We can amplify models by providing them additional time, self-reflexion, demand step by step planning, allow external tools, tune…

Yeah, it makes some sense that you could use a more intense introspection to train weaker ones… I wonder what the human analogue for that looks like.

Maybe working up a proof and then quizzing yourself on it?

As long as we get >N supervision and the difference is more than the model retrograde, it seems that could work. But it seems like there is a definite limit to that. The N-n1 difference will only stay above the improvement delta up to a point.

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#99
post #97

Earlier quoted context omitted.

How can you be confident that by scaling up the models their reasoning skills will not surpass humans

Because, ultimately, the models are modelling human reasoning. Not only that we do not have any evidence to support the idea that reasoning has a higher quality than we can apply to it using logical processes. It can be done faster, it can be done many times concurrently but ultimately 1 || 1 !& 1 is going to be have the same answer. The whole idea of superintelligence except as a measure of speed or quantity seems f…

AI research went off the idea of modelling human reasoning a long time ago. To the extent that current models reproduce reasoning in any sense comparable to a human it is purely for the purpose of user interface. For example if you ask a transformer model like chatgpt to solve some problem and explain its reasoning step by step it will give you a reasonable facsimile of a human thought process, but if you’ve read the transformers paper, you know that its actual process is tokenize -> do a bunch of matrix math -> decode the result into words (entirely different to how a human reasoning process works).

Although things like neural nets were clearly inspired by biology they work completely differently to a biological brain. It would be closer to truth to say that the models use linear algebra and optimization to improve performance at specific tasks. For that reason, the whole debate about whether or not superintelligence is possible/has been achieved etc boils down to an argument about the definition of intelligence (as Turing predicted so long ago).

Re: The Myth of AI Omniscience: AI's Epistemological Limits

#100
post #97

Earlier quoted context omitted.

Because, ultimately, the models are modelling human reasoning. Not only that we do not have any evidence to support the idea that reasoning has a higher quality than we can apply to it using logical processes. It can be done faster, it can be done many times concurrently but ultimately 1 || 1 !& 1 is going to be have the same answer. The whole idea of superintelligence except as a measure of speed or quantity seems f…

AI research went off the idea of modelling human reasoning a long time ago. To the extent that current models reproduce reasoning in any sense comparable to a human it is purely for the purpose of user interface. For example if you ask a transformer model like chatgpt to solve some problem and explain its reasoning step by step it will give you a reasonable facsimile of a human thought process, but if you’ve read the…

I don’t think it’s a foregone conclusion that human reasoning doesn’t operate on the basis of statistical prediction of the next most probable “token” at its most granular level. Humans are certainly capable of hallucination in the LLM sense, and without training we often struggle to produce (or even outright fabricate) the rationale behind our “conclusions”.

We stopped intentionally modelling human reasoning because we have no clue how it works.

We roughly copied the physical devices then figured out how to slap them together with an algorithm that makes them behave similarly to human thought, when trained with a massive quantity of cultural-linguistic-memetic data.

It should not be surprising that if you take a bucketful of engine parts and keep messing with them until they kinda work as an engine that you will probably wind up with something along similar lines as the original intention of the parts. Frankly, it would be quite surprising if you didn’t.

Biological neural networks are fundamentally linear algebra processing device that integrate data into functions through training, so the fact that we understand the process as linear algebra is actually an argument that the process is similar.

As to whether “superintelligence” can exist, I think we are in exactly the same page there. It is a matter of definition. Personally I doubt the existence of a fundamentally superior system of logic type of superintelligence, but certainly a device can made to iterate faster and flawless memory and instant, accurate calculation is going to beat the hell out of my notebook and HP28s lol.

I suspect that superintelligence can be (has been?) achieved in some respects, but in the same sense that a room full of ten educated people with regular computing tools is “superintelligence”.

Post reply on HN