Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI is Mathematically Impossible 2: When Entropy Returns

philarchive.org

Post reply on HN
Print view View original

61–70 of 437 posts

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#61

Post by fellowniusmonk »

ffwd wrote: post #10
like_any_other wrote: post #3

So does the human brain transcend math, or are humans not generally intelligent?

I think humans have some kind of algorithm for deciding what's true and consolidating information. What that is I don't know.

This paper is about the limits in current systems.

Ai currently has issues with seeing what's missing. Seeing the negative space.

When dealing with complex codebases you are newly exposed to you tackle an issue from multiple angles. You look at things from data structures, code execution paths, basically humans clearly have some pressure to go, fuck, I think I lost the plot, and then approach it from another paradigm or try to narrow scope, or based on the increased information the ability to isolate the core place edits need to be made to achieve something.

Basically the ability to say, "this has stopped making sense" and stop or change approach.

Also, we clearly do path exploration and semantic compression in our sleep.

We also have the ability to transliterate data between semantic to visual structures, time series, light algorithms (but not exponential algorithms, we have a known blindspot there).

Humans are better at seeing what's missing, better at not closuring, better at reducing scope using many different approaches and because we operate in linear time and there are a lot of very different agents we collectively nibble away at complex problems over time.

I mean on a 1:1 teleomere basis, due to structural differences people can be as low as 93% similar genetically.

We also have different brain structures, I assume they don't all function on a single algorithmic substrate, visual reasoning about words, semantic reasoning about colors, synesthesia, the weird handoff between hemispheres, parts of our brain that handle logic better, parts of our brain that handle illogic better. We can introspect on our own semantic saturation, we can introspect that we've lost the plot. We get weird feelings when something seems missing logically, we can dive on that part and then zoom back out.

There's a whole bunch of shit the brain does because it has a plurality of structures to handle different types of data processing and even then the message type used seems flexible enough that you can shove word data into a visual processor part and see what falls out, and this happens without us thinking about it explicitly.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#62

Post by ffwd »

ICBTheory wrote: post #52
ffwd wrote: post #10

Earlier quoted context omitted.

I think humans have some kind of algorithm for deciding what's true and consolidating information. What that is I don't know.

I guess so too... but whatever it is: it cannot possibly be something algorithmic. Therefore it doesn't matter in terms of demonstrating that AI has a boundary there, that cannot be transcended by tech, compute, training, data etc.

Why can't it be algorithmic? If the brain uses the same process on all information, then that is an algorithmic process. There is some evidence that it does do the same process to do things like consolidating information, processing the "world model" and so on.

Some processes are undoubtedly learned from experience but considering people seem to think many of the same things and are similar in many ways it remains to be seen whether the most important parts are learned rather than innate from birth.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#63

Post by fellowniusmonk »

geor9e wrote: post #39
ninetyninenine wrote: post #5

Without reading the paper how the heck is agi mathematically impossible if humans are possible? Unless the paper is claiming humans are mathematically impossible? I’ll read the paper but the title comes off as out of touch with reality.

The title is clickbait. He more ends up saying that AGI is practically impossible today, given all our current paradigms of how we build computers, algorithms, and neural networks. There's an exponential explosion in how much computation time it requires to match the out-of-frame leaps and bounds that a human brain can make with just a few watts of power, and researchers have no clever ideas yet for emulating that tr…

In the abstract it explicitly says current systems, the title is 100% click bait.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#64

Post by pvg »

JdeBP wrote: post #55
pvg wrote: post #49

Earlier quoted context omitted.

There's nothing wrong with any of that, for an HN submission. The paper itself could be bad but that's what the discussion thread is for - discussing the thing presented rather than its meta attributes.

And no-one said that there was anything wrong, the inference being yours. But it's important to bear provenance in mind, and not get carried away by something like this more than one would be carried away by, say, an article on Medium propounding the same thing, as the bars to be cleared are about the same height.

The provenance is there for everyone to see so the purpose of the comment, beside some sort of implied aspersion is unclear.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#65

Post by ffwd »

fellowniusmonk wrote: post #61
ffwd wrote: post #10

Earlier quoted context omitted.

I think humans have some kind of algorithm for deciding what's true and consolidating information. What that is I don't know.

This paper is about the limits in current systems. Ai currently has issues with seeing what's missing. Seeing the negative space. When dealing with complex codebases you are newly exposed to you tackle an issue from multiple angles. You look at things from data structures, code execution paths, basically humans clearly have some pressure to go, fuck, I think I lost the plot, and then approach it from another paradigm…

Yep definitely agree with this.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#66

Post by alganet »

mort96 wrote: post #58
alganet wrote: post #56

Earlier quoted context omitted.

I'm not talking about soul. I'm just saying you're mistaking the thing for the the tool we use to describe the thing. I'm also not talking about simulations. Epistemologically, I'm talking about unknown unknowns. There are things we don't know, and we still don't know we don't know yet. Math and physics deal with known unknowns (we know we don't know) and known knowns (we know we know) only. Math and physics do not a…

If a human body is intelligent, and we could in principle set up a computer-simulated universe which has a human body in it and simulate it forward with sufficient accuracy to make the body operate as a real-world human body has, we would have an artificial general intelligence simulated by a computer (i.e using mathematics). If you think there are potential flaws in this line of reasoning other than the ones I alrea…

We currently can't simulate the universe. Not only in capability, but also knowledge. For example, we don't know where or when life started. Can't "simulate forward" from an event we don't understand.

Also, a simulation is not the thing. It's a simulation of the thing. See? The same issue. You're mistaking the thing for the tool we use to simulate the thing.

You could argue that the universe _is_ a simulation, or computational in nature. But that's speculation, not very different epistemologically from saying that a magic wizard made everything.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#67

Post by EPWN3D »

Workaccount2 wrote: post #54
IAmGraydon wrote: post #24

Earlier quoted context omitted.

>We don’t even know how LLMs work. Care to elaborate? Because that is utter nonsense.

We understand and build the trellis that the LLMs "grow" on. We don't have good insight into how a fully grown LLM actually turns any specific input into any specific output. We can follow it through the network, but it's a totally senseless noisy mess. "Cat" lights up a certain set of neurons, but then "cat" looks completely different. That is what we don't really understand. (This is an illustrative example made fo…

We don't know the path for how a given input produces a given output, but that doesn't mean we don't know how LLMs work.

We don't and can't know with certainty which specific atoms will fission in a nuclear reactor either. But we know how nuclear fission works.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#68

Post by ICBTheory »

vessenes wrote: post #32
ICBTheory wrote: post #2

This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions — not due to lack of compute, but because of how entropy behaves in heavy-tailed decision spaces. The idea is called IOpenER: Information Opens, Entropy Rises. It builds on Shannon’s information theory to show that in specific problem classes (those with α ≤ 1), adding information doesn’t reduce uncer…

Thanks for this - Looking forward to reading the full paper. That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible. Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?

Sure I can (and thanks for writing)

Well, given the specific way you asked that question I confirm your self assertion - and am quite certain that your level of Artificiality converges to zero, which would make you a GI without A...

- You stated to "feel" generally intelligent (A's don't feel and don't have an "I" that can feel) - Your nuanced, subtly ironic and self referential way of formulating clearly suggests that you are not a purely algorithmic entity

A "précis" as you wished: Artificial — in the sense used here (apart from the usual "planfully built/programmed system" etc.) — algorithmic, formal, symbol-bound.

Humans as "cognitive system" have some similar traits of course - but obviously, there seems to be more than that.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#69

Post by viralsink »

If I understood correctly, this is about finding solutions to problems that have an infinite solution space, where new information does not constrain it.

Humans don't have the processing power to traverse such vast spaces. We use heuristics, in the same way a chess player does not iterate over all possible moves.

It's a valid point to make, however I'd say this just points to any AGI-like system having the same epistemological issues as humans, and there's no way around it because of the nature of information.

Stephen Wolfram's computational irreducibility is another one of the issues any self-guided, phyiscally grounded computing engine must have. There are problems that need to be calculated whole. Thinking long and hard about possible end-states won't help. So one would rather have 10000 AGIs doing somewhat similar random search in the hopes that one finds something useful.

I guess this is what we do in global-scale scientific research.

Re: AGI is Mathematically Impossible 2: When Entropy Returns

#70

Post by mort96 »

alganet wrote: post #66
mort96 wrote: post #58

Earlier quoted context omitted.

If a human body is intelligent, and we could in principle set up a computer-simulated universe which has a human body in it and simulate it forward with sufficient accuracy to make the body operate as a real-world human body has, we would have an artificial general intelligence simulated by a computer (i.e using mathematics). If you think there are potential flaws in this line of reasoning other than the ones I alrea…

We currently can't simulate the universe. Not only in capability, but also knowledge. For example, we don't know where or when life started. Can't "simulate forward" from an event we don't understand. Also, a simulation is not the thing. It's a simulation of the thing. See? The same issue. You're mistaking the thing for the tool we use to simulate the thing. You could argue that the universe _is_ a simulation, or com…

Of course we can't simulate the universe (or, well, a slice of a universe which obeys the same laws as ours) right now, but we're discussing whether it's possible in principle or not.

I don't understand what fundamental difference you see between a thing governed by a set of mathematical laws and an implementation of a simulation which follows the same mathematical laws. Why would intelligence be possible in the former but fundamentally impossible in the latter, aside from precision limitations?

FWIW, nothing I've said assumes that the universe is a simulation, and I don't personally believe it is.

Post reply on HN