Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI fantasy is a blocker to actual engineering

tomwphillips.co.uk

Post reply on HN
Print view View original

631–640 of 683 posts

Re: AGI fantasy is a blocker to actual engineering

#631

Post by RaftPeople »

naasking wrote: post #382
RaftPeople wrote: post #291

Earlier quoted context omitted.

> This is simply a scaling problem, eg. thousands of single I/O functions can reproduce the behaviour of a function that takes thousands of inputs and produces thousands of outputs. I think it's more than just scaling, you need to understand the functional details to reproduce those functions (assuming those functions are valuable for the end result as opposed to just the way it had to be done given the medium). An i…

Yes, there may be interesting math, but I didn't mean "scaling LLMs", necessarily. I was making a more general point that a single-I/O function can pretty trivially replicate a multi-I/O function, so the OP's point that "LLM neurons" are single-I/O and bio neurons are multi-I/O doesn't mean much. Estimates of brain complexity have already factored this in, which is why we know we're still a few orders of magnitude aw…

> Yes, there may be interesting math, but I didn't mean "scaling LLMs", necessarily.

Ya, I realized later that the LLM scaling part of my post sounded like it misinterpreted what you said when it was really a separate point unrelated to the topic of neurons that just happened to include the word "scaling" also.

I do agree with you somewhat that just because biological neurons are vastly more complex and functional than typical artificial neurons, it just means we need more artificial neurons to achieve similar functionality.

> Estimates of brain complexity have already factored this in

I don't agree with this, the estimates I've seen don't seem to factor it in, and many of those estimates were prior to things discovered within just the last 5 years that expose significantly more complexity and capability that needs to be understood first.

> And there are alternative models such as spiking neural networks which more closely mimic biology, but it's not clear whether these are really that critical.

I kept reading that people wanted to use spiking networks and I thought the same thing as you, it didn't seem to provide a benefit. A while ago I read some paper about why they want to use spiking networks and I can't remember the details but they described some functional capabilities that really were much easier with spiking. I vaguely remember that it had to do with processing real-time sensory information, it was easier to synchronize signals based on frequency instead of trying to rely on precise single signal timing (something like that). And I think there were benefits in other areas also.

Re: AGI fantasy is a blocker to actual engineering

#632

Post by hearsathought »

Apocryphon wrote: post #507
killerstorm wrote: post #58

Earlier quoted context omitted.

AlphaEvolve is a general system which works in many domains. How is that not a step towards general intelligence? And it is effectively a loop around LLM. But my point is that we have evidence that Demis Hassabis knows his shit. Just doubting him on a general vibe is not smart

Has he, his team, or DeepMind used any AGI rhetoric, even just as advertising?

Hassabi has talked about AGI in a lot of interviews. So has members of his Deepmind team. And of course current and former alphabet employees - the most prominent being schmidt. He definitely thinks it is coming and said we should prepare for it. Just search for his interviews on AI and you'll get a bunch of them.

Re: AGI fantasy is a blocker to actual engineering

#633

Post by axkdev »

nitwit005 wrote: post #563
IgorPartola wrote: post #21

It is ultimately a hardware problem. To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs, to the point that some inputs start being processed before they even get inside the cell by structures on the outside of it. An LLM neuron is an approximation of this. We cannot manufacture a human level neuron to be s…

A bee is an autonomous walking, climbing, and flying drone that investigates its environment, collects resources, builds structures, and coordinates with other drones. We're totally incapable of building an AI that can do anything resembling that. We're still at the phase where robots walking on rough terrain without falling over remains a bit impressive. I doubt the limitation is that we can't produce enough raw com…

But can a bee create a video of Will Smith eating pasta?

Re: AGI fantasy is a blocker to actual engineering

#634

Post by jcheng »

jiggawatts wrote: post #574
jcheng wrote: post #562

Earlier quoted context omitted.

Do you have a citation for that factoid?

Apologies, looks like I mis-remembered the numbers from the original debate, the actual number is about 110x 40-foot containers. It can be calculated easily enough from the # shipped, the volume of a charging case, and the volume of a shipping container. It doesn’t alter the conclusion that much: in the grand scheme of human industrial activity, this is nothing . I.e.: if AirPods were made by an independent company,…

Sure, I’m with you on your larger point.

Re: AGI fantasy is a blocker to actual engineering

#635

Post by titzer »

fallingfrog wrote: post #499
titzer wrote: post #474

Earlier quoted context omitted.

Let me guess, you have a truly marvelous demonstration of this proposition which this margin is too narrow to contain.

Well it kind of sucks honestly because im never going to get any sort of recognition, but that's partly by choice and partly because I don't have the right personal connections. I 100% understand why all of you think I'm a significant fool. Thats ok. Like i said, im burying the idea. But its obvious enough that someone else will discover it soon enough.

You literally see no irony here? You are doubling, tripling down on some idea you won't even vaguely describe to people, but you are so utterly convinced that it's right you don't appear to even admit the possibility that your idle thoughts could be wrong, or just silly. Frankly it sounds delusional and you should question yourself more often. If you're so afraid of being right that you won't even write the idea down to have another person look at it for fear it will somehow set the world on fire because of its rightness, then you should seriously consider talking to a therapist. If that comes across as an ad hominem, it isn't, and it isn't meant as an insult or a dismissal of your idea--you've actually offered no idea. Frankly, you should be a little embarrassed as you've done nothing but grouse about how smart you are, and people react negatively to that. Puffing gets us nowhere except drawing attention to yourself and gave us no new ideas or thoughts to discuss, except your ego. I'd recommend you question yourself more and talk to some people you trust about whatever it is you think you discovered.

Re: AGI fantasy is a blocker to actual engineering

#636

Post by pavas »

HarHarVeryFunny wrote: post #628
pavas wrote: post #619

Earlier quoted context omitted.

> As noted, consciousness seems to just be the ability to self-observe, which is useful as another predictive input. As far as I know, consciousness is referring to something other than self-referential systems, especially with regards to the hard problem of consciousness. The [philosophical zombie]( https://en.wikipedia.org/wiki/Philosophical_zombie ) thought experiment is well-known for imagining something with all…

It seems entirely possible that the "philosophical zombie" is an impossible/illogical construct, and that in fact anything with all the structure necessary for consciousness will of necessity be conscious. When considering the structural underpinnings of consciousness, it's interesting to note the phenomena of "blindsight", which is essentially a loss of visual consciousness without an actual loss of vision! Note tha…

> It seems entirely possible that the "philosophical zombie" is an impossible/illogical construct, and that in fact anything with all the structure necessary for consciousness will of necessity be conscious.

Yes, and that's pretty much exactly the point: we don't know of any way of determining whether someone is a p-zombie or a being with conscious phenomenal experience. We can certainly have an opinion or belief or assume that sufficient structure means consciousness, which is a perfectly reasonable stance to take and one that many would take, but we have to be careful to understand that's not a scientific stance since it isn't testable or falsifiable, which is why it's been called the "hard problem" of consciousness. It's an unfounded belief we choose out of reasons like psychological comfort.

With regards to your latter point, I think you are making some sophisticated distinctions regarding the "map and territory" relation, and it seems you've hit upon the crux of the matter: how can we report "what its like" for us to experience something the other person hasn't experienced, if its not deconstructible to phenomenal states they've already experienced (and therefore constructible for them based off of our report)? The landmark paper here is "What Is It Like to Be a Bat?" by Josh Nagel, and if you're ever curious it's a pretty short read.

With regards to "blindsight" since I'm not familiar with it and curious, how do we distinguish between loss of visual consciousness and loss of information transfer between conscious regions, or loss of memory about conscious experience?

Re: AGI fantasy is a blocker to actual engineering

#637

Post by rekrsiv »

DrierCycle wrote: post #627
rekrsiv wrote: post #604

Earlier quoted context omitted.

We use logic to design our technology, but evolution does it by literally shaking all the atoms into place, no design involved. Our brains were created randomly.

It appears logical, but it falls under the same threshold. We say it has function, which is logical, but its used for something we could or may not have planned for. That's evolution. The logic is a pretext.

If it quacks like a Markov chain...

Re: AGI fantasy is a blocker to actual engineering

#638

Post by nawgz »

mcny wrote: post #603
nawgz wrote: post #595

Earlier quoted context omitted.

I’m in agreement with you in that raw compute isn’t really the only missing piece, but I’m not in agreement we have enough compute to fully simulate even simple insect brains

I feel like we are talking past one another here. I disagree that all the computer processors in the world combined don't have enough raw processing power to simulate a single bee brain. That to me is an absurd idea. Now if you meant we don't know enough about bees to actually model what its brain does and create a processor to actually process an artificial bee brain, then yes, we don't have that ability yet. ---- A…

> I disagree that all the computer processors in the world combined don't have enough raw processing power to simulate a single bee brain. That to me is an absurd idea.

A bee weighs 100mg. If they are 5% brain, their brain weighs 5mg. 5mg of carbon is 2.5 * 10^20 atoms.

The largest supercomputers, at hundreds of trillions of transistors, come in at 2-5 * 10^14 transistors.

I don't think that combining all the computers in the world would give us 1 transistor per atom in a bee brain, and I additionally have to imagine transistors are incapable of simulating an atom by multiple orders of magnitude (i.e. 1 atom would require > 1 * 1 ^ 4 transistors) in realtime.

So, I would argue that our level of compute is still insufficient to simulate even a simple insect brain in realtime. Perhaps you could compute 1 second of bee thinking in an hour using all our compute.

And then, of course, comes into play that we have no idea how to simulate an atom in full fidelity.

Re: AGI fantasy is a blocker to actual engineering

#639

Post by HarHarVeryFunny »

pavas wrote: post #636
HarHarVeryFunny wrote: post #628

Earlier quoted context omitted.

It seems entirely possible that the "philosophical zombie" is an impossible/illogical construct, and that in fact anything with all the structure necessary for consciousness will of necessity be conscious. When considering the structural underpinnings of consciousness, it's interesting to note the phenomena of "blindsight", which is essentially a loss of visual consciousness without an actual loss of vision! Note tha…

> It seems entirely possible that the "philosophical zombie" is an impossible/illogical construct, and that in fact anything with all the structure necessary for consciousness will of necessity be conscious. Yes, and that's pretty much exactly the point: we don't know of any way of determining whether someone is a p-zombie or a being with conscious phenomenal experience. We can certainly have an opinion or belief or…

I'm not sure how much, if any, work has been done to study the brains of people with blindsight. I'm also not sure I would differentiate between loss of visual consciousness and loss of information transfer ... my understanding is that it's the loss if information transfer that is causing the loss of consciousness (e.g maybe your visual cortex works fine, so you can see, and you can perform some visual tasks that have been well practiced and/or no longer need general association cortex, but if the connection between visual cortex and association cortex was lost, then perhaps this is where you become unaware of your ability to see, i.e. lose visual consciousness).

I don't think it's a memory issue - one classic test of blindsight is asking the patient to navigate a cluttered corridor full of obstacles, which the patient succeeds in doing despite reporting themselves as blind - so it's a real-time phenomena, not one of memory.

Re: AGI fantasy is a blocker to actual engineering

#640

Post by Libidinalecon »

ACCount37 wrote: post #616
lossolo wrote: post #338

Earlier quoted context omitted.

We have already found limitations of the current LLM paradigm, even if we don't have a theorem saying transformers can never be AGI. Scaling laws show that performance keeps improving with more params, data + compute but only following a smooth power law with sharply diminishing returns. Each extra order of magnitude of compute buys a smaller gain than the last, and recent work suggests we're running into economic an…

A lot of those failings (i.e. COT faithfulness) are straight up human failure modes. LLMs failing the same way as humans do on the same tasks as humans is a weak sign of "this tech is AGI capable", in my eyes. Because it hints that LLMs are angling to do the same things human mind does, and in similar enough ways to share the failure modes. And human mind is the one architecture we know to support general intelligenc…

This is all nonsense and you are just falling for marketing that you want to be true.

The whole space is largely marketing at this point, intentionally conflating all these philosophical terms because we don't want to face the ugly reality that LLMs are a dead end to "AGI".

Not to mention, it is not on those who don't believe in Santa Clause to prove that Santa Clause doesn't exist. It is on those who believe in Santa Clause to show how AGI can possibly emerge from next token prediction.

I would question if you even use the models much really because I thought this in 2023 but I just can't imagine how anyone who uses the models all the time can possibly think we are on the path to AGI with LLMs in 2025.

It is almost like the idea of a thinking being emerging from text was a dumb idea to start with.

Post reply on HN