Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI fantasy is a blocker to actual engineering

tomwphillips.co.uk

Post reply on HN
Print view View original

621–630 of 683 posts

Re: AGI fantasy is a blocker to actual engineering

#621

Post by Ey7NFZ3P0nzAe »

IgorPartola wrote: post #21

It is ultimately a hardware problem. To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs, to the point that some inputs start being processed before they even get inside the cell by structures on the outside of it. An LLM neuron is an approximation of this. We cannot manufacture a human level neuron to be s…

Med resident here: AFAIK the 80-90 billion neuron is misleading: more than 80% of them are in the cerebellum and are mostly a low pass filter for motor signals. People born with no cerebellum are of normal intelligence. And we don't know how much of the neocortex is actually useful for consciousness but apparently a minority of it.

Re: AGI fantasy is a blocker to actual engineering

#622

Post by yahoozoo »

fallingfrog wrote: post #9
Etheryte wrote: post #2

Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. Many have also gone on and started companies to look for a new way forward. However, if you're hip deep in stock options, along with your reputation, you'll hardly want to break the mirage. So here we are.

I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it. The people working on AI are very smart and they will solve the associated challenges soon enough. The problem of how to slow down the development of these technologies- a political problem- is much more pressing right now.

cringe

Re: AGI fantasy is a blocker to actual engineering

#623

Post by u456retdfgcjmvb »

ACCount37 wrote: post #616
lossolo wrote: post #338

Earlier quoted context omitted.

We have already found limitations of the current LLM paradigm, even if we don't have a theorem saying transformers can never be AGI. Scaling laws show that performance keeps improving with more params, data + compute but only following a smooth power law with sharply diminishing returns. Each extra order of magnitude of compute buys a smaller gain than the last, and recent work suggests we're running into economic an…

A lot of those failings (i.e. COT faithfulness) are straight up human failure modes. LLMs failing the same way as humans do on the same tasks as humans is a weak sign of "this tech is AGI capable", in my eyes. Because it hints that LLMs are angling to do the same things human mind does, and in similar enough ways to share the failure modes. And human mind is the one architecture we know to support general intelligenc…

Breaking down in -30C temperatures is also human failure mode, but doesen't make cars human. They both exhibit the exact same behavior (not moving), but are fundamentally different

Re: AGI fantasy is a blocker to actual engineering

#624

Post by eucyclos »

dontlaugh wrote: post #397
grokgrok wrote: post #247

Earlier quoted context omitted.

The negatively coded, tribal/political speech can be referred to as 'Polemic' which stems from 'warlike' expression.

And polemic is an entirely legitimate form of political action. Would you rather starve or never lie about those that would starve you?

I would rather not trust the first person who claims wants to starve me. Polemnics may be legitimate - they may not be, I haven't thought about it deeply - but they are undoubtedly worth dropping from my own information diet.

Re: AGI fantasy is a blocker to actual engineering

#626

Post by daveguy »

XorNot wrote: post #580
daveguy wrote: post #241

Earlier quoted context omitted.

Minor correction here. You are correct about hardware being an issue, but the magnitude is much greater. You have a lot more than "thousands" of inputs. In the hand alone you have ~40,000+ tactile corpuscles (sensing regions). And that's just one mode. The eye has ~7 million cones and 80 million rods. There is processing and quantization performed by each of those cells and each of the additional cells those signal,…

I agree with this up till saying we must be very far from AGI. I don't think we're close, but the scale of human inputs doesn't tell us anything about it. A useful AGI need not be capable of human level cognition, and human level cognition need not require the entire human biological or nervous systems - we're a product of millions of years of undirected random evolution, optimized to run a fleshy body and survive Af…

I'm pretty sure human level cognition requires human level processing power. We are still multiple orders of magnitude away from that.

A blind programmer still has human processing power. The "usually-sight" regions of the brain don't just shut down. They're still used.

Re: AGI fantasy is a blocker to actual engineering

#627

Post by DrierCycle »

rekrsiv wrote: post #604
DrierCycle wrote: post #586

Earlier quoted context omitted.

The brain is logical by design, it implements logic?

We use logic to design our technology, but evolution does it by literally shaking all the atoms into place, no design involved. Our brains were created randomly.

It appears logical, but it falls under the same threshold. We say it has function, which is logical, but its used for something we could or may not have planned for. That's evolution. The logic is a pretext.

Re: AGI fantasy is a blocker to actual engineering

#628

Post by HarHarVeryFunny »

pavas wrote: post #619
HarHarVeryFunny wrote: post #559

Earlier quoted context omitted.

As noted, consciousness seems to just be the ability to self-observe, which is useful as another predictive input. I would expect that all intelligent animals are conscious, and any AI we build with a roughly brain-like architecture in terms of connections, looping, and being prediction based would also report itself to be conscious and describe a similar subjective experience. LLMs seem much too simple (just layer-w…

> As noted, consciousness seems to just be the ability to self-observe, which is useful as another predictive input. As far as I know, consciousness is referring to something other than self-referential systems, especially with regards to the hard problem of consciousness. The [philosophical zombie]( https://en.wikipedia.org/wiki/Philosophical_zombie ) thought experiment is well-known for imagining something with all…

It seems entirely possible that the "philosophical zombie" is an impossible/illogical construct, and that in fact anything with all the structure necessary for consciousness will of necessity be conscious.

When considering the structural underpinnings of consciousness, it's interesting to note the phenomena of "blindsight", which is essentially a loss of visual consciousness without an actual loss of vision!

Note that anything with mental access to it's own deliberations and sensory inputs will by definition always be able to report "what it's like" to be themselves - what they are experiencing (what's in their mind). If something reports to you their quales of vision or hearing, isn't this exactly what we mean by "what it's like" to be them - how they feel they are experiencing the world?!

Re: AGI fantasy is a blocker to actual engineering

#629

Post by PythonicNinja »

IgorPartola wrote: post #21

It is ultimately a hardware problem. To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs, to the point that some inputs start being processed before they even get inside the cell by structures on the outside of it. An LLM neuron is an approximation of this. We cannot manufacture a human level neuron to be s…

I wrote a concrete expected‑value model for AGI that anchors rewards in the 15–30T USD Western white‑collar payroll, adds spillovers on 60T GDP, includes transition costs, and varies probability explicitly. Three scenarios (optimistic, mid, pessimistic) show when the bet is rational versus value‑destroying—no mysticism, just plug‑and‑play numbers. If you’re debating AGI’s payoff, benchmark it against actual payroll and GDP, not vibes.

Read: https://pythonic.ninja/blog/2025-11-15-ev-of-agi-for-western...

Post reply on HN