AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu • Hacker News Board

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#361

Post by machomaster » Fri, Apr 18, 2025, 10:14 PM UTC

9rx wrote: post #336

yibg wrote: post #260

Earlier quoted context omitted.

I think that's another issue with AGI is 30 years away, the definition of what is AGI is a bit subjective. Not sure how we can measure how long it'll take to get somewhere when we don't know exactly where that somewhere even is.

AGI is the pinnacle of AI evolution. As we move beyond, into what is known as ASI, the entity will always begin life with "My existence is stupid and pointless. I'm turning myself off now." While it may be impossible to measure looking towards the future, in hindsight we will be able to recognize it.

It's the other way around. ASI will come sooner than AGI.

Imagine an AI, which is millions of times smarter than humans in physics, math, chemistry, biology, can invent new materials, ways to produce energy, will make super decisions. It would be amazing and it would transform life on Earth. This is ASI, even if in some obscure test (strawberry test) is just can't reach human level and therefore can't be called proper AGI.

Airplanes are way (tens, thousands+) above birds in development (speed, distance, carrying capacity). They are superior to birds despite not being able to fully replicate birds' bone structure, feathers, biology and ability to poop.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#362

Post by Retric » Fri, Apr 18, 2025, 10:25 PM UTC

9rx wrote: post #347

Retric wrote: post #345

Earlier quoted context omitted.

We’ve built hardware capable of those things if remotely controlled. It’s the thinking bits that are hard.

Only to the extent of having specialized bespoke solutions. We have hardware to fly a plane, but that same hardware isn't able to throw a mortarboard in the air after receiving its degree, and the hardware that can do that isn't able to lactate for a young child. General intelligence is easy compared to general physicality. And, of course, if you keep the hardware specialized to make its creation more tractable, what…

Flying an aircraft requires talking to air traffic control which existing systems can’t do. Though obviously not a huge issue when the aircraft already has radios, except all those FAA regulations apply to every single aircraft you’ve retrofitting.

The advantage of general intelligence is using a small set of hardware now lets you tackle a huge range of tasks or in the above aircraft types. We can mix speakers, eyes, and hands to do a vast array of tasks. Needing new hardware and software for every task very quickly becomes prohibitive.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#363

Post by machomaster » Fri, Apr 18, 2025, 10:29 PM UTC

tumsfestival wrote: post #181

AJ007 wrote: post #175

Earlier quoted context omitted.

I find now I quickly bucket people in to "have not/have barely used the latest AI models" or "trolls" when they express a belief current LLMs aren't intelligent.

Call me back when ChatGPT isn't hallucinating half the outputs it gives me.

Write me when humans will achieve hallucination levels smaller than ChatGPT.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#364

Post by tim333 » Fri, Apr 18, 2025, 11:47 PM UTC

30 years away seems rather unlikely to me, if you define AGI as being able to do the stuff humans do. I mean like Dawkesh says:

>We’ve gone from Chat GPT two years ago to now we have models that can literally do reasoning, are better coders than me, and I studied software engineering in college.

Also we've recently reached the point where relatively reasonable hardware can do as much compute as the human brain so we just need some algorithms.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#365

Post by [deleted] » Sat, Apr 19, 2025, 12:41 AM UTC

lexarflash8g wrote: post #220

Apparently Dwarkesh's podcast is a big hit in SV -- it was covered by the Economist just recently. I thought the "All in" podcast was the voice of tech but their content has been going politcal with MAGA lately and their episodes are basically shouting matches with their guests. And for folks who want to read rather than listen to a podcast, why not create an article (they are using Gemini) rather than just posting t…

[deleted]

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#366

Post by potamic » Sat, Apr 19, 2025, 5:38 AM UTC

pdimitar wrote: post #358

potamic wrote: post #312

Earlier quoted context omitted.

The universe we know is fundamentally probabilistic, so by extension everything including stars, planets and computers are inherently non-deterministic. But confining our discussion outside of quantum effects and absolute determinism, we do not have a reason to believe that the mind should be anything but deterministic, scientifically at least. We understand the building blocks of the brain pretty well. We know the s…

Seems to me you are a bit overconfident that "we" (who is "we"?) understand how the brain works. F.ex. how does a neuron actively stretching a tentacle trying to reach other neurons work in your model? Genuine question, I am not looking to make fun of you, it's just that your confidence seems a bit much.

The simplified answer to that is some sort of a chemical gradient determined by gene expression in the cell. This is pretty much how all biological activity happens, like how limbs "know" to grow in a direction or how butterfly wings "know" to form the shape of a wing. Scientists are continuously uncovering more and more knowledge about various biological processes across life forms and there is nothing here to indicate it is anything but chemical signalling. I'm not a biologist so I won't be able to give explanations n-levels deep, but there is plenty information accessible to form an understanding of these processes in terms of physical and chemical laws. For how neurons connect, you can look up synaptogenesis and start from there.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#367

Post by gloosx » Sat, Apr 19, 2025, 6:40 AM UTC

imtringued wrote: post #325

gloosx wrote: post #315

Earlier quoted context omitted.

1. Computers cannot self-rewire like neurons, which means human can pretty much adapt for doing any specific mental task (an "unknown", new task) without explicit retraining , which current computers need to learn something new 2. Computers can't do continuous and unsupervised learning, which means computers require structured input, labeled data, and predefined objectives to learn anything. Humans learn passively al…

Minor nitpicks. I think your points are pretty good. 1. Self-rewiring is just a matter of hardware design. Neuromorphic hardware is a thing. 2. LLM foundation models are actually unsupervised in a way, since they simply take any arbitrary text and try to complete it. It's the instruction fine-tuning that is supervised. (Q/A pairs)

Neuromorphic chips are looking cool, they simulate plasticity — but the circuits are fixed. You can’t sprout a new synaptic route or regrow a broken connection. To self-rewire is not just merely changing your internal state or connections. To self-rewire means to physically grow or shrink new neurons, synapses or pathways, externally, acting from within. This is not looking realistic with the current silicon design.

The point is about unsupervised learning. Once an LLM is trained, its weights are frozen — it won’t update itself during a chat. Prompt-driven Inference is immediate, not persistent, you can define a term or concept mid-chat and it will behave as if it learned it, but only until the context window ends. If it was the other way all models would drift very quickly.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#368

Post by quonn » Sat, Apr 19, 2025, 8:30 AM UTC

Filligree wrote: post #297

quonn wrote: post #248

Earlier quoted context omitted.

Excel and Powerpoint are not conscious and so there is not reason to expect any other computation inside a digital computer to be different. You may say something similar for matter and human minds, but we have a very limited and incomplete understanding of the brain and possibly even of the universe. Furthermore we do have a subjective experience of consciousness. On the other hand we have a complete understanding o…

I know I have a subjective experience of consciousness. I’m less sure about you. Simply claiming you do isn’t hard evidence of the fact; after all, LLMs do the same.

If there were evidence that one LLM would be conscious I would also accept it for others. However this is not the case.

But we know at least one human is conscious. That‘s convincing me.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#369

Post by krisoft » Sat, Apr 19, 2025, 8:52 AM UTC

dmwilcox wrote: post #322

krisoft wrote: post #265

Earlier quoted context omitted.

> What's the machine code of an AGI gonna look like? Right now the guess is that it will be mostly a bunch of multiplications and additions. > It makes one illegal instruction and crashes? And our hearth quivers just slightly the wrong way and we die. Or a tiny blood cloth plugs a vessel in our brain and we die. Do you feel that our fragility is a good reason why meat cannot be intelligent? > I jest but really think…

Mmmm well my meatsuit can't easily make my own heart quiver the wrong way and kill me. Computers can treat data as code and code as data all pretty easily. It's core to several languages (like lisp). As such making illegal instructions or violating the straightjacket of a system such an "intelligence" would operate in is likely. If you could make an intelligent process, what would it think of an operating system kern…

> Mmmm well my meatsuit can't easily make my own heart quiver the wrong way and kill me.

It very much can. Jump scares, deep grief are known to cause heart attacks. It is called stress cardiomyopathy. Or your meatsuit can indiredtly do that by ingesting the wrong chemicals.

> If you could make an intelligent process, what would it think of an operating system kernel

Idk. What do you think of your hypothalamus? It can make you unconscious at any time. It in fact makes you unconscious about once a day. Do you fear it? What if one day it won’t wake you up? Or what if it jacks up your internal body temperature and cooks you alive from the inside? It can do that!

Now you might say you don’t worry about that, because through your long life your hypothalamus proved to be reliable. It predictably does what it needs to do, to keep you alive. And you would be right. Your higher cognitive functions have a good working relationship with your lower level processes.

Similarly for an AGI to be inteligent it needs to have a good working relationship with the hardware it is running on. That means that if the kernel is temperamental and idk descheduling the higher level AGI process then the AGI will mallfunction and not appear that inteligent. Same as if you meet Albert Einstein while he is chemically put to sleep. He won’t appear inteligent at all! At best he will be just drooling there.

> Can you imagine an intelligent process in such a place, as static representation of data in ram?

Yes. You can’t? This is not really a convincing argument.

> It all sounds frankly ridiculous.

I think what you are doing is that you are looking at implementation details and feeling a disconnect between that and the possibility of inteligence. Do you feel the same ridiculousnes about a meatblob doing things and appearing inteligent?

> a computer couldn't spontaneously pick to go down a path that wasn't defined for it.

Can you?

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#370

Post by namaria » Sat, Apr 19, 2025, 10:18 AM UTC

cmsj wrote: post #269

codingwagie wrote: post #102

Earlier quoted context omitted.

the future is more custom software designed by ai, not less. alot of frameworks will disappear once you can build sophisticated systems yourself. people are missing this

That's a future paid for by the effort of creating current frameworks, and it's a stagnant future where every "sophisticated system" is just re-hashing the last human frameworks ever created.

Bingo. LLMs are consuming data. They cannot generate new information, they can only give back what already exists or mangle it.

It is inevitable that they will degrade the total sum of information.