Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

dwarkesh.com

Post reply on HN
Print view View original

261–270 of 394 posts

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#261

Post by dmwilcox »

preommr wrote: post #246
dmwilcox wrote: post #213

I've been saying this for a decade already but I guess it is worth saying here. I'm not afraid AI or a hammer is going to become intelligent (or jump up and hit me in the head either). It is science fiction to think that a system like a computer can behave at all like a brain. Computers are incredibly rigid systems with only the limited variance we permit. "Software" is flexible in comparison to creating dedicated ci…

> why is it so hard to get a cryptographically secure random number? Because computers are pure unadulterated determinism Then you've missed the part of software. Software isn't computer science, it's not always about code. It's about solving problems in a way we can control and manufacture. If we needed random numbers, we could easily use a hardware that uses some physics property, or we could pull in an observation…

It's not about the random numbers it's about the tree of possibilities having to be defined up front (in software or hardware). That all inputs should be defined and mapped to some output and that this process is predictable and reproducible.

This makes computers incredibly good at what people are not good at -- predictably doing math correctly, following a procedure, etc.

But because all of the possibilities of the computer had to be written up as circuitry or software beforehand, it's variability of outputs is constrained to what we put into it in the first place (whether that's a seed for randomness or model weights).

You can get random numbers and feed it into the computer but we call that "fuzzing" which is a search for crashes indicating unhandled input cases and possible bugs or security issues.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#262

Post by yibg »

threatofrain wrote: post #205
timewizard wrote: post #171

Earlier quoted context omitted.

That we don't have a single unified explanation doesn't mean that we don't have very good hints, or that we don't have very good understandings of specific components. Aside from that the measure really, to me, has to be power efficiency. If you're boiling oceans to make all this work then you've not achieved anything worth having. From my calculations the human brain runs on about 400 calories a day. That's an absur…

We'll be experiencing extreme social disruption well before we have to worry about the cost-efficiency of strong AI. We don't even need full "AGI" to experience socially momentous change. We might even be on the verge of self driving cars spreading to more cities. We don't need very powerful AI to do very powerful things.

It's not just a energy cost issue with AGI though. With autonomous vehicles we might not have the technology, but we can build a good mental model of what the thing can look like and how various pieces can function long before we get there. We have different classifications of incremental steps to get there as well. e.g. level 1, 2 and so on where we can make incremental progress.

With AGI, as far as I know, no one has a good conceptual model of what a functional AGI even looks like. LLM is all the rage now, but we don't even know if it's a stepping stone to get to AGI.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#263

Post by yibg »

babyent wrote: post #190
yibg wrote: post #159

Earlier quoted context omitted.

I think having a real life JARVIS would be super cool and useful, especially if it's plugged into various things and can take action. Yes, also potentially dangerous, but I want to feel like Ironman.

Except only Iron Man had JARVIS.

Spidey got EDITH :)

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#264

Post by cmsj »

Workaccount2 wrote: post #179
kmeisthax wrote: post #127

Earlier quoted context omitted.

Because that path lies skill atrophy. AI research has a thing called "the bitter lesson" - which is that the only thing that works is search and learning. Domain-specific knowledge inserted by the researcher tends to look good in benchmarks but compromise the performance of the system[0]. The bitter- er lesson is that this also applies to humans. The reason why humans still outperform AI on lots of intelligence tasks…

>Because that path lies skill atrophy. I wonder how many programmers have assembly code skill atrophy? Few people will weep the death of the necessity to use abstract logical syntax to communicate with a computer. Just like few people weep the death of having to type out individual register manipulations.

I would say there's a big difference with AI though.

Assembly is just programming. It's a particularly obtuse form of programming in the modern era, but ultimately it's the same fundamental concepts as you use when writing JavaScript.

Do you learn more about what the hardware is doing when using assembly vs JavaScript? Yes. Does that matter for the creation and maintenance of most software? Absolutely not.

AI changes that, you don't need to know any computer science concepts to produce certain classes of program with AI now, and if you can keep prompting it until you get what you want, you may never need to exercise the conceptual parts of programming at all.

That's all well and good until you suddenly do need to do some actual programming, but it's been months/years since you last did that and you now suck at it.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#265

Post by krisoft »

dmwilcox wrote: post #251
CooCooCaCha wrote: post #221

Earlier quoted context omitted.

This is why I think philosophy has become another form of semi-religious kookery. You haven't provided any actual proof or logical reason for why a computer couldn't be intelligent. If randomness is required then sample randomness from the real world. It's clear that your argument is based on feels and you're using philosophy to make it sound more legitimate.

I tried to keep my long post short so I cut things. I gestured at it -- there is nothing in a computer we didn't put there. Take the same model weights give it the same inputs, get the same outputs. Same with the pseudo-random number generator. And the "same inputs" is especially limited versus what humans are used to. What's the machine code of an AGI gonna look like? It makes one illegal instruction and crashes? If…

> What's the machine code of an AGI gonna look like?

Right now the guess is that it will be mostly a bunch of multiplications and additions.

> It makes one illegal instruction and crashes?

And our hearth quivers just slightly the wrong way and we die. Or a tiny blood cloth plugs a vessel in our brain and we die. Do you feel that our fragility is a good reason why meat cannot be intelligent?

> I jest but really think about the metal.

Ok. I'm thinking about the metal. What should this thinking illuminate?

> The inside of modern computers is tightly controlled with no room for anything unpredictable.

Let's assume we can't make AGI because we need randomness and unpredictability in our computers. We can very easily add unpredictability. The simple and stupid solution is to add some sensor (like a camera CCD) and stare at the measurement noise. You don't even need a lens on that CCD. You can cap it so it sees "all black", and then what it measures is basically heat noise of the sensors. Voila. Your computer has now unpredictability. People who actually make semiconductors probably can come up with even simpler and easier ways to integrate unpredictability right on the same chip we compute with.

You still haven't really argued why you think "unpredictableness" is the missing component of course. Beside the fact that it just feels right to you.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#266

Post by [deleted] »

krupan wrote: post #215
AstroBen wrote: post #192

Earlier quoted context omitted.

Artificial general intelligence (AGI) refers to the hypothetical intelligence of a machine that possesses the ability to understand or learn any intellectual task that a human being can. Generalization ability and Common Sense Knowledge [1] If we go by this definition then there's no caring, or a noticing of drudgery? It's simply defined by its ability to generalize solving problems across domains. The narrow AI that…

It's really hard to picture general intelligence that's useful that doesn't have any intrinsic motivation or initiative. My biggest complaint about LLMs right now is that they lack those things. They don't care even if they give you correct information or not and you have to prompt them for everything! That's not anything close to AGI. I don't know how you get to AGI without it developing preferences, self-motivation…

[deleted]

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#267

Post by glial »

Balgair wrote: post #203
yibg wrote: post #162

Might as well be 10 - 1000 years. Reality is no one knows how long it'll take to get to AGI, because: 1) No one knows what exactly makes humans "intelligent" and therefore 2) No one knows what it would take to achieve AGI Go back through history and AI / AGI has been a couple of decades away for several decades now.

I'm reminded of the the old adage: You don't have to be faster than the bear, just faster than the hiker next to you. To me, the Ashley Madison hack in 2015 was 'good enough' for AGI. No really. You somehow managed to get real people to chat with bots and pay to do so. Yes, caveats about cheaters apply here, and yes, those bots are incredibly primitive compared to today. But, really, what else do you want out of the…

For me it was twitter bots during the 2016 election, but same principle.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#268

Post by cmsj »

ben_w wrote: post #132
kazinator wrote: post #117

Earlier quoted context omitted.

So you could stick your own copyright notice on the result, for one thing.

What's the point holding copyright on a new technical solution, to a problem that can be solved by anyone asking an existing AI, trained on last year's internet, independently of your new copyright?

There is one very specific risk worth mentioning: AI code is a potentially existential crisis for Open Source.

An ecosystem that depends on copyright can't exist if its codebase is overrun by un-copyrightable code.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#269

Post by cmsj »

codingwagie wrote: post #102
torginus wrote: post #56

Earlier quoted context omitted.

Dunno, in work we wanted to implement a task runner that we could use to periodically queue tasks through a web UI - it would then spin up resources on AWS and track the progress and archive the results. We looked at the existing solutions, and concluded that customizing them to meet all our requirements would be a giant effort. Meanwhile I fed the requirement doc into Claude Sonnet, and with about 3 days of promptin…

the future is more custom software designed by ai, not less. alot of frameworks will disappear once you can build sophisticated systems yourself. people are missing this

That's a future paid for by the effort of creating current frameworks, and it's a stagnant future where every "sophisticated system" is just re-hashing the last human frameworks ever created.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#270

Post by recursive »

missingrib wrote: post #242
ggreer wrote: post #237

Earlier quoted context omitted.

Is there any specific mental task that an average human is capable of that you believe computers will not be able to do? Also does this also mean that you believe that brain emulations (uploads) are not possible, even given an arbitrary amount of compute power?

Yes, they can't have understanding or intentionality.

Coincidentally, there is no falsifiable/empirical test for understanding or intentionality.
Post reply on HN