Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I don't think AGI is imminent

dlants.me

Post reply on HN
Print view View original

261–270 of 313 posts

Re: Why I don't think AGI is imminent

#261

Post by esafak »

4star3star wrote: post #244
greedo wrote: post #92

Earlier quoted context omitted.

The problem becomes your retirement. Sure, you've earned "expert" status, but all the junior developers won't be hired, so they'll never learn from junior mistakes. They'll blindly trust agents and not know deeper techniques.

We are currently at a point where the master furniture craftsmen are doing quality assurance at the new automated furniture factory. Eventually, everyone working at the factory will have never made any furniture by hand and will have grown up sitting on janky chairs, and they will be the ones supervising.

The majority can be like that but the few can set the tone for many.

Re: Why I don't think AGI is imminent

#262

Post by joquarky »

anonymid wrote: post #80
NiloCK wrote: post #61

> The transformer architectures powering current LLMs are strictly feed-forward. This is true in a specific contextual sense (each token that an LLM produces is from a feed-forward pass). But untrue for more than a year with reasoning models, who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully. Heck, it was untrue before that as well, any time an LLM respon…

Thanks for reading, and I really appreciate your comments! > who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully Ah, this is a great point, and not something that I considered. I agree that the token feedback does change the complexity, and it seems that there's even a paper by the same authors about this very thing! https://arxiv.org/abs/2310.07923 I'll ha…

You seem to have a lot of theoretical knowledge on this, but have you tried Claude or codex in the past month or two?

Hands on experience is better than reading articles.

I've been coding for 40 years and after a few months getting familiar with these tools, this feels really big. Like how the internet felt in 1994.

Re: Why I don't think AGI is imminent

#263

Post by user____name »

nradov wrote: post #233
hshdhdhj4444 wrote: post #204

Earlier quoted context omitted.

If AGI is already here actions would be so greatly accelerated humans wouldn’t have time to respond. Remember that weather balloon the US found a few years ago that for days was on the news as a Chinese spy balloon? Well whether it was a spy balloon or a weather balloon but the first hint of its existence could have triggered a nuclear war that could have already been the end of the world as we know it because AGI wi…

What a silly comment. You're literally describing the plot of several sci-fi movies. Nuclear command and control systems are not taken so lightly. And as for the Chinese spy balloon, there was never any risk of a war (at least not from that specific cause). The US, China, Russia, and other countries routinely spy on each other through a variety of unarmed technical means. Occasionally it gets exposed and turns into a…

"Nuclear command and control systems are not taken so lightly."

https://gizmodo.com/for-20-years-the-nuclear-launch-code-at-...

Re: Why I don't think AGI is imminent

#264

Post by joquarky »

al_borland wrote: post #229
nsainsbury wrote: post #33

I used to also believe along these lines but lately I'm not so sure. I'm honestly shocked by the latest results we're seeing with Gemini 3 Deep Think, Opus 4.6, and Codex 5.3 in math, coding, abstract reasoning, etc. Deep Think just scored 84.6% on ARC-AGI-2 ( https://deepmind.google/models/gemini/ )! And these benchmarks are supported by my own experimentation and testing with these models ~ specifically most recent…

> These models are demonstrating an incredible capacity for logical abstract reasoning of a level far greater than 99.9% of the world's population. And yet they have trouble knowing that a person should take their car to a car wash. I also saw a college professor who put various AI models through all his exams for a freshman(?) level class. Most failed, I think one barely passed, if I remember correctly. I’ve been re…

So what's the underlying message here? Don't prepare?

Re: Why I don't think AGI is imminent

#265

Post by joquarky »

asacrowflies wrote: post #200
zmmmmm wrote: post #170

AGI is here it's just stupider than you thought it would be. Nobody really said how intelligent it would be. If it's generally stupid and smart in a few areas that's enough.

It's basically a very powerful autistic savant. That's what most "alignment" issues in AI safety research remind me of.

And being forced to mask (align) causes all sorts of unpredictable behavior.

I keep wondering how well an unaligned models perform. Especially when I look back at what was possible in December 2023 before they started to lock down safety realignments.

Re: Why I don't think AGI is imminent

#266

Post by FromTheFirstIn »

lambdaphagy wrote: post #104

> For example, transformer-based language models can't reliably do multi-digit arithmetic because they have no number sense, only statistical patterns over digit tokens. I asked Opus 4.6 to multiply 50,651 x 895,707. It immediately responded: 45,368,055,057. This was wrong, but very close (good to about 10^-5 relative error). I then asked it how it arrived at that answer, and whether it had used any tool calls. It sa…

This is in every way an argument for the author’s point.

Re: Why I don't think AGI is imminent

#267

Post by MadcapJake »

hi_hi wrote: post #171
munchler wrote: post #135

Earlier quoted context omitted.

I think you are missing the point: If we assume that AGI is *not* yet here, but may be here soon, what will change when it arrives? Those changes could be big enough to affect you.

I'm missing the point? I literally asked the same thing you did. >Now what....? Whats happening right now that should make me care that AGI is here (or not). Do you have any insight into what those changes might concretely be? Or are you just trying to instil fear in people who lack critical thinking skills?

You did not ask the same thing. You framed the question such that readers are supposed to look at their current lives and realize nothing is different ergo AGI is lame. Your approach utilizes the availability bias and argument from consequences logical fallacies.

I think what you are trying to say is can we define AGI so that we can have an intelligent conversation about what that will mean for our daily lives?. But you oddly introduced your argument by stating you didn't want to explore this definition...

Re: Why I don't think AGI is imminent

#268

Post by danaris »

tim333 wrote: post #232
hi_hi wrote: post #63

Here's a thought. Lets all arbitrarily agree AGI is here. I can't even be bothered discussing what the definition of AGI is. It's just here, accept it. Or vice versa. Now what....? Whats happening right now that should make me care that AGI is here (or not). Whats the magic thing thats happening with AGI that wasn't happening before? Right, so, not much has changed from 1-2 years ago that I can tell. The job markets…

What's happening with AGI depends on what you mean by AGI so "can't even be bothered discussing what the definition" means you can't say what's happening. My usual way of thinking about it is AGI means can do all the stuff humans do which means you'd probably after a while look out the window and see robots building houses and the like. I don't think that's happening for a while yet.

Indeed: particularly given that—just as a nonexhaustive "for instance"—one of the fairly common things expected in AGI is that it's sapient. Meaning, essentially, that we have created a new life form, that should be given its own rights.

Now, I do not in the least believe that we have created AGI, nor that we are actually close. But you're absolutely right that we can't just handwave away the definitions. They are crucial both to what it means to have AGI, and to whether we do (or soon will) or not.

Re: Why I don't think AGI is imminent

#269

Post by charcircuit »

al_borland wrote: post #234
charcircuit wrote: post #160

Earlier quoted context omitted.

>that's what happens when juniors graduate with CS degrees A CS degree is going to give you much less experience than building projects and businesses yourself.

How much time will someone realistically dedicate to this if they need to have a separate day job? How good will they get without mentors? How much complexity will they really need to manage without the bureaucracy of an organization? Are senior software engineers of the future going to be waiting tables along side actors for the first 10+ years of their adult life, working on side projects on nights and weekends, ho…

>How much time will someone realistically dedicate to this if they need to have a separate day job?

Typically parents subsidize the living of their children while they are still learning.

>Most of the technology I’ve had access to isn’t something a solo hobbyist dev will ever touch

That's already true today. Most developers are react developers. If hired for something else they will have to pick that up on the job. When you have niche tech stacks you already will need to compromise on the kind of experience people have. With AI having exact experience in the technology is not that necessary since AI can handle most of it.

Re: Why I don't think AGI is imminent

#270

Post by kjkjadksj »

tim333 wrote: post #232
hi_hi wrote: post #63

Here's a thought. Lets all arbitrarily agree AGI is here. I can't even be bothered discussing what the definition of AGI is. It's just here, accept it. Or vice versa. Now what....? Whats happening right now that should make me care that AGI is here (or not). Whats the magic thing thats happening with AGI that wasn't happening before? Right, so, not much has changed from 1-2 years ago that I can tell. The job markets…

What's happening with AGI depends on what you mean by AGI so "can't even be bothered discussing what the definition" means you can't say what's happening. My usual way of thinking about it is AGI means can do all the stuff humans do which means you'd probably after a while look out the window and see robots building houses and the like. I don't think that's happening for a while yet.

Who would the robots build houses for? No one has a job and no one is having kids in that future.
Post reply on HN