Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I don't think AGI is imminent

dlants.me

Post reply on HN
Print view View original

271–280 of 313 posts

Re: Why I don't think AGI is imminent

#271

Post by anonymid »

rfv6723 wrote: post #199
FloorEgg wrote: post #175

https://archive.is/D4EYW For anyone seeing 404

The skepticism surrounding AGI often feels like an attempt to judge a car by its inability to eat grass. We treat "cognitive primitives" like object constancy and causality as if they are mystical, hardwired biological modules, but they are essentially just high-dimensional labels for invariant relationships within a physical manifold. Object constancy is not a pre-installed software patch; it is the emergent realiza…

Hey, thanks for responding. You're a very evocative writer!

I do want to push back on some things:

> We treat "cognitive primitives" like object constancy and causality as if they are mystical, hardwired biological modules, but they are essentially just

I don't feel like I treated them as mystical - I cite several studies that define what they are and correlate them to certain structures in the brain that have developed millennia ago. I agree that ultimately they are "just" fitting to patterns in data, but the patterns they fit are really useful, and were fundamental to human intelligence.

My point is that these cognitive primitives are very much useful for reasoning, and especially the sort of reasoning that would allow us to call an intelligence general in any meaningful way.

> This "all-at-once" calculation of relationships is fundamentally more powerful than the biological need to loop signals until they stabilize into a "thought."

The argument I cite is from complexity theory. It's proof that feed-forward networks are mathematically incapable of representing certain kinds of algorithms.

> Furthermore, the obsession with "fragility"—where a model solves quantum mechanics but fails a child’s riddle—is a red herring.

AGI can solve quantum mechanics problems, but verifying that those solutions are correct still (currently) falls to humans. For the time being, we are the only ones who possess the robustness of reasoning we can rely on, and it is exactly because of this that fragility matters!

Re: Why I don't think AGI is imminent

#272

Post by CamperBob2 »

lostmsu wrote: post #251
DangitBobby wrote: post #249

Earlier quoted context omitted.

The top reasoning models suggest taking a car to the car wash.

Not 100% of time according to comments.

SotA doesn't matter, though. Only the first couple of time derivatives matter. Looking good for the clankers, not so much for us...

Re: Why I don't think AGI is imminent

#273

Post by MadcapJake »

> What if we built simulated environments where AIs could gather embodied experience? Would we be able to create learning scenarios where agents could learn some of these cognitive primitives, and could that generalize to improve LLMs? There are a few papers that I found that poke in this direction.

Simulation Theory boosted! We're all just models in training.

Re: Why I don't think AGI is imminent

#274

Post by al_borland »

charcircuit wrote: post #269
al_borland wrote: post #234

Earlier quoted context omitted.

How much time will someone realistically dedicate to this if they need to have a separate day job? How good will they get without mentors? How much complexity will they really need to manage without the bureaucracy of an organization? Are senior software engineers of the future going to be waiting tables along side actors for the first 10+ years of their adult life, working on side projects on nights and weekends, ho…

>How much time will someone realistically dedicate to this if they need to have a separate day job? Typically parents subsidize the living of their children while they are still learning. >Most of the technology I’ve had access to isn’t something a solo hobbyist dev will ever touch That's already true today. Most developers are react developers. If hired for something else they will have to pick that up on the job. W…

Parents can only subsidize children if they are doing well themselves, most aren’t.

That “learning” phase used to end in the 18-25 range. Getting rid of juniors and making someone get enough experience on side projects to be considered a senior would take considerably longer. Exactly how long are parents supposed to be subsidizing their children’s living expenses? How can the parents afford to retire when they still have dependents? And all of this is built on the hope that the kid will actually land that job in 10 years? That feels like a bad bet. What happens if they fail? Not a big deal when the kid is 27, but a pretty big deal at 40 when they have no other marketable skills and have been living off their parents.

The difference is there are juniors getting familiar with those enterprise products today. If they go away, they will step into it as senior people and be unprepared. It’s not just about the syntax of a different language, I’m talking more about dealing with things like Active Directory, leveraging ITSM systems effectively, reporting, metrics, how to communicate with leadership, how to deal with audits. AI might help with some of this, but not all of it. For someone without experience with it, they don’t know what they don’t know… in which case the AI won’t help at all.

I even see this when dealing with people from a small company being acquired by a larger company. They don’t know what is available to them or the systems that are in place, and they don’t even know enough to ask. Someone from another large company knows to ask about these things, because they have that experience.

Re: Why I don't think AGI is imminent

#275

Post by al_borland »

joquarky wrote: post #264
al_borland wrote: post #229

Earlier quoted context omitted.

> These models are demonstrating an incredible capacity for logical abstract reasoning of a level far greater than 99.9% of the world's population. And yet they have trouble knowing that a person should take their car to a car wash. I also saw a college professor who put various AI models through all his exams for a freshman(?) level class. Most failed, I think one barely passed, if I remember correctly. I’ve been re…

So what's the underlying message here? Don't prepare?

To remain skeptical of extraordinary claims.

Re: Why I don't think AGI is imminent

#276

Post by esafak »

latentsea wrote: post #188
esafak wrote: post #68

Earlier quoted context omitted.

An agent need not have wants, so why would it try to increase its efficiency to obtain things?

Just put "keep yourself alive" in the SOUL.md. Might be all that it takes.

I swear people don't know what's good for them.

Re: Why I don't think AGI is imminent

#277

Post by causal »

al_borland wrote: post #239
causal wrote: post #58

Earlier quoted context omitted.

This is my greatest cause for alarm regarding LLM adoption. I am not yet sure AI will ever be good enough to use without experts watching them carefully; but they are certainly good enough that non-experts cannot tell the difference.

My dad is retired and enamored with ChatGPT. He’s been teaching classes to seniors and evangelizing the use to all his friends. Every time he calls he gives me an update on who he’s converted into a ChatGPT user. He seems disappointed with anyone who doesn’t use it for everything after he tells them about it. A couple days ago he was telling me one lady he was trying to sell on it wouldn’t use it. She took the positi…

Thanks for sharing that anecdote. I think everyone is susceptible to misinformation, and seniors might be especially unprepared to adapt to LLMs tricks.

Re: Why I don't think AGI is imminent

#278

Post by therobots927 »

kjkjadksj wrote: post #270
tim333 wrote: post #232

Earlier quoted context omitted.

What's happening with AGI depends on what you mean by AGI so "can't even be bothered discussing what the definition" means you can't say what's happening. My usual way of thinking about it is AGI means can do all the stuff humans do which means you'd probably after a while look out the window and see robots building houses and the like. I don't think that's happening for a while yet.

Who would the robots build houses for? No one has a job and no one is having kids in that future.

The billionaire elite. Isn’t it obvious? They want to get rid of us

Re: Why I don't think AGI is imminent

#279

Post by tim333 »

danaris wrote: post #268
tim333 wrote: post #232

Earlier quoted context omitted.

What's happening with AGI depends on what you mean by AGI so "can't even be bothered discussing what the definition" means you can't say what's happening. My usual way of thinking about it is AGI means can do all the stuff humans do which means you'd probably after a while look out the window and see robots building houses and the like. I don't think that's happening for a while yet.

Indeed: particularly given that—just as a nonexhaustive "for instance"—one of the fairly common things expected in AGI is that it's sapient . Meaning, essentially, that we have created a new life form, that should be given its own rights. Now, I do not in the least believe that we have created AGI, nor that we are actually close. But you're absolutely right that we can't just handwave away the definitions. They are c…

I'm not sure how the rights thing will go. Humans have proved quite able not to give many rights to animals or other groups of humans even if they are quite smart. Then again there was that post yesterday with a lady accusing OpenAI of murdering her AI boyfriend by turning off 4o so no doubt there will be lots of arguments over that stuff. (https://news.ycombinator.com/item?id=47020525)

Re: Why I don't think AGI is imminent

#280

Post by elfly »

kjkjadksj wrote: post #270
tim333 wrote: post #232

Earlier quoted context omitted.

What's happening with AGI depends on what you mean by AGI so "can't even be bothered discussing what the definition" means you can't say what's happening. My usual way of thinking about it is AGI means can do all the stuff humans do which means you'd probably after a while look out the window and see robots building houses and the like. I don't think that's happening for a while yet.

Who would the robots build houses for? No one has a job and no one is having kids in that future.

Where are the robots going to sleep? Outside in the rain?
Post reply on HN