Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

Post reply on HN
Print view View original

201–210 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#201

Post by RandomLensman »

ACCount37 wrote: post #195
baxtr wrote: post #170

Earlier quoted context omitted.

What does AI want? Without ambitions you have to be told what to do.

What makes you think that AI can't any ambitions? Today's systems are mostly good at following instructions. But push them far enough, and you already get weirdness like alignment faking, instrumental self-preservation and more. We know because we've seen it in lab settings, while probing for extreme failure modes on purpose - but the world is large and strange enough that edge cases like that are liable to surface n…

Current AI systems have started to do things without ever being prompted for anything at all?

Re: Humanity isn't ready for the coming intelligence explosion

#202

Post by dbspin »

weregiraffe wrote: post #91
JuniperMesos wrote: post #87

Earlier quoted context omitted.

I see driverless cars all the time in San Francisco and I'm eagerly awaiting the regulatory decisions that will allow them to be widely available in the bay area at large.

It's not driverless until you can sleep while it drives.

It's not driverless unless it doesn't have a driver. Some poor guy in hydrobad 'monitoring' (read frequently taking over from) ten cars at a time, while being paid a statistically insignificant portion of a western taxi drivers salary does not count as 'self driving'.

Re: Humanity isn't ready for the coming intelligence explosion

#203

Post by esailija »

threethirtytwo wrote: post #135
al_borland wrote: post #73

Earlier quoted context omitted.

From the 2nd link I posted: > Anthropic CEO Dario Amodei, who once claimed AI could eliminate 50% of white-collar jobs, now says automation may actually expand the work people do. Is pivoting from 50% elimination to actually expanding work not a drastic enough? Do you think AI has eliminated writing code? I still write code every day. The AI is more a thing I ask questions and it gives me right answers about 40% of t…

YOU still write code. I would say over 65-70% of modern programmers no longer write code. The paradigm shift happened within the last 365 days.

65-70% "programmers" never wrote code in the first place. Before AI cargo culters just copy pasted code from tutorials, stack overflow, github and so on. With AI they merely continue not writing code. Some people didn't do anything, not even copy paste, see r/overemployed although I have not checked it after AI... maybe it's no longer possible to hold a dev job without actually doing anything.

Programmers still write code. They have enough expertise from actually programming to understand that the untouched output from AI is not good enough to be professionally responsible for.

Re: Humanity isn't ready for the coming intelligence explosion

#204

Post by whilenot-dev »

lukan wrote: post #200
whilenot-dev wrote: post #191

Earlier quoted context omitted.

Just because something can impact a living system/organism, that doesn't mean that that thing is alive. An asteroid colliding with us on Earth could have tremendous consequences, but nobody would argue that an asteroid is alive.

"but nobody would argue that an asteroid is alive" Many animists would disagree: https://en.wikipedia.org/wiki/Animism

TIL, thanks. Holy fuck we humans are crazy sometimes.

Re: Humanity isn't ready for the coming intelligence explosion

#205

Post by alecco »

Lerc wrote: post #194
alecco wrote: post #189

Earlier quoted context omitted.

DNNs/LLMs can only predict next tokens based on training data. They often make big direction mistakes as they are particularly bad at common sense. Kind of like the Paperclip Maximizer scenario. They need a human with deep knowledge to drive them and to catch them when they go off the rails.

> DNNs/LLMs can only predict next tokens based on training data. How do they decide between using 'a' or 'an'?

They pick random top-k next token based on their amazing 4chan/reddit training data, duh.

Re: Humanity isn't ready for the coming intelligence explosion

#206

Post by ccozan »

tedggh wrote: post #166

Why should we assume AI can rapidly turn into super intelligence when physical and critical resources like energy and materials remain under human control?

This is a valid concern for any runaway AI, but if energy is limited, then the AI would work on efficiency.

Still, Ai can move into physical space, as more and more robots of all kind are unleashed.

Re: Humanity isn't ready for the coming intelligence explosion

#207

Post by discreteevent »

seydor wrote: post #199

Oh I'm ready. I ve had enough of useless tech that evolved to a global network where people vent their frustration about technology. Let's do some cool stuff instead

AI is designed to produce variations on a theme. Not sure why you think it will produce something different.

Re: Humanity isn't ready for the coming intelligence explosion

#208

Post by latexr »

bob1029 wrote: post #186
seydor wrote: post #141

Earlier quoted context omitted.

In the pecking order, it is under, not on your side. Your toaster never demanded you to make a toast.

Once you actually use the toaster the hierarchy inverts. The toaster runs your entire household while it is powered on. Unattended ChatGPT will rarely burn your breakfast or start fires.

> Once you actually use the toaster the hierarchy inverts. The toaster runs your entire household while it is powered on.

More than once I have gone to my toaster in the morning and found cold bread there that someone toasted the day before and forgot about. Because obviously you push the toast down and go do something else. You don’t need to be staring at it continuously, nor are other people impacted by your toast unless they are waiting to make their own. Toasting bread doesn’t have to affect an entire household.

Re: Humanity isn't ready for the coming intelligence explosion

#209

Post by wartywhoa23 »

consumer451 wrote: post #88
holmesworcester wrote: post #53

Earlier quoted context omitted.

If AI is now ascending an economic learning curve from: 1. Extremely useful (Claude Code & Waymo now) 2. Doing ~everything we do (AGI & Optimus in a few years? 10?) 3. RSI (?) 4. Being smarter than any living person at every intellectual task (?) 5. Being smarter than the best-organized aggregate of all humans (10-100 years?) ...And all of the scientific and resource-allocation institutions that brought us the comput…

Here is an issue I think about often, but I am not quite sure how to put it into words. We have many extremely smart people in various fields. Executives, politicians, and society generally ignore them and do whatever they want. I don't believe that lack of access to intelligence is our problem. How is "free" intelligence going to improve this? I don't just mean climate, but business planning, health, risk assessment…

Exactly.

If you're so hellbent on building the Tower of Babel 2.0, at least build it on a firm foundation.

Current construction is taking place on a millenia old bog.

Post reply on HN