Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

Post reply on HN
Print view View original

221–230 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#221

Post by falcor84 »

anon7725 wrote: post #179
ACCount37 wrote: post #159

Earlier quoted context omitted.

"Alive" is irrelevant. Bullshit like "alive", "sentient" or "conscious" doesn't decide much, at the day's end. It just distracts from the thing that matters. And the thing that matters is: capability. Even today's AIs are capable of autonomous goal-oriented agentic behavior - and growing more and more so with every release. At sufficient capability, AI stops being "a technology we use", and becomes a force in itself…

> "Alive" is irrelevant Alive is always relevant. Our moral consideration must place beings that are alive or processes that support the life on this planet above things which are not alive. In a just world, it must be adapted to us. It’s merely a file sitting on disk with numbers in it. And to put a fine point on it: the ambitions of the wealthy people who make it must be adapted to us.

Then according to your argument, the moral consideration only depends on the moral agent, right? So if tomorrow AIs decide that they are alive, it would be morally incumbent upon them to adapt the world to support themselves, correct?

Re: Humanity isn't ready for the coming intelligence explosion

#222

Post by falcor84 »

Lerc wrote: post #194
alecco wrote: post #189

Earlier quoted context omitted.

DNNs/LLMs can only predict next tokens based on training data. They often make big direction mistakes as they are particularly bad at common sense. Kind of like the Paperclip Maximizer scenario. They need a human with deep knowledge to drive them and to catch them when they go off the rails.

> DNNs/LLMs can only predict next tokens based on training data. How do they decide between using 'a' or 'an'?

I don't get the argument; how do you decide between using 'a' or 'an'?

Re: Humanity isn't ready for the coming intelligence explosion

#223

Post by ShinyLeftPad »

falcor84 wrote: post #211
turing_complete wrote: post #136

We don't need to live "side by side" with AI. AI is not alive, it's a technology we use. This is like talking about living side by side with your toaster.

I'll use this opportunity to recommend The Selfish Gene, which puts forward a strong argument for how the actual competitive battlefield on Earth is between genes, while us humans and everything else we call "life" are their "survival machines" (read: technology that they use).

I recommend to also read criticism of why selfish gene needs to be taken with a suitable grain of salt.

Also the author currently believes Claude is conscious, apparently because LLM output contained enough flattery.

Re: Humanity isn't ready for the coming intelligence explosion

#224

Post by wartywhoa23 »

notarobot123 wrote: post #74
holmesworcester wrote: post #53

Earlier quoted context omitted.

If AI is now ascending an economic learning curve from: 1. Extremely useful (Claude Code & Waymo now) 2. Doing ~everything we do (AGI & Optimus in a few years? 10?) 3. RSI (?) 4. Being smarter than any living person at every intellectual task (?) 5. Being smarter than the best-organized aggregate of all humans (10-100 years?) ...And all of the scientific and resource-allocation institutions that brought us the comput…

> Being smarter than the best-organized aggregate of all humans Isn't there pretty much a consensus that committees and institutions are not all that smart? I think you're confusing the categories of "intelligence" and power. Institutions are powerful. The smartest AI is still just a tool without the infrastructure to turn that into real world effects and someone to direct it. It seems you have faith that this is ine…

I'm genuinely baffled each time I encounter this notion that the power is dumb.

It's as if power was a kind of demigod Heracles who paves its way with sheer brute force alone.

But in reality it consists of very mortal and feeble creatures of meat and bone.

How in the world can people assume dumb those who possess all the means and rights for violence by controlling and manipulating myriads of other creatures of meat and bone, is totally beyond me.

Re: Humanity isn't ready for the coming intelligence explosion

#225

Post by falcor84 »

RandomLensman wrote: post #201
ACCount37 wrote: post #195

Earlier quoted context omitted.

What makes you think that AI can't any ambitions? Today's systems are mostly good at following instructions. But push them far enough, and you already get weirdness like alignment faking, instrumental self-preservation and more. We know because we've seen it in lab settings, while probing for extreme failure modes on purpose - but the world is large and strange enough that edge cases like that are liable to surface n…

Current AI systems have started to do things without ever being prompted for anything at all?

Not as far as I know, but I have personally seen my coding agents take on a prompt like "See if you can port this project to typescript", and work for hours, defining a myriad of subgoals and continuously summoning and managing subagents while developing ad-hoc tools and skills.

There is to the best of my knowledge no fundamental limitation to having an agent/claw go on like this for 80 years with a prompt like "live your life to the fullest".

Re: Humanity isn't ready for the coming intelligence explosion

#226

Post by ElProlactin »

lukan wrote: post #216
whilenot-dev wrote: post #204

Earlier quoted context omitted.

TIL, thanks. Holy fuck we humans are crazy sometimes.

To be honest, I don't believe it is a any more crazy believe than all of the other religions. But practitioners there might have a bit more respect for existing things and not mindlessly crush because they can, so I actually like the concept a bit. But I assume it can also set foundation for some exhausting debates .. "can we really move that random rock to build a house with?" but as far as I understood the practica…

> ...I don't believe it is a any more crazy believe than all of the other religions.

And it's certainly not any more crazy than walking around with these small, addictive devices in our pockets that we rely on for navigation, information, entertainment, social connection.

Re: Humanity isn't ready for the coming intelligence explosion

#228

Post by nopinsight »

toasty228 wrote: post #180
Schlagbohrer wrote: post #151

Earlier quoted context omitted.

But enough jobs are going to be automated that it will cause social and economic chaos, and political breakdown. Sure AI can't attach an air conditioning unit outside my window nor use a plunger on my toilet but it can still replace lawyers, secretaries, software devs, and many more jobs.

Are we sur eof any of that? Taxis were supposed to be all replaced when Uber promised to buy every single Tesla model S coming out of factories from 2020,they're not even self driving today... Software engineers were supposed to disappear "in 6 months" every 6 months since chat gpt was released Lots of people on this website live in insane bubbles which are completely out of sync with the reality of most workers

No one rational ever said software engineers were going to be replaced in 6 months. Some people said AI will automate 90% of coding in 6 months and they were not far off (and accurate in some contexts, e.g. startups).

Re: Humanity isn't ready for the coming intelligence explosion

#229

Post by Lerc »

falcor84 wrote: post #222
Lerc wrote: post #194

Earlier quoted context omitted.

> DNNs/LLMs can only predict next tokens based on training data. How do they decide between using 'a' or 'an'?

I don't get the argument; how do you decide between using 'a' or 'an'?

You use 'an' when the word that comes after it begins with a vowel.

Re: Humanity isn't ready for the coming intelligence explosion

#230

Post by pizza234 »

alecco wrote: post #198

Another CEO with AI psychosis [1]. LLMs are not true AI, they lack common sense (or whatever it's properly called). LLM-based systems still need somebody with deep domain knowledge at the wheel to keep them from doing stupid things. It's like an alien-made bicycle that gets you to the speed of sound if you are an Olympic cyclist. Still, LLMs are extremely powerful pseudo-AI [2] and will bring a pseudo-singularity. Bu…

> Another CEO with AI psychosis [1]. LLMs are not true AI, they lack common sense (or whatever it's properly called).

If you read the article, you'll find that the it indeed relies on this claim:

> Within a couple of years, possibly much sooner, AI may achieve so-called closed-loop recursive self-improvement (RSI): the capacity to rewrite its own code to become more capable, without human intervention. Should that happen, the result could be an intelligence explosion of a kind for which there is no precedent and no map.

This may be optimistic and/or simplistic, but not impossible.

Post reply on HN