Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

241–250 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#241

We don't need to live "side by side" with AI. AI is not alive, it's a technology we use. This is like talking about living side by side with your toaster.

I'll use this opportunity to recommend The Selfish Gene, which puts forward a strong argument for how the actual competitive battlefield on Earth is between genes, while us humans and everything else we call "life" are their "survival machines" (read: technology that they use).

Genes have had their time.

Re: Humanity isn't ready for the coming intelligence explosion

#242

Earlier quoted context omitted.

Current AI systems have started to do things without ever being prompted for anything at all?

All the time, yes. But you have to keep two things separate in your thinking: - Prompted as in prompt made of tokens -> for LLMs, tokens double as a clock signal. Time only flows when tokens are pushed through them. - Prompted as in specific request placed in the stream of tokens -> Yeah, they do that all the time whether it's getting into infinite loops of repeating same pattern, or suddenly deciding to do things ba…

So without a token pushed into them they do something? Not sure I understand...

In the current UIs is there a lot of suppression then as I have not seen things start on their own?

I meant an LLM doing something without any external prompt at all. Not doing something different etc but rather do something without a token/prompt ever flowing to it.

Re: Humanity isn't ready for the coming intelligence explosion

#243

We don't need to live "side by side" with AI. AI is not alive, it's a technology we use. This is like talking about living side by side with your toaster.

I think most people would understand something like living "side by side with cars".

Even if not alive, you can live very close and shape your life by and with technology.

I don't take any issue with the OP's formulation.

Re: Humanity isn't ready for the coming intelligence explosion

#244
post #215

Earlier quoted context omitted.

You might be trolling, but I like Schrödingers definition of life: living things avoid decay into equilibrium. Inference engines are trying to converge any input to a certain limit, and I presume that's close enough to an equilibrium to not count as living thing.

I am no troll, and I hate to look like one when, to the appeal of Schrodinger, I am forced to remember that announcement this month about Claude or something acting in self-preservation when threatened during testing.

Do you have a link to that announcement?

I wouldn't like to debate a process handling the SIGTERM signal, or a container engine applying some restart policy.

Re: Humanity isn't ready for the coming intelligence explosion

#245
post #243

We don't need to live "side by side" with AI. AI is not alive, it's a technology we use. This is like talking about living side by side with your toaster.

I think most people would understand something like living "side by side with cars". Even if not alive, you can live very close and shape your life by and with technology. I don't take any issue with the OP's formulation.

And Americans' lives have been heavily shaped by cars and the car industry in a negative way. So, a pretty apt comparison.

Re: Humanity isn't ready for the coming intelligence explosion

#246
post #114
post #68

Reminds me of all of those ethics debates back when fully self driving cars were basically already here ca 10 years ago. The ones talking about how hard it would be to chose which person to run over. Additionally, I find it hard to believe that this would be a case of the future just not being distributed evenly. But sure. The AI labs relying on hype stating 15-50% risk of building a magic entity is certainly a relia…

I have had my license for over 20 years and to this day have not encountered a single event where I had to decide who to kill

Neither have I. The number of people who have encountered such events is probably close to negligible.

However I bet there is a good chance that will not be the case for self-driving cars. The reason is simple: human brains are slow.

When caught by surprise by an unexpected sudden obstacle a human driver takes about a second to realize something must be done and decide on braking or swerving in the fastest path, where the brain bypasses conscious decision making and uses the lower level fight or flight system. (Then another maybe 0.2 second to actually start swerving or 0.4 seconds to actually start pressing the brake--brake takes longer because of the need to move the foot to the brake pedal whereas the hands should already be on the steering wheel).

If they try to actually evaluate whether swerving or braking is the better choice before acting add another 0.3 to 0.5 seconds. That brings it up to around 1.6 to 2 seconds before they start actually acting, which most of the time is going to too late to actually affect things. Hence essentially no trolly problems for human drivers.

Compare to a self-driving car. The can detect the obstacle and predict the collision, and do a cycle of path planning before the retina has finished registering the obstacle and started transmitting that information to the brain.

By the time a human reaches the point where the brain knows something bad is happening and is starting to deal with it the self-driving car has had time to calculate numerous alternative trajectories to try to get around the object or stop before reaching it, taking into account predictions where all surrounding objects will move during the next few seconds, and calculate the damages for the trajectories that cannot avoid hitting something.

Re: Humanity isn't ready for the coming intelligence explosion

#247

Earlier quoted context omitted.

People seem to oddly not care what Hinton thinks anymore after he hasn't aligned with the groupthink naysaying of AGI risks: 'Hinton said there was a “10% to 20%” chance that AI would lead to human extinction within the next three decades.' “Because the situation we’re in now is that most of the experts in the field think that sometime, within probably the next 20 years, we’re going to develop AIs that are smarter th…

Hinton also said we should stop training radiologists 10 years ago because it would be solved by AI.

That's not accurate, see https://www.lesswrong.com/posts/xE2HzcHWPFS9EsJqN/radiology-...

Re: Humanity isn't ready for the coming intelligence explosion

#248
post #198

Another CEO with AI psychosis [1]. LLMs are not true AI, they lack common sense (or whatever it's properly called). LLM-based systems still need somebody with deep domain knowledge at the wheel to keep them from doing stupid things. It's like an alien-made bicycle that gets you to the speed of sound if you are an Olympic cyclist. Still, LLMs are extremely powerful pseudo-AI [2] and will bring a pseudo-singularity. Bu…

> Another CEO with AI psychosis [1]. LLMs are not true AI, they lack common sense (or whatever it's properly called). If you read the article, you'll find that the it indeed relies on this claim: > Within a couple of years, possibly much sooner, AI may achieve so-called closed-loop recursive self-improvement (RSI): the capacity to rewrite its own code to become more capable, without human intervention. Should that ha…

>the result could be an intelligence explosion of a kind for which there is no precedent and no map.

Or it will simply reach the end of the softmax faster.

Re: Humanity isn't ready for the coming intelligence explosion

#249

Humanity doesn't fully leverage the intelligence it already has, so I think people are overestimating the disruption ahead.

Depends what country you are in. The US government seems to be averse to intelligence, whereas China fosters it. Kinda strange that the US is falling behind, I never imagined the day.

Don't fall for 5D PR chess stunts.

That which must be embraced in the future per certain agenda, must be made looking like it's persecuted and oppressed in the present, per any textbook on narratives.

Re: Humanity isn't ready for the coming intelligence explosion

#250
post #166

Why should we assume AI can rapidly turn into super intelligence when physical and critical resources like energy and materials remain under human control?

The first time an AI goes rogue, figures out subterfuge, blackmails a human to do its real-world bidding, will be a defining moment.
Post reply on HN