Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

Post reply on HN
Print view View original

381–390 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#381

Post by randusername »

Nevermark wrote: post #99
cortesoft wrote: post #27

I couldn't read the whole article, but just from the part I could read: > Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest ai laboratories. I don't know, they also have an incentive to make their technology seem transformative and powerful, and saying that your technology has the power to cause a m…

> an incentive to make their technology seem transformative and powerful I have no frame of reference to process this. Humans species perhaps 300,000 years, we are essentially the same. Transistor, 79 years, explosive growth in numbers and power. Integrated circuit, 68 years, explosive growth in numbers and power. "Attention is all you need", 9 years. ChatGPT, 4 years, explosive growth in instances and power. Humans…

You're applying broad, long-term, and high-level reasoning to motivations that are more likely short-term and driven by simplistic incentives.

Begging for regulation (and input into precisely what that regulation is) relieves competition, consolidating dependency among the established players, and spreads a very effective fear narrative that what these companies are doing will change everything and therefore they are worthy of mind-boggling capex and valuations.

We have to quit seeing humanist visionaries where the evidence shows businessmen with access to the best strategy and PR counsel around.

Re: Humanity isn't ready for the coming intelligence explosion

#382

Post by pheaded_while9 »

MSFT_Edging wrote: post #320
calf wrote: post #215

Earlier quoted context omitted.

I am no troll, and I hate to look like one when, to the appeal of Schrodinger, I am forced to remember that announcement this month about Claude or something acting in self-preservation when threatened during testing.

> Claude or something acting in self-preservation That was an Ad. An Ad in the same style of many before it. Not through an ad agency, but the AI companies hyping their product by making their product seem scary and dangerous.

Does the press being an ad constitute the same thing as the information being false? If I advertize that my gun is dangerous, does the medium impact the truth value?

Re: Humanity isn't ready for the coming intelligence explosion

#383

Post by pheaded_while9 »

wrasee wrote: post #286
calf wrote: post #215

Earlier quoted context omitted.

I am no troll, and I hate to look like one when, to the appeal of Schrodinger, I am forced to remember that announcement this month about Claude or something acting in self-preservation when threatened during testing.

But, how is that different to any of its other output? I am sure there are many references to concepts of self-preservation in its training set that models will mimic just like all other concepts in its training set.

If we are still following Schroedinger, this imbues humanity with the ability to train non-life into life, not that thing being not alive. Not that I do or am, just going by GP's criteria.

Re: Humanity isn't ready for the coming intelligence explosion

#384

Post by root_axis »

ceejayoz wrote: post #379
root_axis wrote: post #375

Earlier quoted context omitted.

This just underscores the point. Despite the advancement of "AI medical devices", the labor market numbers for radiology show Hinton was spectacularly incorrect. If we had followed his advice we would be experiencing a major crisis in medicine today.

But is it a "he was wrong" scenario, or is it a "he wasn't quite right on the timing" bit, ala https://en.wikipedia.org/wiki/Michael_Burry ?

He is wrong by every conceivable measure. Not only was his timing wrong, where he specifically noted 10 years as the absolute maximum worst case scenario, but the trend has been accelerating in completely the opposite direction.

To his credit, he actually made a falsifiable prediction, unlike most folks that rely on infinite timelines so they never have to be accountable for their prognostications.

Re: Humanity isn't ready for the coming intelligence explosion

#385

Post by pheaded_while9 »

anavat wrote: post #232
falcor84 wrote: post #218

Earlier quoted context omitted.

Being a process doesn't preclude something from being an entity. In particular, all of us living organisms have our constituent atoms constantly replaced, and we don't typically consider that a risk to our sense of self.

It comes down to "what stays the same when all its atoms are replaced" and "how can we tell it is the same entity and not just an identical but different one". I think the answers are "shape" AND "continuity". E.g. all "atoms" of a glider in Conway's Life get replaced every couple turns but an observer can tell it is still a glider because it keeps the shape, and it the same glider because it continues it's previous…

Then this is a "Death of the gaps" which implies algorithm, eg Conway's, may solve AI non-life. So, not a fundamental denial but an architectual one.

Re: Humanity isn't ready for the coming intelligence explosion

#386

Post by pheaded_while9 »

bcjdjsndon wrote: post #331
Valakas_ wrote: post #303

Earlier quoted context omitted.

Imagine a smart monkey talking to other smart monkeys trying to figure out how humans would ever be able to escape the (simple) prison they built for them. The humans would find a way to escape. No matter how ingenious the monkeys think they are. You're asking a question, to which you (and most humans) don't know the answer, and you're (wrongly) assuming a being much more intelligent than you also wouldn't. And by "m…

> You need a similar degree of open mindedness and imagination to be able to discern what such an intelligence being would be capable of. Or a naive grasp of reality... You can posit all the super intelligence you like, actually creating it is another story entirely. For instance, it's clear the transformer architecture is brute force and memory hungry... Why hasn't ai improved on this? After ten years still no luck?…

You're gesturing at some fundamental limit that, if it exists, there is no evidence of peaking yet. It is not rigorous to say AI has not contributed to architecture improvement.

Re: Humanity isn't ready for the coming intelligence explosion

#387

Post by rpdillon »

bluefirebrand wrote: post #368
embedding-shape wrote: post #311

Earlier quoted context omitted.

My personal "wow" moment was less about it being able to do the same that I could do with it, just faster, but instead that together we could could build things that I wasn't able to build before (or unwilling to spend the time to learn maybe). So a bunch of stuff are suddenly "unlocked", as long as you know how to verify it properly.

If you weren't able to build it before then you aren't able to properly verify it Verifying is harder than building!

Verifying is harder than building for some problems, but not all. I think part of the trick of using AI is applying it where verification is easy.

Re: Humanity isn't ready for the coming intelligence explosion

#388

Post by perarneng »

The economy isn't ready. Basically, all companies right now are sprinting toward increasing profit margins by trying to do more with fewer people. The problem is that, on this scale, they will lose income because unemployed people make terrible consumers, they will only buy what is strictly necessary. Consequently, society is in a race to the bottom, which could lead to extreme suffering and potential revolutions in many countries. Democracies will likely have it easier in general because everyone shares the responsibility, in a sense, through voting. However, the USA, which is extremely polarized—will have a tough time because animosity between the political sides is already at an all-time high, even without an economic collapse. China, on the other hand, will also be in a difficult situation. If the West collapses financially, consumption of Chinese-produced goods will drop drastically. This will create massive unemployment in China, causing public rage to increase. In a non-democratic country, it is much easier for the population to point the blame in a single direction.

Re: Humanity isn't ready for the coming intelligence explosion

#389

Post by assimpleaspossi »

>>Giving birth to a superintelligence would be the most consequential moment in human history—and it is likely to be irreversible, as any “off” switch humanity might design will probably fail.

Nothing will be able to stop you from pulling the plug.

Re: Humanity isn't ready for the coming intelligence explosion

#390

Post by WarmWash »

embedding-shape wrote: post #311
grumpymuppet wrote: post #292

Earlier quoted context omitted.

I had my "wow" moment recently and saw an AI agent do something do over a weekend something that I spent years developing a relatively niche skill might have needed a month to complete. I think the message the team lead takes away right now is "oh, this can be done a lot faster, let's do more!" This is exciting for a bit, but I think it comes at the cost of learning -- and being able to communicate --architectural le…

My personal "wow" moment was less about it being able to do the same that I could do with it, just faster, but instead that together we could could build things that I wasn't able to build before (or unwilling to spend the time to learn maybe). So a bunch of stuff are suddenly "unlocked", as long as you know how to verify it properly.

There was a period where computer+grandmaster was the ultimate winning force in chess. It lasted a few years before it was just computer.
Post reply on HN