Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

Post reply on HN
Print view View original

61–70 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#61

Post by jdw64 »

edg5000 wrote: post #60
jdw64 wrote: post #39

I want to name the AI era as the corruption of the elite. The rhetorical structure of talking about uncertain risks and then trying to concentrate the authority to manage those risks in their own hands sounds utterly ridiculous to ordinary people like me. It's just a simple hypothesis that AI will become uncontrollable to humans once it becomes superintelligent. Isn't the fact that a reinforcement learning agent impr…

The AI labs will govern, effectively if not officially. Government is anchored in power, which the labs have. We used to have the church and state rule here in Europe. In the future, A government not closely ruling with their AI labs won't have sufficient power to exist. You're unhappy about the labs; I think you're just not ready to accept their rule; you consider them nothing but scrappy startups, which they are. B…

You could be right. In that case, I should try to get into an AI research institute here in my country, though it won't be easy.

Re: Humanity isn't ready for the coming intelligence explosion

#62

Post by g-b-r »

g-b-r wrote: post #32

Is it really likely that a "recursive self-improvement" capability would lead to a great acceleration of AIs capabilities? Isn't the preponderant bottleneck in improving the models the need to train them at scale to verify the hypotheses, and the time and cost that it takes? Or does someone think that they could get magically able to predict big improvements without training?

Weird, there was a good comment about this but it vanished

Re: Humanity isn't ready for the coming intelligence explosion

#63

Post by SpicyLemonZest »

al_borland wrote: post #52
SpicyLemonZest wrote: post #12

Earlier quoted context omitted.

Do you have an example of a prediction they’ve made in the last 3 years that was drastically wrong? This is not my impression.

Do you think all white collar work will be replaced by AI in 18 months? https://fortune.com/article/why-microsoft-ai-chief-mustafa-s... Altman and Amodei recently hard to start walking back their earlier predictions. https://fortune.com/2026/05/26/sam-altman-dario-amodei-walki...

Again, do you have an example of a prediction that was drastically wrong? I don't think it's productive to speculate about which things might or might not come true in the next 18 months. There were people last year making fun of Dario for predicting that writing code would be automated in the next year.

Re: Humanity isn't ready for the coming intelligence explosion

#64

Post by raincole »

al_borland wrote: post #2

> Experts in artificial intelligence estimate... The predictions of these "experts" have been drastically wrong for the last 3 years. At what point does someone lose their "expert" title?

> The predictions of these "experts"... last 3 years.

Which ones? Please be specific.

Re: Humanity isn't ready for the coming intelligence explosion

#65

Post by DrBazza »

Waterluvian wrote: post #28
al_borland wrote: post #2

> Experts in artificial intelligence estimate... The predictions of these "experts" have been drastically wrong for the last 3 years. At what point does someone lose their "expert" title?

Those are the experts who were wrong. These are the experts who are right. We sacked the experts who are wrong and replaced them with the experts who are right. “How do we know they’re right?” I hear you ask. We know because we who are wrong and these ones aren’t them. So they can’t make wrong predictions because those who make wrong predictions have been sacked.

Channeling your inner Douglas Adams.

Re: Humanity isn't ready for the coming intelligence explosion

#66

Post by cat_plus_plus »

Models consume a lot of memory and power to slowly generate autocompletions of existing content one token at a time. Letting this text control important things in real world is a human decision and control of nuclear weapons or penetration testing agents should be well regulated. But these should already be well regulated without AI. So how about we focus on drawing down nuclear arsenals, which is a present danger one idiot (or faulty AI) away from world shattering consequences rather than fearmongering about unknown future. Before effective AI regulation can be drafted, we need to anyway know more about AI specific dangers rather than humans committing already very common crimes like hacking for ransom with new tools?

Re: Humanity isn't ready for the coming intelligence explosion

#67

Post by fnordpiglet »

al_borland wrote: post #41
fnordpiglet wrote: post #21

Earlier quoted context omitted.

It’s hard to take seriously a statement like this that’s grounded in “the last three years” when trends have tended to play out over decades. The fact the goal posts have moved from thousands to hundreds to decades to single digit years for massive transformation should give you a moment of pause in your bag saying. I would however agree there are no experts. Not because of some prediction made on a short time scale…

I said 3 years, because that’s how long ago ChatGPT was released to the public. Since then we’ve been told that all of us should be out of a job in 6 months, every 6 months, for those 3 years. I agree that change happens on a longer timeline. This is why I’m so tired of statements like this… “Within a couple of years, possibly much sooner…” These “experts” are pulling timelines out of the sky, and these predictions a…

Ultimately we agree but for slightly different reasons. My assertion is there are no experts because there can’t be as not enough time passed for expertise to form. Further the rate of change is such that by time someone becomes an expert in some aspect, it’s been made irrelevant. Hence expertise in this space is unattainable at the moment and all expert advice is fraudulent.

Re: Humanity isn't ready for the coming intelligence explosion

#68

Post by hypfer »

Reminds me of all of those ethics debates back when fully self driving cars were basically already here ca 10 years ago.

The ones talking about how hard it would be to chose which person to run over.

Additionally, I find it hard to believe that this would be a case of the future just not being distributed evenly.

But sure. The AI labs relying on hype stating 15-50% risk of building a magic entity is certainly a reliable number.

Re: Humanity isn't ready for the coming intelligence explosion

#69

Post by SJC_Hacker »

fnordpiglet wrote: post #59
bdamm wrote: post #34

Earlier quoted context omitted.

Autonomous weapons are actually a really interesting branch of this and deserves a little more. Yes, autonomous weapons were explored and were found to be poor performers compared to actual pilots. The breakthrough is in terminal guidance and dozens of other little techniques to get quality human control extended into the far reach of the battlefield. And of course, AI assistance in logistics and analysis. But actual…

You don’t need to load the compute onto the platform for autonomy, planning can be done remotely just like with human guidance. The latency is the same.

But if comms get jammed, how will the drone decide what to target ? For that you need inference, which means sizable compute.

Re: Humanity isn't ready for the coming intelligence explosion

#70

Post by thin_carapace »

dataviz1000 wrote: post #49
g-b-r wrote: post #32

Is it really likely that a "recursive self-improvement" capability would lead to a great acceleration of AIs capabilities? Isn't the preponderant bottleneck in improving the models the need to train them at scale to verify the hypotheses, and the time and cost that it takes? Or does someone think that they could get magically able to predict big improvements without training?

I built a self-learning recursive agent that finds academic research about using options data to trade, re-creates the research, and then probes and tests for gaps and potential strategies testing against over one year of out-of-sample trading data with one of several strategies that beat SPY by 10x. [0] One rule is that if a position is opened using the historical data, it can't close the position until the next mor…

the amount of strategies that perform good in back testing dwarfs the amount of strategies that perform good in reality
Post reply on HN