Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

Post reply on HN
Print view View original

121–130 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#121

Post by toasty228 »

nopinsight wrote: post #98
hypfer wrote: post #68

Reminds me of all of those ethics debates back when fully self driving cars were basically already here ca 10 years ago. The ones talking about how hard it would be to chose which person to run over. Additionally, I find it hard to believe that this would be a case of the future just not being distributed evenly. But sure. The AI labs relying on hype stating 15-50% risk of building a magic entity is certainly a relia…

Things in the real world often take longer than expected. Still, in cities where Waymo operates, many people routinely ride autonomous vehicles and prefer them. For software, however, a rapid turn is often a possibility. See: AI for coding over the last 3-4 years. AI autocomplete --> AI coding assistants --> vibe coding --> agent orchestration Coders can now accomplish work that used to take a week or longer in a cou…

Does a good engineer, with the right skills and right tools make up for the thousands of kids basically giving up on education and learning?

Re: Humanity isn't ready for the coming intelligence explosion

#122

Post by nopinsight »

otabdeveloper4 wrote: post #118
nopinsight wrote: post #115

"Ten years from now, I think we will realize that we were standing in the foothills of the singularity now... I believe that we're only a few years away from [AGI], maybe 2030 plus or minus a year... I think [AGI] will be an enormous transformative technology, it's going to effectively be a new human era... We can feel this year, I would say, even though I've been working towards this for 30 years, I think this year…

Someone should document these cringe comments for posterity. As a warning to future generations. It's too bad the previous singularity cult manias 10 and 15 years ago weren't documented and are lost in time.

Ten years ago, most of the skeptics in these threads would have said the success of AlphaFold 3 was impossible.

Re: Humanity isn't ready for the coming intelligence explosion

#123

Post by zzgo »

consumer451 wrote: post #88
holmesworcester wrote: post #53

Earlier quoted context omitted.

If AI is now ascending an economic learning curve from: 1. Extremely useful (Claude Code & Waymo now) 2. Doing ~everything we do (AGI & Optimus in a few years? 10?) 3. RSI (?) 4. Being smarter than any living person at every intellectual task (?) 5. Being smarter than the best-organized aggregate of all humans (10-100 years?) ...And all of the scientific and resource-allocation institutions that brought us the comput…

Here is an issue I think about often, but I am not quite sure how to put it into words. We have many extremely smart people in various fields. Executives, politicians, and society generally ignore them and do whatever they want. I don't believe that lack of access to intelligence is our problem. How is "free" intelligence going to improve this? I don't just mean climate, but business planning, health, risk assessment…

There are lots of incentives for politicians and executives (and anyone else holding the levers of power) to ignore information, intelligence, and advice. I think you're right to be skeptical that "free" intelligence is going to improve anything without first addressing the incentives of the people holding power.

Re: Humanity isn't ready for the coming intelligence explosion

#124

Post by toasty228 »

nopinsight wrote: post #108
RandomLensman wrote: post #103

Earlier quoted context omitted.

But there is also still a huge part that doesn't run on software with so far little change.

With incessant advances in robotics, how long would that continue to be the case? Should we start preparing for something that could be world-changing in the next 10-20 years?

When I graduated a bit over 10 years ago some people were saying we'd have a permanent mars bases by now. When my parents graduated they were told they'd retire at 45 and have 3 days work week due to "automation", they're still working at 60+ today, more than back then actually

People should open history books and gain some political/historical culture, this thread is 90% wishful thinking and "if the lines continues straight from now we'll basically be gods in 5 years"

Re: Humanity isn't ready for the coming intelligence explosion

#125

Post by marcyb5st »

thunderbong wrote: post #20

Honestly, this article has too many words which don't mean anything. 'AI Experts', 'superintelligence', and hand-waving doom scenarios. As an example- > Within a couple of years, possibly much sooner, AI may achieve so-called closed-loop recursive self-improvement (RSI): the capacity to rewrite its own code to become more capable, without human intervention. Should that happen, the result could be an intelligence exp…

Yeah, big lol on the Recursive Self-Improvement.

I mean, firstly you get to have an "Agent" actually capable of really long horizon tasks without getting stuck in tools loops and having its context rot. Secondly, each trial (ie a model fully trained to convergence) costs millions and takes O(weeks). You can probably run 1 or 2 of those experiments in parallel even at big AI labs as the hardware is scarce and they are costly as mentioned before. Assuming this agent needs something like a hundred tries to just show some improvement, we are looking at years.

And you can't early stop training for candidates that are not promising due to "emerging capabilities". At some point you might get a big drop in loss even if the model has been plateuing for a while. And you can't really scale down models for running trials quickly either also due to these emerging capabilities. In fact you might create a model that is great at converging with in small trials (fewer params, fewer tokens), but that is uncapable of developing those unexpected traits. And this will likely happen as you created a sort of evolutionary pressure in this direction: if you are good at learning in the first few epochs you "survive" and get to pass down your traits to future trials.

All of this to say that recursive-self-improvement is waaaaaay out of our grasp as things stand right now. We need another one or two breakthroughs to get there (IMHO).

Re: Humanity isn't ready for the coming intelligence explosion

#126

Post by ipdashc »

esafak wrote: post #17

I think it is so reckless and unethical that AI companies are not spending good money on answering the political, social, economical, moral questions that they are raising.

Right, makes sense, if they just spent a bunch of money they could answer all of those "political, social, economical, moral questions". All of these unprecedented sociological and labor market changes flying towards us are trivially solvable, we just need to... what, hire a few dozen economists? Fund a team of philosophers to think about it really hard for a few months?

Re: Humanity isn't ready for the coming intelligence explosion

#127

Post by threethirtytwo »

thunderbong wrote: post #20

Honestly, this article has too many words which don't mean anything. 'AI Experts', 'superintelligence', and hand-waving doom scenarios. As an example- > Within a couple of years, possibly much sooner, AI may achieve so-called closed-loop recursive self-improvement (RSI): the capacity to rewrite its own code to become more capable, without human intervention. Should that happen, the result could be an intelligence exp…

>Honestly, this is getting extremely tiring. Every new invention that has happened has affected the world in both good and bad ways.

Common human trope is to rely on analogies and past patterns to predict the future. It blinds you reality.

This isn't some new "invention". What's happening is fundamentally more profound. the stakes are much much much more higher and the consequences much more grave. Thus a historical analogy can't save you when the situation is categorically different.

I agree it's getting "tiring" but this is another common human trope. You see the same thing too many times and you lose the sense of danger that comes with it. It's like sky diving all the time. You eventually lose the fear via repetition, but fundamentally speaking, you are still parachuting out of a fucking plane.

Re: Humanity isn't ready for the coming intelligence explosion

#128

Post by Animats »

nopinsight wrote: post #98
hypfer wrote: post #68

Reminds me of all of those ethics debates back when fully self driving cars were basically already here ca 10 years ago. The ones talking about how hard it would be to chose which person to run over. Additionally, I find it hard to believe that this would be a case of the future just not being distributed evenly. But sure. The AI labs relying on hype stating 15-50% risk of building a magic entity is certainly a relia…

Things in the real world often take longer than expected. Still, in cities where Waymo operates, many people routinely ride autonomous vehicles and prefer them. For software, however, a rapid turn is often a possibility. See: AI for coding over the last 3-4 years. AI autocomplete --> AI coding assistants --> vibe coding --> agent orchestration Coders can now accomplish work that used to take a week or longer in a cou…

> Things in the real world often take longer than expected. Still, in cities where Waymo operates, many people routinely ride autonomous vehicles and prefer them.

Yes. The bottleneck has been getting the cars manufactured. It takes a certain amount of time to get a factory going, and then the product starts pouring out. Waymo used up the supply of Jaguars, about 3,500 of them. The Ioniq 5 plant is starting up.[1] Waymo has ordered 50,000 cars, all ready to have the self-driving electronics plugged in. Waymo gets out of the sheet metal bending business with this.

[1] https://electrek.co/2026/02/11/hyundai-supply-waymo-50000-io...

Re: Humanity isn't ready for the coming intelligence explosion

#129

Post by pydry »

thunderbong wrote: post #20

Honestly, this article has too many words which don't mean anything. 'AI Experts', 'superintelligence', and hand-waving doom scenarios. As an example- > Within a couple of years, possibly much sooner, AI may achieve so-called closed-loop recursive self-improvement (RSI): the capacity to rewrite its own code to become more capable, without human intervention. Should that happen, the result could be an intelligence exp…

The Economist reflects the views and biases of economic and political elites.

AI has turned into something of a religion for them because the substitution of technology for labor is a reflection of one of their deepest desires. Whatever the reality of the situation, they want to believe that AGI employees are imminent.

Re: Humanity isn't ready for the coming intelligence explosion

#130

Post by [deleted] »

SpicyLemonZest wrote: post #101
bigstrat2003 wrote: post #85

Earlier quoted context omitted.

> Yes, I think AI has eliminated writing code. Then you are dead wrong. Anyone who gives a shit about doing a good job is still writing code.

No, that’s not accurate. I’m not sure what to tell you. It’s like hearing that no serious programmer uses Python, it’s so far from my experience and that of everyone I know in the field that I don’t know how to engage.

[deleted]
Post reply on HN