Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

Post reply on HN
Print view View original

81–90 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#81

Post by SpicyLemonZest »

al_borland wrote: post #73
SpicyLemonZest wrote: post #63

Earlier quoted context omitted.

Again, do you have an example of a prediction that was drastically wrong? I don't think it's productive to speculate about which things might or might not come true in the next 18 months. There were people last year making fun of Dario for predicting that writing code would be automated in the next year.

From the 2nd link I posted: > Anthropic CEO Dario Amodei, who once claimed AI could eliminate 50% of white-collar jobs, now says automation may actually expand the work people do. Is pivoting from 50% elimination to actually expanding work not a drastic enough? Do you think AI has eliminated writing code? I still write code every day. The AI is more a thing I ask questions and it gives me right answers about 40% of t…

Yes, I think AI has eliminated writing code. Everyone I personally know working in software stopped writing code some time between December and March. (It's true that AI continues to routinely make errors; if you've heard the term "agentic workflows", that's the standard strategy for mitigating the error rate by allowing the AI to check its own work.) That's why I think Amodei's January 2026 prediction that AI could eliminate 50% of entry level white collar jobs in 1-5 years remains plausible.

Your second article says something different, but this is because it's full of misquotes. The link supporting "50% of jobs" specifically says entry level, and the link supporting "reframed automation... not as a destroyer of jobs" has Amodei saying not that jobs won't be destroyed but that new jobs may be created to replace them. If AI moves sufficiently slowly to let that happen, which he explicitly cautions it may not.

Re: Humanity isn't ready for the coming intelligence explosion

#82

Post by cat_plus_plus »

g-b-r wrote: post #32

Is it really likely that a "recursive self-improvement" capability would lead to a great acceleration of AIs capabilities? Isn't the preponderant bottleneck in improving the models the need to train them at scale to verify the hypotheses, and the time and cost that it takes? Or does someone think that they could get magically able to predict big improvements without training?

The preponderant bottleneck is inventing new architectures to make AI actually good at human and superhuman tasks. For example, AI agent harnesses add tool calls and long term state management, allowing AI to autonomously complete complex tasks. Once these are in place, finetuning models with examples of good tool calls helps, but somebody first needed to invent the fundamental capability. Now try to implement humanlike long term memory for AI to be your coworker or life assistant working on tasks that last month. Even if necessary low level technologies are already there, structuring them to be practically useful is non trivial.

Re: Humanity isn't ready for the coming intelligence explosion

#83

Post by gensym »

frognumber wrote: post #43
Retric wrote: post #31

Earlier quoted context omitted.

Do you have specific and objective examples of things people got right? From my perspective there’s very slow but very real progress happening in the AI space. I see people making wild predictions in both directions, but in terms of actual unsupervised utility there’s definitely progress abet wildly slower than most hype.

This was their prediction for 2026: "The bet of using AI to speed up AI research is starting to pay off. OpenBrain continues to deploy the iteratively improving Agent-1 internally for AI R&D. Overall, they are making algorithmic progress 50% faster than they would without AI assistants—and more importantly, faster than their competitors. The AI R&D progress multiplier: what do we mean by 50% faster algorithmic progre…

So you think Anthropic is using internal AI assistants to pull away from competitors and the leapfrogging we've seen over the last several years is now done?

That seems to me to be the most concrete and least obvious prediction in the quoted text.

I don't think that's happening. If that were generally accepted as true I would expect OpenAI to be unable to successfully IPO.

Re: Humanity isn't ready for the coming intelligence explosion

#84

Post by bigstrat2003 »

somesortofthing wrote: post #71
al_borland wrote: post #2

> Experts in artificial intelligence estimate... The predictions of these "experts" have been drastically wrong for the last 3 years. At what point does someone lose their "expert" title?

Have they? I don't like it either but the headline bull predictions(reliable agents and most code written by AI by mid-2026, dramatic progress against jailbreaks, prompt injection, hallucination, etc., major improvements despite pretraining data exhaustion, continued exponential growth on the METR trendline) did come true, often ahead of even aggressive schedules. What major wrong predictions did you have in mind?

Literally none of the predictions you listed in your post came true. We don't have reliable agents, AI isn't writing most code, hallucinations still happen all the time, improvement has been basically non-existent. Despite the constant claims of these experts and AI boosters, AI is still not a tool one can use to get meaningful work done.

Re: Humanity isn't ready for the coming intelligence explosion

#85

Post by bigstrat2003 »

SpicyLemonZest wrote: post #81
al_borland wrote: post #73

Earlier quoted context omitted.

From the 2nd link I posted: > Anthropic CEO Dario Amodei, who once claimed AI could eliminate 50% of white-collar jobs, now says automation may actually expand the work people do. Is pivoting from 50% elimination to actually expanding work not a drastic enough? Do you think AI has eliminated writing code? I still write code every day. The AI is more a thing I ask questions and it gives me right answers about 40% of t…

Yes, I think AI has eliminated writing code. Everyone I personally know working in software stopped writing code some time between December and March. (It's true that AI continues to routinely make errors; if you've heard the term "agentic workflows", that's the standard strategy for mitigating the error rate by allowing the AI to check its own work.) That's why I think Amodei's January 2026 prediction that AI could…

> Yes, I think AI has eliminated writing code.

Then you are dead wrong. Anyone who gives a shit about doing a good job is still writing code.

Re: Humanity isn't ready for the coming intelligence explosion

#86

Post by knollimar »

gensym wrote: post #83
frognumber wrote: post #43

Earlier quoted context omitted.

This was their prediction for 2026: "The bet of using AI to speed up AI research is starting to pay off. OpenBrain continues to deploy the iteratively improving Agent-1 internally for AI R&D. Overall, they are making algorithmic progress 50% faster than they would without AI assistants—and more importantly, faster than their competitors. The AI R&D progress multiplier: what do we mean by 50% faster algorithmic progre…

So you think Anthropic is using internal AI assistants to pull away from competitors and the leapfrogging we've seen over the last several years is now done? That seems to me to be the most concrete and least obvious prediction in the quoted text. I don't think that's happening. If that were generally accepted as true I would expect OpenAI to be unable to successfully IPO.

We wait for gemini 3.5 pro and cope?

Re: Humanity isn't ready for the coming intelligence explosion

#87

Post by JuniperMesos »

bandrami wrote: post #76
hypfer wrote: post #68

Reminds me of all of those ethics debates back when fully self driving cars were basically already here ca 10 years ago. The ones talking about how hard it would be to chose which person to run over. Additionally, I find it hard to believe that this would be a case of the future just not being distributed evenly. But sure. The AI labs relying on hype stating 15-50% risk of building a magic entity is certainly a relia…

I mean, I'm very much tech-forward and have spent the past decade in major world cities and I've seen a driverless car once in my life. Sometimes things that seem inevitable really aren't.

I see driverless cars all the time in San Francisco and I'm eagerly awaiting the regulatory decisions that will allow them to be widely available in the bay area at large.

Re: Humanity isn't ready for the coming intelligence explosion

#88

Post by consumer451 »

holmesworcester wrote: post #53
cortesoft wrote: post #27

I couldn't read the whole article, but just from the part I could read: > Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest ai laboratories. I don't know, they also have an incentive to make their technology seem transformative and powerful, and saying that your technology has the power to cause a m…

If AI is now ascending an economic learning curve from: 1. Extremely useful (Claude Code & Waymo now) 2. Doing ~everything we do (AGI & Optimus in a few years? 10?) 3. RSI (?) 4. Being smarter than any living person at every intellectual task (?) 5. Being smarter than the best-organized aggregate of all humans (10-100 years?) ...And all of the scientific and resource-allocation institutions that brought us the comput…

Here is an issue I think about often, but I am not quite sure how to put it into words.

We have many extremely smart people in various fields. Executives, politicians, and society generally ignore them and do whatever they want. I don't believe that lack of access to intelligence is our problem. How is "free" intelligence going to improve this?

I don't just mean climate, but business planning, health, risk assessment, everything.

Re: Humanity isn't ready for the coming intelligence explosion

#89

Post by Haunt1000 »

bloaf wrote: post #51
g-b-r wrote: post #32

Is it really likely that a "recursive self-improvement" capability would lead to a great acceleration of AIs capabilities? Isn't the preponderant bottleneck in improving the models the need to train them at scale to verify the hypotheses, and the time and cost that it takes? Or does someone think that they could get magically able to predict big improvements without training?

I don't think anyone really knows. I consider these scenarios: 1) We stumble onto an algorithmic improvement in intelligence. This isn't just "what humans do but faster", its "better than what humans do". I've got no idea what that might mean (it could be fundamentally different heuristics, it could be that we've got some intellectual blind spot that they cast off). It doesn't matter, the instant this happens AI is s…

> 2) AI gets good enough at physics and engineering that they can really quickly use up all "the room at the bottom" as Feyman put it. They design and build a factory that produces a mystery metal amalgam that computes at some small percentage of the minimum predicted by the Landauer principle, within a few percent of Bremermann's limit. It's not "smarter" its just suddenly tens-of-orders of magnitude faster. But those orders of magnitude matter: there's only 8 billion of us, and there's plenty more than a factor of 10 billion "at the bottom".

Actually your comment made me sign up for an account just so I could say this is the real reason why AI won't take over in the way you say. This kind of stuff requires an enormous amount of experimentation. You can ask any theoretical physicist or chemist versus an experimental one and the conclusion is the experimental people actually find out what happens and how the great puzzle of the universe is solved. And humans could just refuse to collaborate. But that's the big weakness with AI I think it has no real world knowledge or empirical experience.

Re: Humanity isn't ready for the coming intelligence explosion

#90

Post by fnordpiglet »

tancop wrote: post #80
SJC_Hacker wrote: post #69

Earlier quoted context omitted.

But if comms get jammed, how will the drone decide what to target ? For that you need inference, which means sizable compute.

you can put the compute on a big expensive reusable control drone and send commands down with lasers. thats way harder to jam than any form of radio. if thats not enough try fiber optic like they do in ukraine.

There are also now drones with long spooling wires. Regardless, autonomous or not, connectivity to home is important.

I would note that a lot of drones are quite large too - small airplanes. They can carry more than enough hardware for autonomy. You can also have motherships doing planning and smaller kamikaze drones for combat.

These aren’t hard problems to solve, the harder problem is the reluctance to give over to automation. But that’s going to happen faster and faster.

Post reply on HN