Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

Post reply on HN
Print view View original

361–370 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#361

Post by pfdietz »

fnordpiglet wrote: post #90
tancop wrote: post #80

Earlier quoted context omitted.

you can put the compute on a big expensive reusable control drone and send commands down with lasers. thats way harder to jam than any form of radio. if thats not enough try fiber optic like they do in ukraine.

There are also now drones with long spooling wires. Regardless, autonomous or not, connectivity to home is important. I would note that a lot of drones are quite large too - small airplanes. They can carry more than enough hardware for autonomy. You can also have motherships doing planning and smaller kamikaze drones for combat. These aren’t hard problems to solve, the harder problem is the reluctance to give over to…

> long spooling wires

Long spooling optical fibers.

Re: Humanity isn't ready for the coming intelligence explosion

#362

Post by timbert »

embedding-shape wrote: post #311
grumpymuppet wrote: post #292

Earlier quoted context omitted.

I had my "wow" moment recently and saw an AI agent do something do over a weekend something that I spent years developing a relatively niche skill might have needed a month to complete. I think the message the team lead takes away right now is "oh, this can be done a lot faster, let's do more!" This is exciting for a bit, but I think it comes at the cost of learning -- and being able to communicate --architectural le…

My personal "wow" moment was less about it being able to do the same that I could do with it, just faster, but instead that together we could could build things that I wasn't able to build before (or unwilling to spend the time to learn maybe). So a bunch of stuff are suddenly "unlocked", as long as you know how to verify it properly.

^ This!

I have been able to build applications that I couldn't dream of before, not because of a lack of capability on my part, but because of time: I don't have the time to learn all of the new APIs and frameworks to accomplish my goal. Now I can build far more complex useful applications, with functions and features that would take me months or more to learn how to code.

The downside is that I didn't learn how to use those APIs, but it can be argued that these skills are low value to begin with. APIs change, there are competing APIs that do the same thing, and you may only use that skill for one project and then move on. Gaining those skills can be a big time sink and may not result in long-term gain.

Think of all the time spent learning the ins and outs of come-and-gone APIs and frameworks over the decades (I chose not to provide examples to save the resulting side-arguments, but I know you can rattle some off in your head). Those APIs and frameworks were bridges to make developing easier for the common developer, but they change so quickly that I'd argue they waste our time, and lead to almost endless re-work as we re-architect our software for the framework de jour. And they grow in complexity over time, ultimately becoming the next bloated framework. It's an endless, wasteful cycle.

Maybe AI can help us get out of that.

Re: Humanity isn't ready for the coming intelligence explosion

#363

Post by squidbeak »

turing_complete wrote: post #136

We don't need to live "side by side" with AI. AI is not alive, it's a technology we use. This is like talking about living side by side with your toaster.

This is the denialist Hy-Brasil line of thought. But superintelligence won't be a tool or a toaster and it would be reckless in the extreme for policy makers to adopt your line of thought.

Re: Humanity isn't ready for the coming intelligence explosion

#364

Post by izzydata »

spicyusername wrote: post #348
grumpymuppet wrote: post #292

Earlier quoted context omitted.

I had my "wow" moment recently and saw an AI agent do something do over a weekend something that I spent years developing a relatively niche skill might have needed a month to complete. I think the message the team lead takes away right now is "oh, this can be done a lot faster, let's do more!" This is exciting for a bit, but I think it comes at the cost of learning -- and being able to communicate --architectural le…

Everyone is afraid about what will happen when people's skills atrophy, but the tools aren't going anywhere. If they truly can always recognize problems and communicate them in understandable ways, then people's skills won't atrophy. Everyone's skills will be enhanced. New developers will learn all of the things they need to learn through using the tool.

The tools can definitely go away. They could get exponentially more expensive to where it is no longer financially possible to keep it subsidized. The bubble could pop and most of the companies could go bankrupt. It's not exactly like a calculator because it requires such an astronomical amount of resources to keep running.

Re: Humanity isn't ready for the coming intelligence explosion

#365

Post by kobe_bryant »

grumpymuppet wrote: post #292
swalsh wrote: post #257

There's issues where bad people with bad intentions use AI to do bad things. But to an extent the guardrails already built-in are sufficient. The real danger in AI is how it impacts society, our economy, and our perception of self worth. I'm not afraid of rouge agents nearly as much as I'm afraid of us building a permanent underclass of people dependent on whatever scraps the people who devalued their labor decide th…

I had my "wow" moment recently and saw an AI agent do something do over a weekend something that I spent years developing a relatively niche skill might have needed a month to complete. I think the message the team lead takes away right now is "oh, this can be done a lot faster, let's do more!" This is exciting for a bit, but I think it comes at the cost of learning -- and being able to communicate --architectural le…

feel free to share even a single detail

Re: Humanity isn't ready for the coming intelligence explosion

#366

Post by ilamont »

The first priority should be an agreement between the two heavyweights of ai: America and China. Donald Trump and Xi Jinping should affirm the principle that humans must remain stewards of ai systems until adequate frameworks for reliability and security have been built.

This is naïve. The goals of both men have nothing to do with protecting humanity, but rather furthering their own personal agendas.

In Trump’s case, it’s all about amassing more wealth and power.

For Xi, it’s realizing an ethno-nationalist dream where China under the CCP is at the center of world power, the independent nation of Taiwan as well as disputed border areas that are currently controlled by India and Russia and the Philippines are annexed by China, and Xi’s eternal legacy is remembered as the savior of the Chinese people.

International cooperation and touchy-feely rhetoric about saving humanity from AI have no place in either man’s worldview.

Re: Humanity isn't ready for the coming intelligence explosion

#367

Post by schleck8 »

reducesuffering wrote: post #162
Schlagbohrer wrote: post #148

Why is someone from Planet Labs writing this? Why not, for example, Geoffrey Hinton, the scientist who won the nobel prize for LLM architecture? Or a sci-fi author?

People seem to oddly not care what Hinton thinks anymore after he hasn't aligned with the groupthink naysaying of AGI risks: 'Hinton said there was a “10% to 20%” chance that AI would lead to human extinction within the next three decades.' “Because the situation we’re in now is that most of the experts in the field think that sometime, within probably the next 20 years, we’re going to develop AIs that are smarter th…

Why do you cite him from 2024 when he spoke about it much more recently? This is a quickly moving field after all

> “I think we’re going to see AI get even better,” Hinton replied. “It’s already extremely good. We’re going to see it having the capabilities to replace many, many jobs. It’s already able to replace jobs in call centers, but it’s going to be able to replace many other jobs.”

https://fortune.com/2025/12/28/geoffrey-hinton-godfather-of-...

Re: Humanity isn't ready for the coming intelligence explosion

#368

Post by bluefirebrand »

embedding-shape wrote: post #311
grumpymuppet wrote: post #292

Earlier quoted context omitted.

I had my "wow" moment recently and saw an AI agent do something do over a weekend something that I spent years developing a relatively niche skill might have needed a month to complete. I think the message the team lead takes away right now is "oh, this can be done a lot faster, let's do more!" This is exciting for a bit, but I think it comes at the cost of learning -- and being able to communicate --architectural le…

My personal "wow" moment was less about it being able to do the same that I could do with it, just faster, but instead that together we could could build things that I wasn't able to build before (or unwilling to spend the time to learn maybe). So a bunch of stuff are suddenly "unlocked", as long as you know how to verify it properly.

If you weren't able to build it before then you aren't able to properly verify it

Verifying is harder than building!

Re: Humanity isn't ready for the coming intelligence explosion

#369

Post by root_axis »

falcor84 wrote: post #218
root_axis wrote: post #165

Earlier quoted context omitted.

It has no body, metabolism or reproductive capability. It's a process, not an entity.

Being a process doesn't preclude something from being an entity. In particular, all of us living organisms have our constituent atoms constantly replaced, and we don't typically consider that a risk to our sense of self.

> Being a process doesn't preclude something from being an entity

In a tortured semantics sense sure, but in a practical sense they refer to different types of things.

Re: Humanity isn't ready for the coming intelligence explosion

#370

Post by pheaded_while9 »

ilamont wrote: post #366

The first priority should be an agreement between the two heavyweights of ai: America and China. Donald Trump and Xi Jinping should affirm the principle that humans must remain stewards of ai systems until adequate frameworks for reliability and security have been built. This is naïve. The goals of both men have nothing to do with protecting humanity, but rather furthering their own personal agendas. In Trump’s case,…

Agreed, and their zeroeth order prerogative isn't much better: to ensure the continued need and demand for the thrall of centralized authority that allows them to achieve their disparate goals. This ensures strong AI is never going to be able to empower autonomy that removes, causes questioning, or even causes awareness of their zeroeth order.
Post reply on HN