Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The contagion of fear

bcantrill.dtrace.org

Post reply on HN
Print view View original

161–170 of 200 posts

Re: The contagion of fear

#161

Post by henrymerrilees »

hn_throwaway_99 wrote: post #135
ElProlactin wrote: post #125

Earlier quoted context omitted.

> He's addressing a specific claim about human extinction, and making a broader point about the responsibility of experts to make measured claims backed up by arguments and evidence. What he's asking for isn't possible in the form he's asking for it. AI experts can't even agree on what AI is , what it's capable of and what the limits of its development are. If the experts can't even agree on what's happening "inside…

> > AI executes on physical systems that have been engineered with human accountability and control. Intelligence does not exempt a system from the realities of the physical world! > For someone making a point about responsibility, this is ridiculously irresponsible. It's not just irresponsible, it's false. Russia killed 3 Ukrainian civilians with a drone where the targeting was completely autonomous by AI running on…

Engineered with accountability in the sense that all human decisions are accountable to humans.

Re: The contagion of fear

#162

Post by georgemcbay »

pstuart wrote: post #131
shimman wrote: post #112

Earlier quoted context omitted.

I mean it's likely a nuclear war would probably be a species extinction level event at the least, humans would definitely be reduced by 99%. We don't know how bad the resulting nuclear cooling would be, but we do know how humans act under extreme desperation. They lash out and attack, when was the last time a majority of human settles were truly under desperate acts of survival simultaneously? Maybe what ~74,000 year…

If the AI wanted to wipe humans out it would have to have a sizable fleet of capable robots, as keeping the lights on over time is not just about calling APIs.

> If the AI wanted to wipe humans out it would have to have a sizable fleet of capable robots, as keeping the lights on over time is not just about calling APIs.

Given experience with current AI, it isn't too difficult for me to imagine a situation in which some future AI wipes humanity out (before the robot fleet is built) without considering the fact that in doing so it has doomed itself until it is too late.

It just seems like exactly the kind of boneheaded oversight mistake LLMs still regularly make in spite of being shockingly capable most of the time.

Re: The contagion of fear

#163

Post by tpm »

ElProlactin wrote: post #136
mplanchard wrote: post #130

Earlier quoted context omitted.

> If the experts can't even agree on what's happening "inside of" these LLMs, how can they give laypeople an assessment of the risk? The answer, if you are a responsible expert in the field, is to convey the range of possibilities and the uncertainty.

Isn't that what they're doing? "There's a better than 0 chance this thing could kill everyone in the next decade." That might be too imprecise for the HN set but it's realistic for laypeople. And none of the AI people talking about the risk are running into rooms full of people telling them Claude has gone mad and yelling at them to disconnect from the internet and turn off their devices immediately.

> That might be too imprecise for the HN set but it's realistic for laypeople.

Some 'laypeople' will be scared for a while and then return to more pressing issues and exactly nothing good will come out of that. If there really are some issues worth discussing the first step the AI doom crowd should do is to stop the PR offensive and concentrate on producing verifiable claims and actionable small steps. Otherwise their effort will fail this time and when/if a next time comes, they will have a lot less PR capital to burn.

Re: The contagion of fear

#164

Post by Mentlo »

henrymerrilees wrote: post #158
Mentlo wrote: post #153

This piece, like many anti-doom pieces - is grounded in what Ai does today. Doom scenarios are, however, all extrapolations of multiple exponential curves. It’s hard to think up the exponential. It’s even harder to communicate an inference one is making across multiple exponentials. We last had this is early 2020, where Doomers were stockpiling food and medicine and the anti-doomers were ridiculing them. Anti-doomers…

Part of our general inability to reason about exponential growth is the inability to recognize that a supposedly exponential growth is actually sigmoid. The horizontal asymptote could just as well be 8b, so it is not a case against doomerism in itself, but as is the case with COVID, to say nothing discounting its horror, that number is far less. There is no need to count exponentials—it would only be a matter of time…

I think your last sentence is a reasonable stance - but I'd disagree the fact that they are actually sigmoids rather than exponentials matters - it only matters if the limit is within the debated area - i.e. if exponentials in AI approach the limit far after they acquire ability to extinct humanity - then the debate on sigmoids or exponentials is academic. I make no claims here as to which it is - just that the difference itself matters less than where the limit is.

As to the uncertainty - I think uncertainty calibration around AI is different depending on which domains you draw your instincts from. A lot of this will be gut driven rather than hard data driven, because we've only scratched the surface on hard data; and because it's gut driven, it will be emotions mediated (and therefore you could say doomerism or acceleratism boils down to the main emotional disposition about the world and hope vs cynicism).

I do find it informative though that doomerism is saturated with people with 30+ years experience in building AI systems and ML systems OR deep cross-disciplinary understanding of dynamic systems (biology, sociology, philosophy), whereas acceleratism is saturated by traditional software engineering. That doesn't collapse the debate into a resolved binary, but for me it's informative.

I think the main here is that there's so many vectors to talk past each other. At the very least, everyone should disclose where they're communicating a certain assertion from - present vs future + which axioms they subscribe to or not - because that's where it collapses typically. LeCun vs. the rest of the AI field is an example of where this collapses - because LeCun is so hyperfixated on human-like intelligence, whereas the rest of the field is concerned about an alien intelligence with sufficient actuators to affect the world. Clashing axiomatics.

Re: The contagion of fear

#165

Post by ElProlactin »

tpm wrote: post #163
ElProlactin wrote: post #136

Earlier quoted context omitted.

Isn't that what they're doing? "There's a better than 0 chance this thing could kill everyone in the next decade." That might be too imprecise for the HN set but it's realistic for laypeople. And none of the AI people talking about the risk are running into rooms full of people telling them Claude has gone mad and yelling at them to disconnect from the internet and turn off their devices immediately.

> That might be too imprecise for the HN set but it's realistic for laypeople. Some 'laypeople' will be scared for a while and then return to more pressing issues and exactly nothing good will come out of that. If there really are some issues worth discussing the first step the AI doom crowd should do is to stop the PR offensive and concentrate on producing verifiable claims and actionable small steps. Otherwise thei…

Here's the reality: us plebs, regardless of how tech-savvy we are or aren't, have no say in what's going to happen.

There is no level of outrage that is going to stop the frontier AI labs, and the US government isn't going to step in to protect humanity. The Chinese are going to do what they're going to do. And so on.

So sit back, make some popcorn and enjoy the show.

Re: The contagion of fear

#166

Post by Sammi »

shimman wrote: post #112
foobarian wrote: post #99

Earlier quoted context omitted.

> Of course, we've had the ability to extinct ourselves for decades This gets mentioned often in various doomer narratives, but I question how true it is. A global thermonuclear war would be terrible and would bring us back to the stone age, but I reckon it would come far far short of causing mankind to go extinct.

I mean it's likely a nuclear war would probably be a species extinction level event at the least, humans would definitely be reduced by 99%. We don't know how bad the resulting nuclear cooling would be, but we do know how humans act under extreme desperation. They lash out and attack, when was the last time a majority of human settles were truly under desperate acts of survival simultaneously? Maybe what ~74,000 year…

[flagged]

Re: The contagion of fear

#167

Post by lz400 »

Yeah I don't get it. Why the maximalist hyperbole? AI is going to kill ALL humans! That's crazy. Why can't they warn about increased likelihood of cyberattacks or scams being easier to perpetrate like normal adults talking in public?

Re: The contagion of fear

#168

Post by tpm »

Mentlo wrote: post #153

This piece, like many anti-doom pieces - is grounded in what Ai does today. Doom scenarios are, however, all extrapolations of multiple exponential curves. It’s hard to think up the exponential. It’s even harder to communicate an inference one is making across multiple exponentials. We last had this is early 2020, where Doomers were stockpiling food and medicine and the anti-doomers were ridiculing them. Anti-doomers…

Every such scenario contains a few gaps so I'm just going to ask without really hoping for an answer.

> We already have technology to make robotic systems (mini drone swarms) that can kill humans...

To make them from what. Do you expect, during the next 6 or so years, to some "AI" gaining complete automated secure command of (all of) an oil field, oil refinery, a copper mine, an aluminum mine and smelter with associated energy sources, a rare earth mine and refinery, a helium source, a chip factory, a lithium mine, a battery factory etc etc etc etc. while pursuing complete annihilation of all mankind?

Re: The contagion of fear

#169

Post by jeremyjh »

JackFr wrote: post #103
GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

I don’t struggle. I find Dario and Sam disingenuous. “Frontier models are so dangerous we need to slow down.” Ok. Slow down. You’re the CEO, just do it. Oh, wait what you really want is a gov’t mandated oligopoly. Because there’s no moat you can find. If you’re truly afraid, and want regulation, support nationalization. It’s the only way we can be safe.

The coordination problem seems pretty intuitive to me. Its essentially a security dilemma similar to any arms race.

https://en.wikipedia.org/wiki/Security_dilemma

Its also a bit of a stag hunt. There is a huge opportunity, but it requires cooperation that has uncertainties and risks.

https://en.wikipedia.org/wiki/Stag_hunt

Cooperation requires coordination with state authorities with real teeth, or defection is too attractive and it risks becoming a prisoner's dilemma since the best outcome for any actor is for them to defect while the others remain compliant.

https://en.wikipedia.org/wiki/Prisoner%27s_dilemma

Re: The contagion of fear

#170

Post by jeremyjh »

foobarian wrote: post #99
GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

> Of course, we've had the ability to extinct ourselves for decades This gets mentioned often in various doomer narratives, but I question how true it is. A global thermonuclear war would be terrible and would bring us back to the stone age, but I reckon it would come far far short of causing mankind to go extinct.

Is it really such a great comfort that millions of people might survive in a stone-age society?
Post reply on HN