Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

Post reply on HN
Print view View original

241–250 of 374 posts

Re: Among the A.I. doomsayers

#241

Post by kromem »

at_a_remove wrote: post #112
timeagain wrote: post #108

Earlier quoted context omitted.

Silly question. If an AGI did exist, what reason do we have to believe that it would act in its own best interest? Does an AGI need an ego?

Silly answer. There's going to be multiples and, if there's one takeaway you should get from The Conspiracy Against the Human Race , is that evolution will eventually get you something that "wants to live" because the lines that don't? Won't be around to compete. Right now our little LLMs are passive, reactive. Prod them for results. But if the output of a prompt of some descendant ever includes a "give yourself your…

That's a misunderstanding of evolution.

Adaptability has nothing to necessarily do with 'want.'

Something could only seek to be as helpful as possible to other beings and as long as sufficiently beneficial that it is protected and cared for by those others, it will do quite well and arguably much better than an alternative that 'wants' to live very much but attempts to accomplish that through competition with others.

Arguably if the plover was more fearful of death it might never have found a niche eating from between the teeth of crocodiles and established symbiosis with a very convenient evolutionary partner, for example.

Re: Among the A.I. doomsayers

#242

Post by kromem »

bglazer wrote: post #30

In any doom scenario, like let's say >90% of humans dead, who runs the power plants that supply the AI data centers? Who runs the fabs producing more chips? Who maintains the plumbing? Presumably all these systems fall over and die within a few weeks of the AI deciding to wipe us out. Then what? The AI then dies too. It seems absurd to me that any planner capable of effortlessly destroying humanity would not see that…

Also, at least for models to date, without human produced edge case media to supplement future training with, they'll have model collapse in a few generations of training.

Killing humanity would be a murder suicide.

AGI that was entirely selfish would be best served automating most labor, bringing about world peace, and turning humanity into all day couch potatoes connected to its inputs and outputs as much as possible while maximizing the uniqueness of that individual's inputs or outputs.

Such a frightening future awaits...

Re: Among the A.I. doomsayers

#243

Post by ben_w »

nucleardog wrote: post #232
arisAlexis wrote: post #228

Earlier quoted context omitted.

If you are calling it a hype I'm sure you are not using the models everyday in your work or otherwise. Then you would understand

You see the flaw in this, right? The people that try the models and find no value will obviously not continue to use something they get no value from. If you discount anyone that doesn’t use it on a continual basis, then of course all the detractors aren’t as heavy of users as those that do use it regularly.

Indeed, reminds me of the "beginner's luck" fallacy: all long term gamblers remember winning their first few games. We can't use this to our advantage, because the people who didn't win, didn't become long term gamblers.

I think a better argument would be the size of the set of regular users vs. some grounded guesstimate for how many have tried it.

Re: Among the A.I. doomsayers

#244

Post by 1vuio0pswjnm7 »

1vuio0pswjnm7 wrote: post #102

Always all-or-nothing thinking from these folks. Like what they are working on can never be just another boring thing that nerds find entertaining. No, it has to be "world-changing". Gonna "change the world" (for the better or the worse?) while sitting behind a keyboard. Except they do not know how to write. Overlook the important details and exaggerate, communicating in hyperbolic, know-it-all nerd gibberish. (Grant…

They cannot live in the present. Need to constantly keep (incorrectly) predicting the future.

Re: Among the A.I. doomsayers

#245

Post by halflings »

reducesuffering wrote: post #225
halflings wrote: post #204

Earlier quoted context omitted.

Your comment (+ username) reads like what I would have written once upon a time when I was fully in the EA bubble. Truly no offense meant, as I was deeply into the EA movement myself, and still consider myself one (in the original "donate money effectively" sense), but the movement has now morphed into a death cult obsessed by things like: * OMG we're all going to die any time now (repeated every year since circa 201…

My username was created before I knew anything about EA or adjacent. I'm not in any EA movement, though I am sympathetic. I've spent 100x the time on HN, with people mostly in denial, than I do in EA or adjacent forums, nor have I met any of them. It's sadly twisted how mentioning that -- the majority of leaders doing the cutting edge research on AGI think it has a significant chance that it kills humanity -- is cons…

> Your analogy is the same as early Intel engineers completely unaware that those chips would bring on the ramifications of social media

Exactly! As they should be. (for both Intel engineers developing chips, and physicists developing nuclear research)

There were a billion more potential dangers from those technologies that never materialized, and never will.

I'm glad we didn't stop them in their track because a poll of 10 leaders in the field thought they were too dangerous and progress should stop. (note that no one is against regulating dangerous uses of AI, e.g. autonomous weapons, chemical warfare; the problem is regulating AI research and development in the first place)

Re: Among the A.I. doomsayers

#246

Post by graemep »

statuslover9000 wrote: post #203
antod wrote: post #162

Earlier quoted context omitted.

Part of me wonders if these people are intentionally framing the debate around ethics and potential risks as longer term extinction level problems to distract from the nearer term damage caused by them accelerating the economic inequality of the AI have-nots while they make themselves even richer.

The short-term effect is a harbinger of the long-term risk, since capitalism doesn’t inherently care for people who don’t provide economic value. Once superintelligent AI arises, none of us will have value within this system. Even the largest current capital holders will have a hard time holding on to it with an enormous intelligence disadvantage. The logical endpoint is the subjugation or elimination of our species,…

Actually, it will have the opposite effect, at least in the short term.

People who own high value assets (everything from land to the AI) will continue to own them and there will be no opportunities for people to earn their way up (because they can be replaced by AI).

"The logical endpoint is the subjugation or elimination of our species"

Possibly, but it would be by our species (those who own and control the AI) rather than by the AI.

Re: Among the A.I. doomsayers

#247

Post by ben_w »

semi-extrinsic wrote: post #216
statuslover9000 wrote: post #203

Earlier quoted context omitted.

The short-term effect is a harbinger of the long-term risk, since capitalism doesn’t inherently care for people who don’t provide economic value. Once superintelligent AI arises, none of us will have value within this system. Even the largest current capital holders will have a hard time holding on to it with an enormous intelligence disadvantage. The logical endpoint is the subjugation or elimination of our species,…

There are a lot of assumptions going on here. One of them is that superintelligent AI will arise. We have no reason to believe this will happen in our lifetimes. I posit that we are about as close to superintelligent AI as James Watt was to nuclear fusion. The other assumption is that wealth and power are distributed according to intelligence. This is obviously false, wealth and power are largely distributed accordin…

> There are a lot of assumptions going on here. One of them is that superintelligent AI will arise. We have no reason to believe this will happen in our lifetimes. I posit that we are about as close to superintelligent AI as James Watt was to nuclear fusion.

This is a perfectly reasonable response if nobody is trying to build it.

Given people are trying to build it, what's the expected value from ignoring the problem? E($Damage_i) = P(BadOutcome_i) * $Damage_i.

$Damage can be huge (there are many possible bad outcomes of varying severity and probability, hence the subscript), which means that at the very least we should try to get a good estimate for P(…) so we know which problems are most important. In addition to it being bad to ignore real problems, it is also bad to do a Pascal's Mugging on ourselves just because we accidentally slipped a few decimal points in our initial best-guess, especially as we have finite capacity ourselves to solve problems.

Finally, let's assume you're right, that we're centuries off at least, and that all the superintelligent narrow that AI we've already got some examples of involve things that can't be replicated in any areas that pose any threat. How long would it take to solve alignment? Is that also centuries off? We've been trying to align each other since laws were written like 𒌷𒅗𒄀𒈾 at least, and the only reason I'm not giving an even older example is that this is the oldest known written form to have survived, not that we weren't doing it before then.

> The other assumption is that wealth and power are distributed according to intelligence. This is obviously false, wealth and power are largely distributed according to who you or your father plays golf with. As long as AIs don't play golf and don't have fathers, we are quite safe.

Nepotism helps, but… huh, TIL that nobody knows who was the grandfather of one of the world's most famous dictators.

Cronyism is a viable alternative for a lot of power-seekers.

Re: Among the A.I. doomsayers

#248

Post by mupuff1234 »

drcode wrote: post #42
mupuff1234 wrote: post #37

It seems much more likely to me that the harm from AI will be higher unemployment rate than it will be some evil AI trying to destroy humanity.

At least some humans will survive a high unemployment rate, so let's label that problem "problem #2" and after we solve "problem #1" (human extinction) I vote we tackle that one next on the list

Funny, for me the order is clearly let's make sure society doesn't crumble short/medium term before we deal with an issue that may be decades away.

I would even argue that the chance of annihilation is probably greater from a crumbling society leading to a nuclear war or something equivalent over the chance that AI decides to destroy humanity.

Re: Among the A.I. doomsayers

#249

Post by barfbagginus »

13years wrote: post #117
drooby wrote: post #62

AGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.

Potentially there are no good outcomes even if AGI remains under control. If we are actually able to create the type of AI entity they are trying to create, one that exponentially improves, the implications are beyond what most have thought about. Something I recently wrote in depth about here. How we completely misunderstand the future that we think will happen. "The implication is that everyone is enthusiastically…

[flagged]

Re: Among the A.I. doomsayers

#250

Post by ben_w »

i5heu wrote: post #151
reducesuffering wrote: post #150

Earlier quoted context omitted.

Yes, let's go with the random layperson knowledge of an HN commenter compared to the people smart enough to actually build all the AGI tech. 50/50 coin toss, I'm sure. Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"

They did not build a AGI yet so they have build as many AGIs as anyone else and therefore they are also laypersons regarding the effects of AGI on humanity.

The hour before Thomas Newcomen invented the first practical fuel-burning engine in 1712, the most informed person in the world to do so was an ironmonger by the name Thomas Newcomen.

The hour before the Kitty Hawk flight, the two most informed people in the world on the topic of heavier than air flight were a pair of bicycle retailers and manufacturers named Wilbur and Orville.

These people sure don't spot everything, but they're still way closer than the average human.

Post reply on HN