Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

Post reply on HN
Print view View original

331–340 of 374 posts

Re: Among the A.I. doomsayers

#331

Post by narag »

crotchfire wrote: post #229
mitthrowaway2 wrote: post #170

Earlier quoted context omitted.

Corporations are indeed a difficult alignment problem, as anyone who has tried to design an incentive scheme eventually realizes. But corporations are still composed of humans, beholden (if only loosely) to human values, limited (if loosely) by human intelligence, and are not very well-coordinated decisionmakers with coherent goals.

> Corporations are indeed a difficult alignment problem No, it's a very easy alignment problem: just impose a tax on size . That which you tax, you get less of. The tax is revenue-neutral, so it doesn't affect the government budget one way or the other. Full details here: https://news.ycombinator.com/item?id=39235533 https://news.ycombinator.com/item?id=39236786 The problem is that the government likes gigantic corpo…

No, it's a very easy alignment problem: just impose a tax on size.

That's a very bad idea when you're dealing with small to medium companies. You're taxing economies of scale and innovation. That's a huge problem in my country. There's no tax per se, but many regulations that trigger with certain size. Guess what happens.

For medium to big companies is calling for loopholes and cartels.

Re: Among the A.I. doomsayers

#333

Post by 13years »

tivert wrote: post #318
13years wrote: post #278

Earlier quoted context omitted.

>That is a very good and insightful article. Thank you! > The only issue with the article is sometimes it's too optimistic, e.g. The reply below is criticizing for being too pessimistic, lol :-) I agree, it might be only a transient state. We really can't comprehend how we would deal with such existence if at all. Would we manage to keep our sanity? Nonetheless, I don't claim to have the right answers, just questions…

> What I do find interesting, are those that read it and still don't perceive it at all. Their response is "That is exactly what I want." I think there's a basically (new movement) religious desire to see a lot of these technologies come to fruition, either radically fundamentalist or lazier and less informed, that interferes with the application of the implications. They believe in the idol of technology, as told in…

Yes, I agree that is some component of it as I have felt the same vibe from some.

Re: Among the A.I. doomsayers

#334

Post by namaria »

semi-extrinsic wrote: post #291
namaria wrote: post #270

Earlier quoted context omitted.

Because, as unlikely as it is, if we're discussing risk scenarios for AI getting out of hand. Well then a monolithic superintelligence is just one of the possibilities. What about a swarm of dumb AIs that are nonetheless capable of reasoning and decision making and they become a threat? That's pretty much what we did. There's no super intelligent monkey in charge. As much as some have tried to pretend, material or ot…

The difference is that we have 100% complete control of these AIs. We can just go into the power grid substation next to the data center and throw the big breaker, and the AI ceases to exist. When humans developed, we did not displace an external entity that had created us and that had complete power to kill us all in an instant.

If shutting down datacenters is your solution to AI getting out of hand, it isn't as much a solution as fall out.

Re: Among the A.I. doomsayers

#335

Post by XorNot »

theptip wrote: post #283
XorNot wrote: post #193

Earlier quoted context omitted.

Or it could just be a normal outcome? I'm not sure where the malevolence comes in here. Or to put it another way, replace AI with "capitalism" or something - something we know isn't a conscious agent. The outcome just...is an outcome. No one's trying to eliminate anyone. The problems only creep in at the edges, which is where you get your paperclip optimizer sorts of problems: if the directive at it's core is "maximi…

> No one's trying to eliminate anyone Probably not any consolation to the Neanderthal or any other animal species that is out-competed in nature. Doesn’t have to be malevolent or intentional for a more capable species to render others extinct. (Indeed that would seem to be the base case, historically speaking.) of course, symbiosis is possible; we could be puppies to the AI. Many people find disempowerment to be almo…

> Probably not any consolation to the Neanderthal or any other animal species that is out-competed in nature.

It is not clear from the geological record if the Neanderthal was "outcompeted" except in the broadest possible terms - i.e. a unique species of Neanderthal no longer exists genetically.

But that doesn't tell us what happened in the interim - i.e. modern day humans can have up to 4% neandertal origin DNA[1]. So, did "humans" wipe out neanderthals? Or did human plan genetics simply become the more dominant type, no malevolent will required, and does that matter?

Because for a whole lot of people - myself included - the explicit goal of technology like AI is something which from a fossil record would look like human extinction - that's the whole idea of transhumanism, that ultimately we would become something sharing perhaps only a glancing similarity with our ancestors.

All the rest of the problems you talk about suffer from a lack of detail - namely, they assume a quantum leap in technological capability with no intervening steps - i.e. there is a single AI, or that AIs are better negotiators or what have you - as soon as you get into the details, the threat model proposes evaporates because we just don't know - the technology doesn't exist (i.e. it would all be a lot more interesting if fully automated production lines were already around, but we're nowhere near that).

[1] https://humanorigins.si.edu/evidence/genetics/ancient-dna-an...

Re: Among the A.I. doomsayers

#336

Post by semi-extrinsic »

ben_w wrote: post #247
semi-extrinsic wrote: post #216

Earlier quoted context omitted.

There are a lot of assumptions going on here. One of them is that superintelligent AI will arise. We have no reason to believe this will happen in our lifetimes. I posit that we are about as close to superintelligent AI as James Watt was to nuclear fusion. The other assumption is that wealth and power are distributed according to intelligence. This is obviously false, wealth and power are largely distributed accordin…

> There are a lot of assumptions going on here. One of them is that superintelligent AI will arise. We have no reason to believe this will happen in our lifetimes. I posit that we are about as close to superintelligent AI as James Watt was to nuclear fusion. This is a perfectly reasonable response if nobody is trying to build it. Given people are trying to build it, what's the expected value from ignoring the problem…

So I propose the Musk supremacy criterion to be the following.

Suppose that a wealthy and powerful human (such as Elon Musk) were to suddenly obtain the exact same sinister goals as the hypothetical superintelligent AI in question. Suppose further that this human was able to convince/coerce/bribe another N (say 1000) humans to follow his bidding.

A BadOutcome is said to be MuskSupreme if it could be accomplished by the superintelligent AI, but not by the suddenly-evil Musk and his accomplices.

Obviously[citation needed] it is only the MuskSupreme BadOutcomes we care about. Do there exist any?

Re: Among the A.I. doomsayers

#337

Post by ben_w »

semi-extrinsic wrote: post #336
ben_w wrote: post #247

Earlier quoted context omitted.

> There are a lot of assumptions going on here. One of them is that superintelligent AI will arise. We have no reason to believe this will happen in our lifetimes. I posit that we are about as close to superintelligent AI as James Watt was to nuclear fusion. This is a perfectly reasonable response if nobody is trying to build it. Given people are trying to build it, what's the expected value from ignoring the problem…

So I propose the Musk supremacy criterion to be the following. Suppose that a wealthy and powerful human (such as Elon Musk) were to suddenly obtain the exact same sinister goals as the hypothetical superintelligent AI in question. Suppose further that this human was able to convince/coerce/bribe another N (say 1000) humans to follow his bidding. A BadOutcome is said to be MuskSupreme if it could be accomplished by t…

Not sufficiently detailed to be answerable.

For example 1000 people — but only if you get to choose who — is sufficient to take absolute control of both the US congress and the Russian State Duma (or a supermajority of those two plus the Russian Federation Council), which gives them the freedom to pass arbitrary constitutional amendments… so your scenario includes "gets crowned King of the USA and Russia, 90% of the global nuclear arsenal is now their personal property" as something we don't care about.

Re: Among the A.I. doomsayers

#338

Post by semi-extrinsic »

namaria wrote: post #334
semi-extrinsic wrote: post #291

Earlier quoted context omitted.

The difference is that we have 100% complete control of these AIs. We can just go into the power grid substation next to the data center and throw the big breaker, and the AI ceases to exist. When humans developed, we did not displace an external entity that had created us and that had complete power to kill us all in an instant.

If shutting down datacenters is your solution to AI getting out of hand, it isn't as much a solution as fall out.

Look at the measures that were implemented during covid. Many of them were a lot more extreme than shutting down datacentres, yet they were aimed to mitigate a risk far less than "existential".

Re: Among the A.I. doomsayers

#339

Post by K0balt »

graemep wrote: post #246
statuslover9000 wrote: post #203

Earlier quoted context omitted.

The short-term effect is a harbinger of the long-term risk, since capitalism doesn’t inherently care for people who don’t provide economic value. Once superintelligent AI arises, none of us will have value within this system. Even the largest current capital holders will have a hard time holding on to it with an enormous intelligence disadvantage. The logical endpoint is the subjugation or elimination of our species,…

Actually, it will have the opposite effect, at least in the short term. People who own high value assets (everything from land to the AI) will continue to own them and there will be no opportunities for people to earn their way up (because they can be replaced by AI). "The logical endpoint is the subjugation or elimination of our species" Possibly, but it would be by our species (those who own and control the AI) rat…

I would venture to say that transhumanism will be the path and goal of the capital class, as that will be a tangible advantage potentially within their grasp.

I suppose then that they would become “homo sapiens sapiens sapiens” or some other similarly hubris laden label, and go on to abandon, dominate or subjugate the filthy hordes of mere Homo sapiens sapiens.

Re: Among the A.I. doomsayers

#340

Post by supertimor »

p1esk wrote: post #332
HankB99 wrote: post #314

Earlier quoted context omitted.

Collectively, yes. Individually not so much.

A superior race (AI or aliens) might also see some purely aesthetic value in individual humans, like butterfly collectors.

Ahh, so we are to be kept as pets or have our dead bodies arranged in display cases.
Post reply on HN