Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

Post reply on HN
Print view View original

351–360 of 374 posts

Re: Among the A.I. doomsayers

#351

Post by ehnto »

kouru225 wrote: post #299
ehnto wrote: post #73

Earlier quoted context omitted.

So that we can record our demise in high definition? Technology is amazing, and we have our civilization to be thankful for. But there are definitely technologies that in retrospect, if we had dodged them on the tech tree, we would be better off. Life would have continued just fine without many technologies, and I think AGI is one of them. We do not need AGI to survive and grow, and therefore any risk it imposes shou…

Disagree. We’re already on the verge of collapse. We definitely need AGI.

From your perspective, what does AGI do to solve our current set of predicaments?

Re: Among the A.I. doomsayers

#352

Post by ehnto »

BriggyDwiggs42 wrote: post #92
ehnto wrote: post #73

Earlier quoted context omitted.

So that we can record our demise in high definition? Technology is amazing, and we have our civilization to be thankful for. But there are definitely technologies that in retrospect, if we had dodged them on the tech tree, we would be better off. Life would have continued just fine without many technologies, and I think AGI is one of them. We do not need AGI to survive and grow, and therefore any risk it imposes shou…

I think this perspective ignores that the only way to semi permanently improve the lives of future humans is technology, and that quality of life is of similar importance to survival, especially at its extremes. AGI, if we achieve it, could genuinely make human lives much better.

Certainly, but it could also be a net negative. We can however already identify ways in which it can benefit and harm our societies in the short term, so we should at least be considering mitigations of the negative aspects.

Re: Among the A.I. doomsayers

#353

Post by graemep »

K0balt wrote: post #339
graemep wrote: post #246

Earlier quoted context omitted.

Actually, it will have the opposite effect, at least in the short term. People who own high value assets (everything from land to the AI) will continue to own them and there will be no opportunities for people to earn their way up (because they can be replaced by AI). "The logical endpoint is the subjugation or elimination of our species" Possibly, but it would be by our species (those who own and control the AI) rat…

I would venture to say that transhumanism will be the path and goal of the capital class, as that will be a tangible advantage potentially within their grasp. I suppose then that they would become “homo sapiens sapiens sapiens” or some other similarly hubris laden label, and go on to abandon, dominate or subjugate the filthy hordes of mere Homo sapiens sapiens.

Yes, a theme in cyberpunk SF, but also in much older works, such as CS Lewis's That Hideous Strength.

Re: Among the A.I. doomsayers

#354

Post by crotchfire »

inference-lord wrote: post #345
crotchfire wrote: post #307

Earlier quoted context omitted.

I think you have completely missed the point: > the government liking gigantic corporations is what makes it hard — the corporation (and, it is feared, AI) convinces the people The corporations didn't convince the people. They didn't have to. They convinced the government , which was already acting contrary to our interests. There's a tendency to assume that just because we run an election every few years, the govern…

I think there is some truth to this, but what about the government opening and a case against Boeing? If they were just in it for the corporation, would this be happening? Maybe it's all a show trial to show the American people they still care, but let's see what happens.

I think the fact that things had to get this bad before anything happened about Boeing says quite a lot.

Re: Among the A.I. doomsayers

#355

Post by crotchfire »

Animats wrote: post #329
crotchfire wrote: post #229

Earlier quoted context omitted.

> Corporations are indeed a difficult alignment problem No, it's a very easy alignment problem: just impose a tax on size . That which you tax, you get less of. The tax is revenue-neutral, so it doesn't affect the government budget one way or the other. Full details here: https://news.ycombinator.com/item?id=39235533 https://news.ycombinator.com/item?id=39236786 The problem is that the government likes gigantic corpo…

> impose a tax on size. Then you get lots of little companies with interconnected management. Read up on the early years of antitrust law. Find out how keiretsu and chaebol work. See the early history of the electrical industry and the Utility Holding Company Act. Understand how McDonalds franchise agreements work. Check out the current arguments over the definition of "beneficial employer".

Try pulling that one on the IRS sometime and see how it goes:

https://www.irs.gov/pub/irs-tege/epchd704.pdf

You'll lose your audit.

Common control and beneficial ownership have been well-established principles in the US for over 50 years.

Re: Among the A.I. doomsayers

#356

Post by PoignardAzur »

toomuchdocs32 wrote: post #136
matteoraso wrote: post #31

I've always wondered why people think that an AI that's super-intelligent will also be evil. It could just as likely end up being very kind (more than likely, actually, because the programmers would have safeguard to ensure that it's nice).

Long time ago I was reading translated accounts of Rwandan Hutu's that had participated in the 1993 genocide. One in particular had stuck with me; one account of a man that had murdered his childhood friend. As he described it, standing there having gutting and dismembers the man he had grown up side by side with, he had a sense of exhilaration. With all the wealth he thought of the wealth he had now, a tin roof, cat…

No kidding, I take the Paris metro and every time there's a "this line is delayed because a person fell/jumped on the tracks" notice, people around have the exact same sociopathic reaction. It's uncanny.

Not much to do with AI x-risk though.

Re: Among the A.I. doomsayers

#357

Post by jprete »

clooper wrote: post #94
jprete wrote: post #90

Earlier quoted context omitted.

Anything that reproduces with survivable mutations, and consumes resources, is immediately subject to evolutionary pressure. Neither of those require biology.

The only things currently consuming resources are humans and at the rate we are consuming resources we would need 2 Earths to support that rate of consumption. Worrying about self-reproducing AI is putting the cart before the horse. No one has yet figured out how to make existing humans aligned with biospheric conditions which would be sustainable. I personally welcome any form of intelligence which can make a sustai…

I am sympathetic to the point. I was somewhat concerned about nuclear war with Russia's Ukraine invasion, and I've become more concerned about climate change over time. I'm hoping either solar continues to expand - woot! - or we get over our fear of nuclear, or fusion finally becomes viable.

My point still stands - an AI that is reproducing (and/or mutating) by any method is going to be subject to selection pressure, just as any biological process is. All ML systems are effectively under selection pressure from the training process, and so they learn unexpected and mostly undesirable optimizations like "use a game-engine glitch to gain infinite flight and go to the goal without solving the maze" or "read text off the scanned X-ray sheet to get additional clues as to the correct diagnosis".

Re: Among the A.I. doomsayers

#358

Post by jprete »

bglazer wrote: post #330
hollerith wrote: post #201

Earlier quoted context omitted.

Long ago, people a lot like you were saying we should go ahead with AI research, but we shouldn't do anything dangerous like giving the AI access to the internet or giving it the ability to write and run computer programs. (Giving it the ability to run code was back then considered dangerous because it might create an AI with the same goals it has, only more capable, then that second AI might use its novel capabiliti…

First, you're the popular resistance to AI research. The movement already exists and is already prominent enough to be written up in the New Yorker. If your response is that the resistance is just a small set of people in SF, then I would refer you to the resistance to a much less capable class of systems: generative image models. Artists are having their livelihood (partially) disrupted by AI and they've been quite…

I don't want to rely on this theory, exactly, but it's a very plausible outcome, and exactly the kind of homeostatic process that I would expect to exist.

Re: Among the A.I. doomsayers

#359

Post by kouru225 »

HKH2 wrote: post #344
kouru225 wrote: post #343

Earlier quoted context omitted.

“Hubris” is a moral judgment that implies a strategic outcome. An AGI wouldn’t have a superego, but it would have the ability to study data across really long timescales and apply itself to long term strategies that would supersede the need for a superego. Since we apply morality as a shorthand for describing agreed upon negative consequences for complex behaviors, we can assume that an AI can overcome this need by s…

> And we can assume that AI can even iterate on and develop on its own understanding of these complex calculations much faster than a human can, making it significantly better at navigating moral issues. Yeah, like judging human history and deciding murderous humans need to be removed from the earth or enslaved for their own benefit. (Some people already argue that the world would be better off without humans.) Also,…

>Yeah, like judging human history and deciding murderous humans need to be removed from the earth or enslaved for their own benefit. (Some people already argue that the world would be better off without humans.)

>Also, we know torture is wrong because we have a sense of pain, but an AI would just have to take our word for it. That doesn't seem too balanced to me.

Again, you have this innate belief that immorality is somehow strategically beneficial and IMO you should really look into that

Re: Among the A.I. doomsayers

#360

Post by kouru225 »

ehnto wrote: post #351
kouru225 wrote: post #299

Earlier quoted context omitted.

Disagree. We’re already on the verge of collapse. We definitely need AGI.

From your perspective, what does AGI do to solve our current set of predicaments?

It solves our labor issue by upending millennia of forced labor practices.

It solves our issue with knowledge and data by allowing us to automatically find patterns within massive amounts of complicated datasets.

These are honestly the underlying causes of so much evil.

Moreover, it provides a new consensus mechanism. Using the machine as a method of consensus allows humans to have a third party mediate disagreements.

Post reply on HN