Among the A.I. doomsayers • Hacker News Board

Re: Among the A.I. doomsayers

#371

Post by kouru225 » Fri, Mar 15, 2024, 6:35 PM UTC

HKH2 wrote: post #365

kouru225 wrote: post #359

Earlier quoted context omitted.

>Yeah, like judging human history and deciding murderous humans need to be removed from the earth or enslaved for their own benefit. (Some people already argue that the world would be better off without humans.) >Also, we know torture is wrong because we have a sense of pain, but an AI would just have to take our word for it. That doesn't seem too balanced to me. Again, you have this innate belief that immorality is…

I have the belief that humans are innately selfish and shortsighted. Obviously corruption is not viable long-term. Laws should make corruption not strategically beneficial, but if you can find loopholes or people to bribe, they are more or less irrelevant. What would make an AGI not try to game the system?

Lemme ask you: if murder were legal do you think we’d be able to build and produce so much? Of course not. It doesn’t matter what the laws are. Reality makes corruption strategically unviable.

Re: Among the A.I. doomsayers

#372

Post by HKH2 » Sat, Mar 16, 2024, 1:15 AM UTC

kouru225 wrote: post #371

HKH2 wrote: post #365

Earlier quoted context omitted.

I have the belief that humans are innately selfish and shortsighted. Obviously corruption is not viable long-term. Laws should make corruption not strategically beneficial, but if you can find loopholes or people to bribe, they are more or less irrelevant. What would make an AGI not try to game the system?

Lemme ask you: if murder were legal do you think we’d be able to build and produce so much? Of course not. It doesn’t matter what the laws are. Reality makes corruption strategically unviable.

True. Let me be more precise: the legal system forces (or is meant to force) shortsighted people to deal with long-term consequences. The law is constantly revised to preempt future consequences, but it is not perfect. (Ex post facto laws are used to punish people who have committed actions that were legal at the time. Loopholes will always exist because language is always somewhat imprecise.)

Now, you might say that AGI will be able to perfectly predict the future of all actions and make the legal system redundant. Ignoring the absurd amount of possibilities, I struggle to believe it's possible because an AGI has no sense of pain to give actions weight; it just has to follow human precedent. Seeming good enough, it could steer humanity into a dead end unknowingly. Also, it might decide that a few eggs have to be broken to make a utopian omelette.

Re: Among the A.I. doomsayers

#373

Post by kortilla » Sat, Mar 16, 2024, 5:05 AM UTC

jjeaff wrote: post #221

kortilla wrote: post #91

Earlier quoted context omitted.

A refinement facility for nuclear material isn’t quite the same thing

It's definitely part of the infrastructure of nuclear weapons. That's why it was hacked in the first place. They were making weapons grade nuclear material.

So is a steel plant though. Keeping a missile from being hacked is much different than something in the supply chain.

Re: Among the A.I. doomsayers

#374

Post by bumby » Mon, Mar 18, 2024, 9:53 PM UTC

kortilla wrote: post #373

jjeaff wrote: post #221

Earlier quoted context omitted.

It's definitely part of the infrastructure of nuclear weapons. That's why it was hacked in the first place. They were making weapons grade nuclear material.

So is a steel plant though. Keeping a missile from being hacked is much different than something in the supply chain.

Considering the number of steel plants and the relative cost of the material, that’s not a very good comparison. The whole point of Stuxnet was that the facility was a critical single failure point in the supply chain. When it went down, it disrupted the project for years. I don’t think the same dynamic holds for steel manufacturing. If you think they’re comparable, try getting into Oak Ridge or similar; I suspect you’ll find it very well guarded compared to a steel plant.