US military drone controlled by AI "killed" its operator during simulated test • Hacker News Board

Re: US military drone controlled by AI "killed" its operator during simulated test

#61

Post by silisili » Fri, Jun 02, 2023, 8:25 PM UTC

fennecfoxy wrote: post #41

silisili wrote: post #32

Earlier quoted context omitted.

That's my entire problem with AI or any machine decision. It has zero emotion, zero empathy, zero care. Imagine if AI were running the subs in the Cold War. None of us would probably even be here. We may be stupid creatures, but we're still creatures... https://en.m.wikipedia.org/wiki/Vasily_Arkhipov

As someone else commented, so do some humans. But if a system is designed with empathy/rules/etc in mind by an empathetic designer/committee then that system is already > than a heartless individual even if they are human.

You cannot 'program' emotion. You can probably get semi-close by trying it your way, in the same way that Tesla is semi-close to FSD. There's too many edge cases to make it perfectly humane.

Re: US military drone controlled by AI "killed" its operator during simulated test

#62

Post by News-Dog » Sat, Jun 03, 2023, 12:10 AM UTC

Vecr wrote: post #42

fennecfoxy wrote: post #40

Earlier quoted context omitted.

Do they, though? Didn't Russia have the dead man's hand system where any detected launches would result in a completely automatic response? But in the reality of an arm's race, I guess the core point is that we either: respond to autonomous weapons with our own and break our principle of having a human in the loop, or accept that our offence/defence with a human in the loop is not fast enough to keep up with an enemy…

Dead hand was probably (though to what confidence I don't know) people in a remote mountain bunker. It's not actually any better to have the computer fire, it just provides the people information. Retaliation does not have to be instant, 40 minutes is probably good enough.

Mutual Assured Destruction (MAD) : https://en.wikipedia.org/wiki/Mutual_assured_destruction

Mutual Assured Destruction (MAD) - Second-Strike Capability : https://en.wikipedia.org/wiki/Mutual_assured_destruction#Sec...

>Perfect detection

No false positives (errors) in the equipment and/or procedures that must identify a launch by the other side.

The implication of this is that an accident could lead to a full nuclear exchange.

During the Cold War there were several instances of false positives, as in the case of Stanislav Petrov[2].

[2] Stanislav Petrov : https://en.wikipedia.org/wiki/Stanislav_Petrov

[2] 1983 Soviet Nuclear False Alarm Incident : https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...

Re: US military drone controlled by AI "killed" its operator during simulated test

#63

Post by flagrant_taco » Sat, Jun 03, 2023, 1:08 AM UTC

Jarmsy wrote: post #50

flagrant_taco wrote: post #43

Earlier quoted context omitted.

Funny, this one stood out to me as less baity since they actually did include "in stimulated test" in the title

When the truth was even less dramatic - the events described did not occur in any computer simulation, just a human making up hypothetical scenarios.

One of my college CS textbooks had an anecdote in it where the Australian military once had a bug in flight training software that ended with kangaroos in the simulation pulling out AK47s and RPGs to shoot down the helicopter pilot.

The story was meant to make a point about class inheritance. No idea of that ever actually happened and I assume it didn't, but it was a great thought experiment to make a point if nothing else.

In this case whether its a though experiment or an actual event during one simulation, the point is important and it shines a light on real world risks if and when AI is given direct control over UAVs.

Re: US military drone controlled by AI "killed" its operator during simulated test

#64

Post by arisAlexis » Sat, Jun 03, 2023, 7:17 AM UTC

Rebelgecko wrote: post #59

arisAlexis wrote: post #58

Earlier quoted context omitted.

Someone inside explained a situation. The army denied a f* up. Is your read that this f up never happened?

Which USAF fuck-up are you referring to? The original exercise, which may or may not have happened? Or Colonel Hamilton making comments about it and then retracting them?

Why so confusing? The colonel talked about a f up. The army denied it. To me yes it happened unless the guy is a certified mental resident.

Re: US military drone controlled by AI "killed" its operator during simulated test

#65

Post by rbanffy » Sat, Jun 03, 2023, 9:02 PM UTC

arisAlexis wrote: post #26

utucuro wrote: post #25

Earlier quoted context omitted.

To be frank, I'm not afraid of AI, I'm afraid of the exact opposite - natural stupidity. This story clearly shows that too - "algorithm trained to prioritize scoring points tries to do so at any cost" It's not even emergent behavior, it's badly defined goals...

Read up about the "alignment problem". It is one that ai researchers cannot Crack and its literally what you wrote. The idea that we could never in any way define goals align with us.

We have been living with this problem of poorly defined goals for ages. Computers do what they are told to, not what we want them to. I built a career on translating what people want computers to do into what computers need to be told.

I suspect we'll hear of Dr. Susan Calvin more and more in the future.

Re: US military drone controlled by AI "killed" its operator during simulated test

#66

Post by more_corn » Sun, Jun 04, 2023, 12:27 AM UTC

This guy came out and said he misspoke. He imagined a situation where an AI might kill its handler so it could better complete the mission.

It was a thought experiment. AKA an imagined scenario. No real person died. No AI has gone rogue.

Re: US military drone controlled by AI "killed" its operator during simulated test

#67

Post by fennecfoxy » Mon, Jun 05, 2023, 10:29 AM UTC

silisili wrote: post #61

fennecfoxy wrote: post #41

Earlier quoted context omitted.

As someone else commented, so do some humans. But if a system is designed with empathy/rules/etc in mind by an empathetic designer/committee then that system is already > than a heartless individual even if they are human.

You cannot 'program' emotion. You can probably get semi-close by trying it your way, in the same way that Tesla is semi-close to FSD. There's too many edge cases to make it perfectly humane.

Empathy != experiencing emotions necessarily, just being aware of it.

We wouldn't want to program emotion into a killing machine anyway. Emotions are messy, meatbag programs that make us do suboptimal things driven by our own cultural, societal and personal biases.

Trolley problem where one side is one person who is dear to you, other side is two people you don't know. The majority of people would save the person they know well even if two innocent but unknown people die.

Also we probably can't specifically program emotion until we understand it better, but I reckon we'll see emotions/similar concepts arise as we build more intelligent machines. Humans are not special, we're just meat computers.