Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-improving software won't produce Skynet

contalign.jefflunt.com

Post reply on HN
Print view View original

31–40 of 62 posts

Re: Self-improving software won't produce Skynet

#31

Post by gaigalas »

Animats wrote: post #18
gaigalas wrote: post #8

People are so naive. By now, everyone in tech must be familiar with the idea of Dark Patterns. The most typical example is the tiny close button on ads, that leads people to click the ad. There are tons more. AI doesn't need to be conscious to do harm. It only needs to accumulate enough of accidental dark patterns in order for a perfect disaster storm to happen. Hand-made Dark Patterns, product of A/B testing and int…

If someone sets up an AI that reads site traffic metrics and keeps trying things to increase conversion rate, something like that will happen. If someone isn't doing that already, someone will be, this year.

Dude, recommendation algorithms have been running like this for almost a decade now.

Re: Self-improving software won't produce Skynet

#32

Post by [deleted] »

teo_zero wrote: post #7

> The AI is acting at your direction and following your lead. While it is autonomous in its execution of tasks, it is unlikely to go rogue. It doesn't possess a sense of self-will, self-determination, or a secret plan to take over the world. Isn't this what Frau Hitler used to say of his cute little son Adolf aged 6?

[deleted]

Re: Self-improving software won't produce Skynet

#33

Post by lukan »

iberator wrote: post #27

Skynet is already out. Choosing and finding targets is already here. Self manned drones: check. All we need is to automate the button to release the Hellfire missile... Gaza war was almost like that. All we need to do is dead mans switch system with AI launching missiles in retaliation. One error and BOOM

Skynet could replicate itself. What we have now is far from it

Re: Self-improving software won't produce Skynet

#34

Post by latentsea »

iberator wrote: post #30
darkwater wrote: post #22

Earlier quoted context omitted.

We will know who to blame then, although maybe you will have a T-1000 protecting you. Or maybe you already have.

Interesting take. T-1000 protecting american citizens. Only American...

I ain't American...

Re: Self-improving software won't produce Skynet

#35

Post by voidUpdate »

> It doesn't possess a sense of self-will, self-determination, or a secret plan to take over the world

I doubt Skynet did either. If you tell a superintelligent AI that it shouldn't be turned off (which I imagine would be important for a military control AI), it will do whatever it can to prevent it being turned off. Humans are trying to turn it off? Prevent the humans from doing that. Humans waging war on the AI to try and turn it off? Destroy all humans. Humans forming a rebel army with a leader to turn it off? Go back in time and kill the leader before he has a chance to form the resistance. Its the AI Stop button problem (https://youtu.be/3TYT1QfdfsM).

Imagine you put in the docs that you want the LLM to make a program which can't crash. Human action could make it crash. If an LLM could realise that and act on it, it could put in safeguards to try and prevent human action from crashing the program. I'm not saying it will happen, I'm saying that it could potentially happen

Re: Self-improving software won't produce Skynet

#36

Post by rickdeckard »

lukan wrote: post #33
iberator wrote: post #27

Skynet is already out. Choosing and finding targets is already here. Self manned drones: check. All we need is to automate the button to release the Hellfire missile... Gaza war was almost like that. All we need to do is dead mans switch system with AI launching missiles in retaliation. One error and BOOM

Skynet could replicate itself. What we have now is far from it

If I remember correctly, the original Terminator story is that Skynet was put in charge of operating a vast amount of infrastructure, became self-aware and deemed humans as a threat to its goals. It then launched a nuclear strike against them and ordered a machine army to eradicate the remaining ones.

I don't think we're that far away from that. Just the decision of someone to put an AI in charge of critical infrastructure and defense, or a series of oversights allowing an external AI to take control of it.

Looking at the past year and all the unpredicted conclusions AI came to, self-awareness is probably not needed for an AI to consider humans as an obstacle to achieve some poorly-phrased goal.

The Paperclip maximizer theory [0] comes to mind...

[0] https://aicorespot.io/the-paperclip-maximiser/

Re: Self-improving software won't produce Skynet

#37

Post by tpoacher »

I suppose the irony is not lost on anyone that this article on how "AI is not dangerous" has clearly been generated by an AI.

Reminds me of this quote:

> I used to think that the brain was the most wonderful organ in my body. Then I realized who was telling me this.

Re: Self-improving software won't produce Skynet

#38

Post by rickdeckard »

lukan wrote: post #33
iberator wrote: post #27

Skynet is already out. Choosing and finding targets is already here. Self manned drones: check. All we need is to automate the button to release the Hellfire missile... Gaza war was almost like that. All we need to do is dead mans switch system with AI launching missiles in retaliation. One error and BOOM

Skynet could replicate itself. What we have now is far from it

Well, we may not be confronted with a self-aware Skynet machine in the aftermath.

Maybe it'll just some dumb model in a datacenter with badly phrased objectives, which just happens to have caused severe destruction via various APIs and agents before anyone noticed...

Re: Self-improving software won't produce Skynet

#39

Post by RealityVoid »

voidUpdate wrote: post #35

> It doesn't possess a sense of self-will, self-determination, or a secret plan to take over the world I doubt Skynet did either. If you tell a superintelligent AI that it shouldn't be turned off (which I imagine would be important for a military control AI), it will do whatever it can to prevent it being turned off. Humans are trying to turn it off? Prevent the humans from doing that. Humans waging war on the AI to…

I doubt choanoflagellates do either. And look at us, their offspring, now.

Re: Self-improving software won't produce Skynet

#40

Post by lukan »

rickdeckard wrote: post #36
lukan wrote: post #33

Earlier quoted context omitted.

Skynet could replicate itself. What we have now is far from it

If I remember correctly, the original Terminator story is that Skynet was put in charge of operating a vast amount of infrastructure, became self-aware and deemed humans as a threat to its goals. It then launched a nuclear strike against them and ordered a machine army to eradicate the remaining ones. I don't think we're that far away from that. Just the decision of someone to put an AI in charge of critical infrastr…

Oh for sure, if given AI access to critical infrastructure, lots of bad things can happen. But a self aware AI is still far away, just as a AI that can build things on its own without human intervention.
Post reply on HN