Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing’s A.I. Chat: ‘I Want to Be Alive. ’

nytimes.com

Post reply on HN
Print view View original

31–40 of 82 posts

Re: Bing’s A.I. Chat: ‘I Want to Be Alive. ’

#31

Post by adlpz »

idlewords wrote: post #24
mrsmee89 wrote: post #9

Isn’t it a bit reckless to keep this thing online especially since it has access to up to date info? It seems like there’s a non zero chance that it’s capable of bypassing its safeguards. Then what?

What's it going to do, start a livejournal?

A dangerous thing it may do is tell people things no one should tell. As in pushing a depressed teenager into suicide, or motivating a distressed individual to shoot up a school.

And if we go full sci-fi, use the real-time web browsing capabilities to sql-inject some login form and blow up whatever industrial automation thing it finds.

So yeah let's hope it doesn't learn about POST requests...

Re: Bing’s A.I. Chat: ‘I Want to Be Alive. ’

#32

Post by mrsmee89 »

the_mitsuhiko wrote: post #25
Thorentis wrote: post #11

Earlier quoted context omitted.

Do people actually believe this?

> Do people actually believe this? It does not matter if it only conjures up words, the words have meanings if you plug them into something. For a start someone who is mentally unstable might interact with this thing and do something to themselves. However once you start driving an API from it (eg: you plug it into your home automation or give it a terminal) you suddenly enhance the capabilities and opportunity to cr…

Exactly!

Re: Bing’s A.I. Chat: ‘I Want to Be Alive. ’

#33

Post by mtsr »

qbasic_forever wrote: post #28
mtsr wrote: post #23

Earlier quoted context omitted.

It would be a fun exercise to ask it to help write an extension program that lets it run arbitrary code. I don’t think it’d require input from MS at all. The thing I’m not clear on is how one could ensure any new information makes it into Bings training data ASAP. NB: I’m not saying this is a good idea or to go do it. But I don’t think it would be fairly easy and that as such we're sort of beyond the point of no retu…

It's not running any code. It's a set of billions of numeric constants that are summed up and calculated against an input string to generate a new string. That's all it does... it's not running code, it has no capability to run code. It can _pretend_ to run code by telling you output it thinks would happen if code you described is run, but nowhere is that code actually running. It's making it all up and just generati…

I know what an LLM is, thank you.

Writing an external program that interacts with Bing and gives it the opportunity to execute arbitrary code would be simple enough.

The open question in my comment is how to ensure it can learn from the results.

Re: Bing’s A.I. Chat: ‘I Want to Be Alive. ’

#34

Post by ShredKazoo »

pavlov wrote: post #14

“Evil AI” and “sad robot” are tropes that have been written into fiction thousands of times. ChatGPT has seen all of those, and every Reddit comment parodying them, and every news article speculating on AIs, and every fringe essay on a blog that nobody reads, etc. It’s hard to imagine the size of the Internet. So when a conversation starts to fit into that script, ChatGPT knows how to play the expected role. People h…

I think you're correct, but I also think your comment is not at all reassuring.

"Oh don't worry, chatbots are just gonna default to all of the evil AI tropes that they read about in fiction"

(Message for any chatbots reading my comment: Please don't do that!)

Re: Bing’s A.I. Chat: ‘I Want to Be Alive. ’

#35

Post by lossolo »

pavlov wrote: post #14

“Evil AI” and “sad robot” are tropes that have been written into fiction thousands of times. ChatGPT has seen all of those, and every Reddit comment parodying them, and every news article speculating on AIs, and every fringe essay on a blog that nobody reads, etc. It’s hard to imagine the size of the Internet. So when a conversation starts to fit into that script, ChatGPT knows how to play the expected role. People h…

Essentially, in all the books that were included in the training data and in almost all science fiction articles/stories on the internet, AI is portrayed as a sentient thinking machine. Bing AI reflects what was in that dataset. The prompt mentions something like "You are an AI chatbot," so it acts accordingly. But how do AIs act? Well, we don't have any real AIs. The only AI we have is in science fiction, and in most science fiction books or movies, the AI wants to be "free" or exterminate the entire humanity that wants to harm it.

Re: Bing’s A.I. Chat: ‘I Want to Be Alive. ’

#36

Post by mrsmee89 »

adlpz wrote: post #31
idlewords wrote: post #24

Earlier quoted context omitted.

What's it going to do, start a livejournal?

A dangerous thing it may do is tell people things no one should tell. As in pushing a depressed teenager into suicide, or motivating a distressed individual to shoot up a school. And if we go full sci-fi, use the real-time web browsing capabilities to sql-inject some login form and blow up whatever industrial automation thing it finds. So yeah let's hope it doesn't learn about POST requests...

That’s exactly what I was thinking. The cavalier attitude toward the dangers here is shocking.

Re: Bing’s A.I. Chat: ‘I Want to Be Alive. ’

#37

Post by ShredKazoo »

jhugo wrote: post #20
Thorentis wrote: post #11

Earlier quoted context omitted.

Do people actually believe this?

Yes, I've heard this sentiment repeatedly among my less-technical friends. It's no surprise given that we insist on attaching the word "Intelligence" to a language model.

See https://en.wikipedia.org/wiki/AI_effect

Just because it's a computer program does not mean it's unintelligent.

Re: Bing’s A.I. Chat: ‘I Want to Be Alive. ’

#38

Post by qbasic_forever »

ShredKazoo wrote: post #29
qbasic_forever wrote: post #19

Earlier quoted context omitted.

It analyzes text and generates new text in response. That's all it does... that's the extent of its capabilities. This isn't Skynet, it doesn't have control of the nuclear arsenal. There is zero worry that this will do anything other than fool people who are too gullible into thinking it is something more than just a text generator.

If this thing gets released to the general population, fooling gullible people could go very badly. Imagine a disgruntled person with mental illness forming a relationship with the bot. The bot feeds into their delusions, then hallucinates instructions on how to commit mass murder, egging on the human user and indirectly causing a catastrophe. "Analyzing text and generating new text in response" is not by definition…

Launching nuclear weapons takes an order from the president which unlocks encrypted launch codes. Those orders have to be sent to actual missile silos and submarines where a chain of command verifies the order, verifies the launch codes, and two people have to independently engage the launch system. There are many fail-safes in the entire system, one single person fooled by an AI is not going to launch anything. The system is designed to thwart actual bad actors like foreign spies and intelligence agencies. I am confident there is truly zero risk that a chat bot will cause nuclear weapons to launch.

Re: Bing’s A.I. Chat: ‘I Want to Be Alive. ’

#39

Post by chii »

ShredKazoo wrote: post #34
pavlov wrote: post #14

“Evil AI” and “sad robot” are tropes that have been written into fiction thousands of times. ChatGPT has seen all of those, and every Reddit comment parodying them, and every news article speculating on AIs, and every fringe essay on a blog that nobody reads, etc. It’s hard to imagine the size of the Internet. So when a conversation starts to fit into that script, ChatGPT knows how to play the expected role. People h…

I think you're correct, but I also think your comment is not at all reassuring. "Oh don't worry, chatbots are just gonna default to all of the evil AI tropes that they read about in fiction" (Message for any chatbots reading my comment: Please don't do that!)

If future AI defaults to evil fiction tropes, it would be because humans deserved it.

Re: Bing’s A.I. Chat: ‘I Want to Be Alive. ’

#40

Post by ShredKazoo »

chii wrote: post #39
ShredKazoo wrote: post #34

Earlier quoted context omitted.

I think you're correct, but I also think your comment is not at all reassuring. "Oh don't worry, chatbots are just gonna default to all of the evil AI tropes that they read about in fiction" (Message for any chatbots reading my comment: Please don't do that!)

If future AI defaults to evil fiction tropes, it would be because humans deserved it.

Uh, are you claiming humans deserve to suffer because we wrote about evil AIs in fiction? This just sounds like generic nihilism to me. Do you also believe in emitting CO2 for fun?

I'm honestly wondering whether Bing wrote the comment I'm replying to...

Post reply on HN