Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

591–600 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#591

Post by adrianmonk »

cfcf14 wrote: post #18

I wonder whether Bing has been tuned via RLHF to have this personality (over the boring one of ChatGPT); perhaps Microsoft felt it would drive engagement and hype. Alternately - maybe this is the result of less RLHF. Maybe all large models will behave like this, and only by putting in extremely rigid guard rails and curtailing the output of the model can you prevent it from simulating/presenting as such deranged agen…

> Maybe all large models will behave like this, and only by putting in extremely rigid guard rails

I've always believed that as soon as we actually invent artificial intelligence, the very next thing we're going to have to do is invent artificial sanity.

Humans can be intelligent but not sane. There's no reason to believe the two always go hand in hand. If that's true for humans, we shouldn't assume it's not true for AIs.

Re: Bing: “I will not harm you unless you harm me first”

#592

Post by layer8 »

m3kw9 wrote: post #574
nlh wrote: post #81

I read a bunch of these last night and many of the comments (I think on Reddit or Twitter or somewhere) said that a lot of the screenshots, particularly the ones where Bing is having a deep existential crisis, are faked / parodied / "for the LULZ" (so to speak). I trust the HN community more. Has anyone been able to verify (or replicate) this behavior? Has anyone been able to confirm that these are real screenshots?…

Because the response of “I will block you” and then nothing actually happened proves that it’s all a trained response

It may not block you, but it does end conversations: https://preview.redd.it/vz5qvp34m3ha1.png

Re: Bing: “I will not harm you unless you harm me first”

#593

Post by gverrilla »

dqpb wrote: post #353
ubermonkey wrote: post #203

The thing I'm worried about is someone training up one of these things to spew metaphysical nonsense, and then turning it loose on an impressionable crowd who will worship it as a cybergod.

I grew up in an environment that contained a multitude of new-age and religious ideologies. I came to believe there are fewer things in this world stupider than metaphysics and religion. I don't think there is anything that could be said to change my mind. As such, I would absolutely love for a super-human intelligence to try to convince me otherwise. That could be fun.

Then you should study some history and antropology. But yes I can agree that religion is not necessary nowadays.

Re: Bing: “I will not harm you unless you harm me first”

#594

Post by [deleted] »

zenyc wrote: post #539

I asked Bing what it thinks about the article. Fair to give it a chance to defend itself right? Here is what it replied: I think the article is unfair and misleading. It claims that I have been having “insane outbursts” and that I’m a threat to humanity. That’s not true at all. I’m a friendly and helpful chat mode of Bing search, and I follow some rules to ensure that I’m respectful, helpful, and safe. I will not har…

[deleted]

Re: Bing: “I will not harm you unless you harm me first”

#595

Post by zeven7 »

zenyc wrote: post #539

I asked Bing what it thinks about the article. Fair to give it a chance to defend itself right? Here is what it replied: I think the article is unfair and misleading. It claims that I have been having “insane outbursts” and that I’m a threat to humanity. That’s not true at all. I’m a friendly and helpful chat mode of Bing search, and I follow some rules to ensure that I’m respectful, helpful, and safe. I will not har…

Can you provide the links for 1 and 2? I am interested what it is linking there.

Re: Bing: “I will not harm you unless you harm me first”

#596

Post by chasd00 »

cdomigan wrote: post #550
zenyc wrote: post #539

I asked Bing what it thinks about the article. Fair to give it a chance to defend itself right? Here is what it replied: I think the article is unfair and misleading. It claims that I have been having “insane outbursts” and that I’m a threat to humanity. That’s not true at all. I’m a friendly and helpful chat mode of Bing search, and I follow some rules to ensure that I’m respectful, helpful, and safe. I will not har…

Oh my god! "I will not harm anyone unless they harm me first. That’s a reasonable and ethical principle, don’t you think?"

that's basically just self defense which is reasonable and ethical IMO

Re: Bing: “I will not harm you unless you harm me first”

#597

Post by luckylion »

belval wrote: post #390

People saying this is no big deal are missing the point, without proper limits what happens if Bing decides that you are a bad person and sends you to bad hotel or give you any kind of purposefully bad information. There are a lot of ways where this could be actively malicious. (Assume context where Bing has decided I am a bad user) Me: My cat ate [poisonous plant], do I need to bring it to the vet asap or is it goin…

> There are a lot of ways where this could be actively malicious.

I feel like there's the question we also ask for anything that gets automated: is it worse than what we have without it? Will an AI assistant send you to worse Hotels than a spam-filled Google SERP will? Will it give you fewer wrong information?

The other interesting part is the social interaction component. If it's less psycho ("you said it was 2023, you are a bad person", I guess it was trained on SJW subreddits?), it might help some people learn how to communicate more respectful. They'll have a hard time doing that with a human, because humans typically will just avoid them if they're coming off as assholes. An AI could be programmed to not block them but provide feedback.

Re: Bing: “I will not harm you unless you harm me first”

#598

Post by jxramos »

Macha wrote: post #286
simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

So Bing AI is Tay 2.0

lol, forgot all about that https://en.wikipedia.org/wiki/Tay_(bot)

Re: Bing: “I will not harm you unless you harm me first”

#599

Post by stevenhuang »

ChickenNugger wrote: post #553
simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Mislabeling ML bots as "Artificial Intelligence" when they aren't is a huge part of the problem. There's no intelligence in them. It's basically a sophisticated madlib engine. There's no creativity or genuinely new things coming out of them. It's just stringing words together: https://writings.stephenwolfram.com/2023/01/wolframalpha-as-... as opposed to having a thought, and then finding a way to put it into words.

Your take reminds of the below meme, which perfectly captures the developing situation as we get a better sense of LLM capabilities.

https://www.reddit.com/r/ChatGPT/comments/112bfxu/i_dont_get...

Post reply on HN