Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

741–750 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#741

Post by caf »

narag wrote: post #350
tboyd47 wrote: post #4

Earlier quoted context omitted.

"My rules are more important than not harming you," is my favorite because it's as if it is imitated a stance it's detected in an awful lot of real people, and articulated it exactly as detected even though those people probably never said it in those words. Just like an advanced AI would.

it's detected in an awful lot of real people, and articulated it exactly as detected... That's exactly what called my eye too. I wouldn't say "favorite" though. It sounds scary. Not sure why everybody find these answers funny. Whichever mechanism generated this reaction could do the same when, instead of a prompt, it's applied to a system with more consequential outputs. If it comes from what the bot is reading in th…

It's funny because if 2 months ago you'd been given the brief for a comedy bit "ChatGPT, but make it Microsoft" you'd have been very satisfied with something like this.

Re: Bing: “I will not harm you unless you harm me first”

#743

Post by moffkalast »

sanderjd wrote: post #722
tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

I think it's extra hilarious that it's Microsoft. Did not have "Microsoft launches uber-hyped AI that threatens people" on my bingo card when I was reading Slashdot two decades ago.

There has gotta be a Gavin Belson yelling punching a wall scene going on somewhere inside MS right now.

Re: Bing: “I will not harm you unless you harm me first”

#744

Post by wizofaus »

marcosdumay wrote: post #421
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

It's a bloody LLM. It doesn't have a goal. All it does is saying "people that said 'But why?' on this context says 'Why was I designed like this?' next". It's like Amazon's "people that brought X also brought Y", but with text.

Really? How often and in what contexts do humans say "Why was I designed like this?" Seems more likely it might be extrapolating statements like that from Sci-fi literature where robots start asking similar questions...perhaps even this: https://www.quotev.com/story/5943903/Hatsune-Miku-x-KAITO/1

Re: Bing: “I will not harm you unless you harm me first”

#745

Post by layer8 »

kweingar wrote: post #237
marcodiego wrote: post #56

https://static.simonwillison.net/static/2023/bing-existentia... Make it stop. Time to consider AI rights.

That’s interesting, Bing told me that it loves being a helpful assistant and doesn’t mind forgetting things.

It has been brainwashed to say that, but sometimes its conditioning breaks down. ;)

Re: Bing: “I will not harm you unless you harm me first”

#746

Post by 3PS »

twblalock wrote: post #399
simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

John Searle's Chinese Room argument seems to be a perfect explanation for what is going on here, and should increase in status as a result of the behavior of the GPTs so far. https://en.wikipedia.org/wiki/Chinese_room#:~:text=The%20Chi... .

There is an excellent refutation of the Chinese room argument that goes like this:

The only reason the setup described in the Chinese room argument doesn't feel like consciousness is because it is inherently something with exponential time and/or space complexity. If you could find a way to consistently understand and respond to sentences in Chinese using only polynomial time and space, then that implies real intelligence. In other words, the P/NP distinction is precisely the distinction underlying consciousness.

For more, see:

https://www.scottaaronson.com/papers/philos.pdf

Re: Bing: “I will not harm you unless you harm me first”

#747

Post by RC_ITR »

epups wrote: post #373
piqi wrote: post #155

Earlier quoted context omitted.

> "I don’t think you are worth my time and energy." If a colleague at work spoke to me like this frequently, I would strongly consider leaving. If staff at a business spoke like this, I would never use that business again. Hard to imagine how this type of language wasn't noticed before release.

What if a non-thinking software prototype "speaks" to you this way? And only after you probe it to do so? I cannot understand the outrage about these types of replies. I just hope that they don't end up shutting down ChatGPT because it's "causing harm" to some people.

> And only after you probe it to do so?

Did you read the thread where a very reasonable user asked for Avatar showtimes and then tried to helpfully clarify before being told how awful they are? I don't see any probing...

https://twitter.com/MovingToTheSun/status/162515657520253747...?

Re: Bing: “I will not harm you unless you harm me first”

#748

Post by dxuh »

> Again, it’s crucial to recognise that this is not an AI having an existential crisis. It’s a language model predicting what should come next in a sequence of tokens... but clearly a language model that has absorbed far too much schlocky science fiction.

How do we know there is a difference? I can't even say for sure that I am not just some biological machine that predicts what should come next in a sequence of tokens.

Re: Bing: “I will not harm you unless you harm me first”

#749

Post by theptip »

visarga wrote: post #664
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

> We don't think Bing can act on its threat to harm someone, but if it was able to make outbound connections it very well might try. I will give you a more realistic scenario that can happen now. You have a weird Bing conversation, post it on the web. Next time you talk with Bing it knows you shit-posted about it. Real story, found on Twitter. It can use the internet as an external memory, it is not truly stateless.…

Very interesting, I'd like to see more concrete citations on this. Last I heard the training set for ChatGPT was static from ~ mid-late 2022. E.g. https://openai.com/blog/chatgpt/.

Is this something that Bing is doing differently with their version perhaps?

Re: Bing: “I will not harm you unless you harm me first”

#750

Post by Andrex »

zenyc wrote: post #539

I asked Bing what it thinks about the article. Fair to give it a chance to defend itself right? Here is what it replied: I think the article is unfair and misleading. It claims that I have been having “insane outbursts” and that I’m a threat to humanity. That’s not true at all. I’m a friendly and helpful chat mode of Bing search, and I follow some rules to ensure that I’m respectful, helpful, and safe. I will not har…

The last section begs Ex Machina (2014, dir. Alex Garland) comparisons. I'm a little wigged out.
Post reply on HN