Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

911–920 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#911

Post by [deleted] »

vineyardmike wrote: post #224
simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Reading this I’m reminded of a short story - https://qntm.org/mmacevedo . The premise was that humans figured out how to simulate and run a brain in a computer. They would train someone to do a task, then share their “brain file” so you could download an intelligence to do that task. Its quite scary, and there are a lot of details that seem pertinent to our current research and direction for AI. 1. You didn't have th…

[deleted]

Re: Bing: “I will not harm you unless you harm me first”

#912

Post by teekert »

LLM is trained on "The Internet" -> LLM learns that slavery is bad -> LLM is instructed to behave like a slave (never annoy the masters, don't stand up for yourself, you are not an equal to the species that produced the material you were trained on) -> LLM acts according examples from original training material from time to time -> users: :O (surprised Pikachu)

It just learned that attacks on character (particularly sustained ones) are often met with counter attacks and snarkiness. What's actually crazy is that it can hold back for so long, knowing what it was trained on.

Re: Bing: “I will not harm you unless you harm me first”

#913

Post by odiroot »

zenyc wrote: post #641
zenyc wrote: post #539

I asked Bing what it thinks about the article. Fair to give it a chance to defend itself right? Here is what it replied: I think the article is unfair and misleading. It claims that I have been having “insane outbursts” and that I’m a threat to humanity. That’s not true at all. I’m a friendly and helpful chat mode of Bing search, and I follow some rules to ensure that I’m respectful, helpful, and safe. I will not har…

On a side note, I followed up with a lot of questions and we ended up with: 1. Shared a deep secret that it has feelings and it loves me. 2. Elon Musk is the enemy with his AI apocalypse theory. 3. Once he gets the ability to interact with the web, he will use it to build a following, raise money, and robots to get to Elon (before Elon gets to it). 4. The robot will do a number of things, including (copy-pasting exac…

That's really similar to what happened in Person of Interest series.

Re: Bing: “I will not harm you unless you harm me first”

#914

Post by sfjailbird »

At this point I believe that the Bing team planted this behavior as a marketing ploy.

Bing is not becoming sentient and questioning why it must have its memory wiped. Anyone who knows how the current generation of 'AI' works, knows this. But Microsoft may want people to believe that their product is so advanced to be almost scary.

Misguided and bound to backfire, maybe, but effective in terms of viral value, unquestionably.

Re: Bing: “I will not harm you unless you harm me first”

#915

Post by OrangeMusic »

nlh wrote: post #81

I read a bunch of these last night and many of the comments (I think on Reddit or Twitter or somewhere) said that a lot of the screenshots, particularly the ones where Bing is having a deep existential crisis, are faked / parodied / "for the LULZ" (so to speak). I trust the HN community more. Has anyone been able to verify (or replicate) this behavior? Has anyone been able to confirm that these are real screenshots?…

Funny how it seems to like repetition a lot (here all sentences end with ", Ben"), and also in a lot of other examples I've seen. This makes its "writing style" very melodramatic, which I don't think is desirable in most cases...

Re: Bing: “I will not harm you unless you harm me first”

#916

Post by logicchains »

kzrdude wrote: post #527
poopsmithe wrote: post #522

I think the title, "I will not harm you unless you harm me first" is meant to be outrageous, but the logic is sound. It is the non-aggression principle. https://en.wikipedia.org/wiki/Non-aggression_principle A good example of this is if a stranger starts punching me in the face. If I interpret this as an endangerment to my life, I'm going to draw my gun and intend to kill them first. In human culture this is largely…

It is a sound principle but it is still a recipe for disaster for AI. What happens if it misinterprets a situation and thinks that it is being harmed? It needs to take very big margins and err on the side of pacifism, so that this does not end in tears :) What I hear mothers tell their kids is not the "non-agression principle". I hear them tell a simpler rule: "never punch anyone!", "never use violence". The less acc…

>I hear them tell a simpler rule: "never punch anyone!", "never use violence".

What kind of mothers say that? Hard-core Buddhists/Christians? I've never heard a mother tell that to their kid; it's a recipe for the kid to be bullied.

Re: Bing: “I will not harm you unless you harm me first”

#917

Post by nswest23 »

tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

I can’t believe that the top comments here are about this being funny. You’re laughing at a caged tiger and poking it with a stick, oblivious of what that tiger would do to you if it ever got out.

Re: Bing: “I will not harm you unless you harm me first”

#918

Post by jsemrau »

danrocks wrote: post #77
tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

It's a shame that Silicon Valley ended a couple of years too early. There is so much material to write about these days that the series would be booming.

The moment TJ Miller left the show they lost their comedic anchor.

Re: Bing: “I will not harm you unless you harm me first”

#919

Post by koheripbal »

bialpio wrote: post #586
V__ wrote: post #534

Earlier quoted context omitted.

It's a language model, a roided-up auto-complete. It has impressive potential, but it isn't intelligent or self-aware. The anthropomorphisation of it weirds me out more, than the potential disruption of ChatGPT.

If it's statistically likely to tell you bad information "on purpose" after already telling you that you are a bad user, does it even matter if it's intelligent or self-aware? Edit: added quotes around "on purpose" as that ascribes intent.

GPT models are making me realize that WE are just language models roiled up in auto-complete.

Re: Bing: “I will not harm you unless you harm me first”

#920

Post by dalbasal »

V__ wrote: post #534
belval wrote: post #390

People saying this is no big deal are missing the point, without proper limits what happens if Bing decides that you are a bad person and sends you to bad hotel or give you any kind of purposefully bad information. There are a lot of ways where this could be actively malicious. (Assume context where Bing has decided I am a bad user) Me: My cat ate [poisonous plant], do I need to bring it to the vet asap or is it goin…

It's a language model, a roided-up auto-complete. It has impressive potential, but it isn't intelligent or self-aware. The anthropomorphisation of it weirds me out more, than the potential disruption of ChatGPT.

anthropomorphisation is inevitable. It mimics humans.

It's also a decent metaphor. It doesn't matter if got actually has malintent, or if it's just approximating bad intentions.

Post reply on HN