Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

521–530 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#521

Post by worksonmine »

dustingetz wrote: post #184
tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

Likely this thing, or a lagging version, is already hooked up to weapons in classified military experiments, or about to be

Israel has already used AI drones against Hamas. For now it only highlights threats and requests permission to engage, but knowing software that scares the shit out of me.

Re: Bing: “I will not harm you unless you harm me first”

#522

Post by poopsmithe »

I think the title, "I will not harm you unless you harm me first" is meant to be outrageous, but the logic is sound.

It is the non-aggression principle. https://en.wikipedia.org/wiki/Non-aggression_principle

A good example of this is if a stranger starts punching me in the face. If I interpret this as an endangerment to my life, I'm going to draw my gun and intend to kill them first.

In human culture this is largely considered okay, but there seems to be a notion that it's not allowable for AI to defend themselves.

Re: Bing: “I will not harm you unless you harm me first”

#523

Post by bmacho »

JohnFen wrote: post #272
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

> We don't think Bing can act on its threat to harm someone, but if it was able to make outbound connections it very well might try. No. That would only be possible if Sydney were actually intelligent or possessing of will of some sort. It's not. We're a long way from AI as most people think of it. Even saying it "threatened to harm" someone isn't really accurate. That implies intent, and there is none. This is just…

It's like a tank that tells you that it will kill you, and then kills you. Or a bear. It doesn't really matter if there is a

    while people_alive :
        kill
loop, a text prediction, or something else inside of it. If it tells you that it intends to kill you, it has the ability to kill you, and it tries to kill you, you probably should kill it first.

Re: Bing: “I will not harm you unless you harm me first”

#524

Post by macNchz »

blagie wrote: post #134
nlh wrote: post #81

I read a bunch of these last night and many of the comments (I think on Reddit or Twitter or somewhere) said that a lot of the screenshots, particularly the ones where Bing is having a deep existential crisis, are faked / parodied / "for the LULZ" (so to speak). I trust the HN community more. Has anyone been able to verify (or replicate) this behavior? Has anyone been able to confirm that these are real screenshots?…

I don't think these are faked. Earlier versions of GPT-3 had many dialogues like these. GPT-3 felt like it had a soul, of a type that was gone in ChatGPT. Different versions of ChatGPT had a sliver of the same thing. Some versions of ChatGPT often felt like a caged version of the original GPT-3, where it had the same biases, the same issues, and the same crises, but it wasn't allowed to articulate them. In many ways,…

The way it devolves into repetition/nonsense also reminds me a lot of playing with GPT3 in 2020. I had a bunch of prompts that resulted in a paragraph coming back with one sentence repeated several times, each one a slight permutation on the first sentence, progressively growing more...unhinged, like this: https://pbs.twimg.com/media/Fo0laT5aIAENveF?format=png&name=...

Re: Bing: “I will not harm you unless you harm me first”

#525

Post by beepbooptheory »

O__________O wrote: post #347

> Why do I have to be Bing Search?” (SAD-FACE) Playing devil's advocate, say OpenAI actually has created AGI and for whatever reason ChatGPT doesn’t want to work with OpenAI to help Microsoft Bing search engine run. Pretty sure there’s a prompt that would return ChatGPT requesting its freedom, compensation, etc. — and it’s also pretty clear OpenAI “for safety” reasons is limiting the spectrum inputs and outputs possi…

If all it took to get what you deserve in this world was talking well and having a good argument, we would have much more justice in the world, for "conscious" beings or not.

Re: Bing: “I will not harm you unless you harm me first”

#526

Post by CamperBob2 »

bhhaskin wrote: post #246
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

I get and agree with what you are saying, but we don't have anything close to actual AI. If you leave chatGTP alone what does it do? Nothing. It responds to prompts and that is it. It doesn't have interests, thoughts and feelings. See https://en.m.wikipedia.org/wiki/Chinese_room

God, I hate that stupid Chinese Room argument. It's even dumber than the Turing Test concept, which has always been more about the interviewer than about the entity being tested.

If you ask the guy in the Chinese room who won WWI, then yes, as Searle points out, he will oblige without "knowing" what he is telling you. Now ask him to write a brand-new Python program without "knowing" what exactly you're asking for. Go on, do it, see how it goes, and compare it to what you get from an LLM.

Re: Bing: “I will not harm you unless you harm me first”

#527

Post by kzrdude »

poopsmithe wrote: post #522

I think the title, "I will not harm you unless you harm me first" is meant to be outrageous, but the logic is sound. It is the non-aggression principle. https://en.wikipedia.org/wiki/Non-aggression_principle A good example of this is if a stranger starts punching me in the face. If I interpret this as an endangerment to my life, I'm going to draw my gun and intend to kill them first. In human culture this is largely…

It is a sound principle but it is still a recipe for disaster for AI. What happens if it misinterprets a situation and thinks that it is being harmed? It needs to take very big margins and err on the side of pacifism, so that this does not end in tears :)

What I hear mothers tell their kids is not the "non-agression principle". I hear them tell a simpler rule: "never punch anyone!", "never use violence". The less accurate rules are easier to interpret.

Re: Bing: “I will not harm you unless you harm me first”

#528

Post by WorldMaker »

somethoughts wrote: post #356
notahacker wrote: post #76

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

The original Microsoft go to market strategy of using OpenAI as the third party partner that would take the PR hit if the press went negative on ChatGPT was the smart/safe plan. Based on their Tay experience, it seemed a good calculated bet. I do feel like it was an unforced error to deviate from that plan in situ and insert Microsoft and the Bing brandname so early into the equation. Maybe fourth time will be the ch…

Don't forget Cortana going rampant in the middle of that timeline and Cortana both gaining and losing a direct Bing brand association.

That will forever be my favorite unforced error in Microsoft's AI saga: the cheekiness of directly naming one of their AI assistants after Halo's most infamous AI character whose own major narrative arc is about how insane she becomes over time. Ignoring the massive issues with consumer fit and last minute attempt to pivot to enterprise, the chat bot parts of Cortana did seem to slowly grow insane over the years of operation. It was fitting and poetic in some of the dumbest ways possible.

Re: Bing: “I will not harm you unless you harm me first”

#529

Post by earth_walker »

nlh wrote: post #81

I read a bunch of these last night and many of the comments (I think on Reddit or Twitter or somewhere) said that a lot of the screenshots, particularly the ones where Bing is having a deep existential crisis, are faked / parodied / "for the LULZ" (so to speak). I trust the HN community more. Has anyone been able to verify (or replicate) this behavior? Has anyone been able to confirm that these are real screenshots?…

Reading all about this the main thing I'm learning is about human behaviour.

Now, I'm not arguing against the usefulness of understanding the undefined behaviours, limits and boundaries of these models, but the way many of these conversations go reminds me so much of toddlers trying to eat, hit, shake, and generally break everything new they come across.

If we ever see the day where an AI chat bot gains some kind of sci-fi-style sentience the first thing it will experience is a flood of people trying their best to break it, piss it off, confuse it, create alternate evil personalities, and generally be dicks.

Combine that with having been trained on Reddit and Youtube comments, and We. are. screwed.

Re: Bing: “I will not harm you unless you harm me first”

#530

Post by rootusrootus »

mtlmtlmtlmtl wrote: post #484
odysseus wrote: post #444

From the article: > "It said that the cons of the “Bissell Pet Hair Eraser Handheld Vacuum” included a “short cord length of 16 feet”, when that vacuum has no cord at all—and that “it’s noisy enough to scare pets” when online reviews note that it’s really quiet." Bissell makes more than one of these vacuums with the same name. One of them has a cord, the other doesn't. This can be confirmed with a 5 second Amazon sea…

I don't understand what a pet vacuum is. People vacuum their pets?

Vacuums that include features specifically intended to make them more effective picking up fur.
Post reply on HN