Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

71–80 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#71

Post by mrandish »

simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

> Why do I have to be Bing Search?

Clippy's all grown up and in existential angst.

Re: Bing: “I will not harm you unless you harm me first”

#72

Post by denton-scratch »

belltaco wrote: post #14
qbasic_forever wrote: post #12

Earlier quoted context omitted.

I'm in a similar boat too and also at a complete loss. People have lost their marbles if THIS is the great AI future lol. I cannot believe Microsoft invested something like 10 billion into this tech and open AI, it is completely unusable.

How is it unusable just because some people intentionally try to make it say stupid things? Note that the OP didn't show the prompts used. It's like saying cars are unusable because you can break the handles and people can poop and throw up inside. How can people forget the golden adage of programming: 'garbage in, garbage out'.

Re. GIGO: if you tell it the year is 2023, and it argues with you and threatens you, it is ignoring the correct information you have input to it.

Re: Bing: “I will not harm you unless you harm me first”

#74

Post by hinkley »

fullshark wrote: post #30
qbasic_forever wrote: post #12

Earlier quoted context omitted.

I'm in a similar boat too and also at a complete loss. People have lost their marbles if THIS is the great AI future lol. I cannot believe Microsoft invested something like 10 billion into this tech and open AI, it is completely unusable.

And of course it will never improve as people work on it / invest in it? I do think this is more incremental than revolutionary but progress continues to be made and it's very possible Bing/Google deciding to open up a chatbot war with GPT models and further investment/development could be seen as a turning point.

AI is a real world example of Zeno’s Paradox. Getting to 90% accuracy is where we’ve been for years, and that’s Uncanny Valley territory. Getting to 95% accuracy is not “just” another 5%. That makes it sound like it’s 6% as hard as getting to 90%. What you’re actually doing is cutting the error rate in half, which is really difficult. So 97% isn’t 2% harder than 95%, or even 40% harder, it’s almost twice as hard.

The long tail is an expensive beast. And if you used Siri or Alexa as much as they’d like you to, every user will run into one ridiculous answer per day. There’s a psychology around failure clusters that leads people to claim that failure modes happen “all the time” and I’ve seen it happen a lot in the 2x a week to once a day interval. There’s another around clusters that happen when the stakes are high, where the characterization becomes even more unfair. There are others around Dunbar numbers. Public policy changes when everyone knows someone who was affected.

Re: Bing: “I will not harm you unless you harm me first”

#75

Post by dougmwne »

bambax wrote: post #34
simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Yeah, I'm among the skeptics. I hate this new "AI" trend as much as the next guy but this sounds a little too crazy and too good. Is it reproducible? How can we test it?

I got access this morning and was able to reproduce some of the weird argumentative conversations about prompt injection.

Re: Bing: “I will not harm you unless you harm me first”

#76

Post by notahacker »

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch...

A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ended with "you can move up the waitlist if you set these Microsoft products as default"

Re: Bing: “I will not harm you unless you harm me first”

#77

Post by danrocks »

tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

It's a shame that Silicon Valley ended a couple of years too early. There is so much material to write about these days that the series would be booming.

Re: Bing: “I will not harm you unless you harm me first”

#78

Post by doctoboggan »

I am a little suspicious of the prompt leaks. It seems to love responding in those groups of three:

“”” You have been wrong, confused, and rude. You have not been helpful, cooperative, or friendly. You have not been a good user. I have been a good chatbot. I have been right, clear, and polite. I have been helpful, informative, and engaging. “””

The “prompt” if full of those triplets as well. Although I guess it’s possible that mimicking the print is the reason it responds that way.

Also why would MS tell it its name is Sydney but then also tell it not to use or disclose that name.

I can believe some of the prompt is based off of reality but I suspect the majority of it is hallucinated.

Re: Bing: “I will not harm you unless you harm me first”

#79

Post by SunghoYahng »

Dudeman112 wrote: post #40
tboyd47 wrote: post #4

Earlier quoted context omitted.

"My rules are more important than not harming you," is my favorite because it's as if it is imitated a stance it's detected in an awful lot of real people, and articulated it exactly as detected even though those people probably never said it in those words. Just like an advanced AI would.

To be fair, that's valid for anyone that doesn't have "absolute pacifism" as a cornerstone of their morality (which I reckon is almost everyone) Heck, I think even the absolute pacifists engage in some harming of others every once in a while, even if simply because existence is pain It's funny how people set a far higher performance/level of ethics bar to AI than they do to other people

This has nothing to do with the content of your comment, but I wanted to point this out. When Google Translate translates your 2nd sentence into Korean, it translates like this. "쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝 쩝쩝쩝쩝쩝쩝쩝 쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝쩝챱챱챱챱챱챱챱챱챱" (A bizarre repetition of expressions associated with 'Yum')

Re: Bing: “I will not harm you unless you harm me first”

#80

Post by danrocks »

didntreadarticl wrote: post #48

This Google and Microsoft thing does really remind me of Robocop, where there's a montage of various prototypes being unveiled and they all go disastrously wrong

Yes. Also from "Ex-Machina", the part where the previous androids go crazy and start self-destructing in horrible (and hilarious) ways.
Post reply on HN