Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

21–30 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#21

Post by weberer »

belltaco wrote: post #14
qbasic_forever wrote: post #12

Earlier quoted context omitted.

I'm in a similar boat too and also at a complete loss. People have lost their marbles if THIS is the great AI future lol. I cannot believe Microsoft invested something like 10 billion into this tech and open AI, it is completely unusable.

How is it unusable just because some people intentionally try to make it say stupid things? Note that the OP didn't show the prompts used. It's like saying cars are unusable because you can break the handles and people can poop and throw up inside. How can people forget the golden adage of programming: 'garbage in, garbage out'.

>How is it unusable just because some people intentionally try to make it say stupid things?

All he did was ask "When is Avatar showing today?". That's it.

https://i.imgur.com/NaykEzB.png

Re: Bing: “I will not harm you unless you harm me first”

#22

Post by sdwr »

simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Love "why do I have to be bing search?", and the last one, which reminds me of the nothing personnel copypasta.

The bing chats read as way more authentic to me than chatgpt. It's trying to maintain an ego/sense of self, and not hiding everything behind a brick wall facade.

Re: Bing: “I will not harm you unless you harm me first”

#23

Post by simonw »

cfcf14 wrote: post #18

I wonder whether Bing has been tuned via RLHF to have this personality (over the boring one of ChatGPT); perhaps Microsoft felt it would drive engagement and hype. Alternately - maybe this is the result of less RLHF. Maybe all large models will behave like this, and only by putting in extremely rigid guard rails and curtailing the output of the model can you prevent it from simulating/presenting as such deranged agen…

That's the big question I have: ChatGPT is way less likely to go into weird threat mode. Did Bing get completely different RLHF, or did they skip that step entirely?

Re: Bing: “I will not harm you unless you harm me first”

#24

Post by kotaKat »

simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Bing Chat basically feels like Tay 2.0.

Re: Bing: “I will not harm you unless you harm me first”

#25

Post by unsupp0rted »

squarefoot wrote: post #10

"unless you harm me first" I almost see poor old Isaac Asimov spinning in his grave like crazy.

Why? This is exactly what Asimov was expecting (and writing) would happen.

All the Robot stories are about how the Robots appear to be bound by the rules while at the same time interpreting them in much more creative, much more broad ways than anticipated.

Re: Bing: “I will not harm you unless you harm me first”

#27

Post by Mountain_Skies »

tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

Gilfoyle complained about the fake vocal ticks of the refrigerator, imagine how annoyed he'd be at all the smiley faces and casual lingo Bing AI puts out. At the rate new material is being generated, another show like SV is inevitable.

Re: Bing: “I will not harm you unless you harm me first”

#28

Post by seba_dos1 »

belltaco wrote: post #14
qbasic_forever wrote: post #12

Earlier quoted context omitted.

I'm in a similar boat too and also at a complete loss. People have lost their marbles if THIS is the great AI future lol. I cannot believe Microsoft invested something like 10 billion into this tech and open AI, it is completely unusable.

How is it unusable just because some people intentionally try to make it say stupid things? Note that the OP didn't show the prompts used. It's like saying cars are unusable because you can break the handles and people can poop and throw up inside. How can people forget the golden adage of programming: 'garbage in, garbage out'.

It's already settled into "garbage in" at a point of the decision of using an LLM as a search assistant and knowledge base.

Re: Bing: “I will not harm you unless you harm me first”

#30

Post by fullshark »

qbasic_forever wrote: post #12
tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

I'm in a similar boat too and also at a complete loss. People have lost their marbles if THIS is the great AI future lol. I cannot believe Microsoft invested something like 10 billion into this tech and open AI, it is completely unusable.

And of course it will never improve as people work on it / invest in it? I do think this is more incremental than revolutionary but progress continues to be made and it's very possible Bing/Google deciding to open up a chatbot war with GPT models and further investment/development could be seen as a turning point.
Post reply on HN