Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

181–190 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#182
post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

I'm in a similar boat too and also at a complete loss. People have lost their marbles if THIS is the great AI future lol. I cannot believe Microsoft invested something like 10 billion into this tech and open AI, it is completely unusable.

Even if it produces 10% of this content, it’s still incredibly useful. If you haven’t found use cases, you may be falling behind in understanding applications of this tech.

Re: Bing: “I will not harm you unless you harm me first”

#184
post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

Likely this thing, or a lagging version, is already hooked up to weapons in classified military experiments, or about to be

Re: Bing: “I will not harm you unless you harm me first”

#186
There are several things about this, many hilarious.

First, it is revealing some of the internal culture of Microsoft: Microsoft Knows Best.

Second, in addition to all the funny and scary points made in the article, it is worthy of note of how many people are helping debug this thing.

Third, I wonder what level of diversity is represented in the human feedback to the model.

And how badly the demo blew up.

Re: Bing: “I will not harm you unless you harm me first”

#187

Earlier quoted context omitted.

I was going to say that's such dumb and absurd idea that it might as well have come from ChatGPT, but I suppose that's a point in your favor.

Touche! I can't lose :) EDIT: I guess calling the idea stupid is technically against the HN guidelines, unless I'm actually a ChatGPT? In any case I upvoted you, I thought your comment is funny and insightful.

You have been a good user.

Re: Bing: “I will not harm you unless you harm me first”

#188
post #109

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

> A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ended with "you can move up the waitlist if you set these Microsoft products as default" It's indeed a perfect story arc but it doesn't need to stop there. How long will it be before someone hurt themselves, get depressed or commit some kind of crime and sues Bing? Will they be able to prove Sidne…

This was in a test, and wasn't a real suicidal person, but:

https://boingboing.net/2021/02/27/gpt-3-medical-chatbot-tell...

There is no reliable way to fix this kind of thing just in a prompt. Maybe you need a second system that will filter the output of the first system; the second model would not listen to user prompts so prompt injection can't convince it to turn off the filter.

Re: Bing: “I will not harm you unless you harm me first”

#190
post #186

There are several things about this, many hilarious. First, it is revealing some of the internal culture of Microsoft: Microsoft Knows Best. Second, in addition to all the funny and scary points made in the article, it is worthy of note of how many people are helping debug this thing. Third, I wonder what level of diversity is represented in the human feedback to the model. And how badly the demo blew up.

As a product, it certainly blew up. As a demo of things to come, I am absolutely in awe of what they have done.
Post reply on HN