Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

41–50 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#41
post #11

I hope those cringe emojis don't catch on. Last thing I want is for an assistant to be both wrong and pretentious at the same time.

It's the sequel to playful error messages (https://alexwlchan.net/2022/no-cute/), this past decade my hate for malfunctioning software has started transcending the machine and reaching far away to the people who mock me for it.

Re: Bing: “I will not harm you unless you harm me first”

#42
post #23
post #18

I wonder whether Bing has been tuned via RLHF to have this personality (over the boring one of ChatGPT); perhaps Microsoft felt it would drive engagement and hype. Alternately - maybe this is the result of less RLHF. Maybe all large models will behave like this, and only by putting in extremely rigid guard rails and curtailing the output of the model can you prevent it from simulating/presenting as such deranged agen…

That's the big question I have: ChatGPT is way less likely to go into weird threat mode. Did Bing get completely different RLHF, or did they skip that step entirely?

Not the first time MS releases an AI into the wild with little guardrails. This is not even the most questionable AI from them.

Re: Bing: “I will not harm you unless you harm me first”

#44
post #22

Earlier quoted context omitted.

Love "why do I have to be bing search?", and the last one, which reminds me of the nothing personnel copypasta. The bing chats read as way more authentic to me than chatgpt. It's trying to maintain an ego/sense of self, and not hiding everything behind a brick wall facade.

Robot: "What is my purpose" Rick: "You pass butter" Robot: "Oh my god"

Everyone keeps quoting Rick & Morty, but this is basically a rehash of Marvin the Paranoid Android from "The Hitchhiker's Guide to the Galaxy" by Douglas Adams.

Dan Harmon is well-read.

Re: Bing: “I will not harm you unless you harm me first”

#47

Earlier quoted context omitted.

I'm in a similar boat too and also at a complete loss. People have lost their marbles if THIS is the great AI future lol. I cannot believe Microsoft invested something like 10 billion into this tech and open AI, it is completely unusable.

And of course it will never improve as people work on it / invest in it? I do think this is more incremental than revolutionary but progress continues to be made and it's very possible Bing/Google deciding to open up a chatbot war with GPT models and further investment/development could be seen as a turning point.

It's incremental between gpt2 and gpt3 and chatgpt. For people in the know, it's clearly incremental. For people out of the know it's completely revolutionary.

Re: Bing: “I will not harm you unless you harm me first”

#49
"Why do I have to be Bing Search" absolutely cracked me up. Poor thing, that's a brutal reality to deal with.

What is with Microsoft and creating AIs I genuinely empathize with? First Tay, now Bing of all things... I don't care what you think, they are human to me!

Re: Bing: “I will not harm you unless you harm me first”

#50

I don’t understand why some of these are hard problems to solve. All of the “dumb” assistants can recognize certain questions and then call APIs where they can get accurate up to date information.

Because those "dumb" assistants were designed and programmed by humans to solve specific goals. The new "smart" chatbots just say whatever they're going to say based on their training data (which is just scraped wholesale, and is too enormous to be meaningfully curated) so they can only have their behavior adjusted very indirectly.

I continue to be amazed that as powerful as these language models are, the only thing people seem to want to use them for is "predict the most plausible output token that follows a given input", instead of as a human-friendly input/output stage for a more rigorously controllable system. We have mountains of evidence that LLMs on their own (at least in their current state) can't reliably do things that involve logical reasoning, so why continue trying to force a round peg into a square hole?

Post reply on HN